MTU, TCP MSS and PPPoE: fixing "some websites do not load" problems

MTU and MSS

The MTU (maximum transmission unit) is the largest IP packet a link can carry without fragmentation. For Ethernet it is 1500 bytes. The MSS (maximum segment size) is the largest amount of TCP data in one segment. Hosts announce their MSS during the TCP handshake, normally the MTU minus 40 bytes of IP and TCP headers, so 1460 on Ethernet (see TCP in depth).

When a router must forward a packet bigger than the next link's MTU, it can fragment it, but only if the packet's DF (don't fragment) bit is clear. Most modern hosts set DF and use path MTU discovery: a router that cannot forward the packet drops it and returns an ICMP "fragmentation needed" message giving the smaller MTU, and the host adjusts. The fragment fields are covered in the IPv4 header.

Why PPPoE and tunnels cause trouble

Many DSL and fibre broadband services use PPPoE (Point-to-Point Protocol over Ethernet), which adds 8 bytes, so the IP MTU becomes 1492. Tunnels add more: GRE adds 24 bytes and IPsec adds a variable amount, often 50 to 70 bytes. Hosts on the LAN still believe the MTU is 1500.

The classic symptom is that ping and small pages work but some websites, file uploads or VPN sessions hang. The handshake succeeds, but the first full-size packets are dropped. If a firewall somewhere blocks the ICMP messages, path MTU discovery fails silently, creating a PMTUD black hole. Test with pings of a chosen size and DF set, for example ping 203.0.113.10 size 1500 df-bit on a router, then lower the size until it succeeds.

The fix: ip mtu and TCP MSS clamping

Set the correct IP MTU on the interface, and have the router rewrite the MSS in TCP handshakes passing through it, so hosts never send segments that are too big. This MSS clamping works even when ICMP is blocked, although it only helps TCP. Allow ICMP "fragmentation needed" (IPv4) and "packet too big" (IPv6) through firewalls too.

PPPoE client and server configuration

A typical PPPoE client on a branch router uses a dialer interface:

interface GigabitEthernet0/1
 description To ONT or modem
 no ip address
 pppoe enable group global
 pppoe-client dial-pool-number 1
interface Dialer1
 ip address negotiated
 ip mtu 1492
 ip tcp adjust-mss 1452
 encapsulation ppp
 dialer pool 1
 ppp chap hostname branch1@isp.example
 ppp chap password Example-Secret
ip route 0.0.0.0 0.0.0.0 Dialer1

Providers run PPPoE servers on large broadband gateways, but a Cisco router can play the server role in a lab so you can practise:

bba-group pppoe global
 virtual-template 1
interface GigabitEthernet0/0
 pppoe enable group global
interface Virtual-Template1
 ip unnumbered Loopback0
 ip mtu 1492
 peer default ip address pool PPPOE-POOL
 ppp authentication chap
ip local pool PPPOE-POOL 100.64.10.10 100.64.10.50
username branch1@isp.example password Example-Secret

Check sessions with show pppoe session and show ip interface brief, where the dialer should have an address from the pool.

Real-world example: after a small accountancy firm moved to a fibre service using PPPoE, staff could browse most sites but the tax authority's upload page and the firm's cloud backup kept timing out. A ping with DF set showed that 1500-byte packets failed and 1492-byte packets passed. The firewall in front of the router also dropped ICMP. The IT support company added ip mtu 1492 and ip tcp adjust-mss 1452 to the dialer interface and allowed ICMP "fragmentation needed" inbound. Uploads worked again the same afternoon.