TCP in depth: header, handshake, windowing, congestion control and resets

What TCP adds to IP

IP delivers packets on a best-effort basis: they may be lost, duplicated or arrive out of order. The Transmission Control Protocol (TCP) fixes that for applications such as web, email, SSH and file transfer. It is connection-oriented, numbers every byte it sends, and keeps resending until the other side confirms receipt.

The header and its flags

The TCP header is 20 bytes plus options. The important fields are:

• Source and destination ports (16 bits each): identify the applications, for example 443 for HTTPS.
• Sequence number: the position of the first data byte in this segment.
• Acknowledgement number: the next byte the receiver expects, which confirms everything before it.
• Flags: SYN (start), ACK (acknowledgement field is valid), FIN (finished sending), RST (abort), PSH (deliver now), URG, plus ECE and CWR for congestion signalling.
• Window: how many more bytes the receiver can accept right now.
• Options: MSS (maximum segment size), window scaling, selective acknowledgement (SACK) and timestamps, mostly agreed during the handshake.

Opening and closing a connection

In the three-way handshake each side picks a random starting sequence number and acknowledges the other's. Random numbers make it hard for an attacker to inject fake segments. After the handshake both sides know each other's MSS and whether window scaling and SACK are in use.

To close gracefully, each direction is shut separately with a FIN that the other side acknowledges. A RST ends a connection immediately. You will see RSTs when a client connects to a port with no listening service, when a firewall rejects (rather than silently drops) a connection, or when an application crashes.

Flow control and congestion control

Flow control protects the receiver. The window field says how much buffer space is free; the sender may have only that many unacknowledged bytes in flight. The original 16-bit field caps the window at 65,535 bytes, so the window scale option multiplies it, allowing windows up to about 1 GB for fast, long-distance links.

Congestion control protects the network. The sender keeps its own congestion window, starts small (slow start), grows it while acknowledgements arrive, and cuts it when it detects loss. Current systems use algorithms such as CUBIC by default, and some use BBR, which estimates bandwidth and delay directly.

Throughput is limited by window size divided by round-trip time. With a 64 KB window and 100 ms of latency, one connection cannot exceed roughly 5 Mbit/s however fast the link is. That is why window scaling matters on WAN links.

Reading TCP problems

Real-world example: backups between two data centres 80 ms apart ran at a fraction of the 1 Gbit/s link speed. A capture showed the backup appliance advertising a fixed 64 KB window without scaling. Enabling window scaling on the appliance raised throughput many times over without any change to the WAN.

Security note: a SYN flood sends many SYNs and never completes the handshake, filling the server's queue of half-open connections. SYN cookies and firewall SYN protection defend against it.