TCP: state, flags & congestion · Prasanta Kumar Behera

Read TCP like a detective: the handshake and teardown, what each flag and state means, why RST, TIME_WAIT and CLOSE_WAIT show up, and how retransmissions, windows and congestion control shape latency.

The handshake and the goodbye

Client                         Server
  │ ── SYN ──────────────────────► │   "let's talk"          (client: SYN_SENT)
  │ ◄──────────────── SYN, ACK ─── │   "sure, let's talk"    (server: SYN_RECV)
  │ ── ACK ──────────────────────► │   "great"               (both: ESTABLISHED)
  │         … data, ACKs …          │
  │ ── FIN ──────────────────────► │   "I'm done sending"    (client: FIN_WAIT)
  │ ◄───────────────────── ACK ─── │                          (server: CLOSE_WAIT)
  │ ◄───────────────────── FIN ─── │   server app closes      (server: LAST_ACK)
  │ ── ACK ──────────────────────► │                          (client: TIME_WAIT, then gone)

A phone call. SYN: "Hello, can you hear me?" SYN-ACK: "Yes! Can you hear me?" ACK: "Yes!" Then you talk. FIN: "I've finished, bye." The other person says "OK" but might still be finishing a sentence (CLOSE_WAIT) before saying "bye" too. An RST is someone slamming the phone down.

Flags

Flag	Meaning
SYN	Open a connection
ACK	Acknowledge received data
FIN	I've finished sending (graceful close)
RST	Abort now: no listener, rejected, or the connection is unknown
PSH	Push data to the application promptly

States that tell a story

$ ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c | sort -rn
   1840 ESTAB
    912 TIME-WAIT
    301 CLOSE-WAIT
     12 SYN-SENT
      4 LISTEN

State piling up	Usually means
SYN-SENT (client side)	SYNs unanswered: firewall drop, host down, wrong route
SYN-RECV (server side)	Handshakes not completing: SYN flood, or clients unable to reply
CLOSE-WAIT	Your app isn't closing sockets the peer already closed: a leak
TIME-WAIT	Many short connections closed locally: normal, but consider keep-alive or pooling
ESTAB (huge, growing)	Connection pools too large, or clients never disconnect

Retransmissions, windows and congestion

TCP guarantees delivery by retransmitting unacknowledged data. Retransmissions mean loss, and loss means latency (a lost packet can cost hundreds of milliseconds).

$ ss -ti dst 10.0.5.20
ESTAB 0 0 10.10.0.21:51722 10.0.5.20:443
     cubic wscale:7,7 rto:204 rtt:1.2/0.4 mss:1448 cwnd:10 bytes_sent:18291 retrans:0/3 …

rtt: round-trip time; cwnd: congestion window (how much may be in flight); retrans: retransmitted segments.
Receive window: how much the receiver can accept. Slow consumers shrink it, and senders stall.
Congestion control (cubic by default on Linux; bbr is an alternative) decides how fast to send and how to back off after loss.

$ nstat -az | grep -E 'TcpRetransSegs|TcpExtListenOverflows|TcpExtListenDrops'
TcpRetransSegs                  4821               0.0
TcpExtListenOverflows           312                0.0
TcpExtListenDrops               312                0.0

ListenOverflows rising = the application's accept queue is full: it isn't calling accept() fast enough, or the backlog (somaxconn, the app's listen backlog) is too small. Clients see timeouts during bursts.

Try it: read TCP in the wild

sudo tcpdump -ni any port 80 while running curl http://localhost against nginx: identify SYN, SYN-ACK, ACK, data, FIN and ACK.
curl http://localhost:81 (nothing listening): capture the SYN followed by RST.
Start python3 -m http.server 8080, connect with nc localhost 8080, kill the server with Ctrl+C, and check ss -tan on the client side: what state is nc's socket in before you close it?
Run for i in $(seq 1 200); do curl -s localhost > /dev/null; done and count TIME-WAIT sockets afterwards.
Check ss -ti for an SSH connection: note rtt and cwnd.

Going deeper: TCP in Kubernetes

Rolling updates can cut connections: combine readiness probes, a preStop sleep (so endpoints are removed before the pod stops), and graceful shutdown on SIGTERM to drain connections cleanly.
Long-lived connections (gRPC, databases) don't rebalance when you scale up. New pods get no traffic until clients reconnect (lesson 11).
Idle timeouts on load balancers and NAT gateways silently drop idle connections. Use TCP keepalives or application pings shorter than the smallest idle timeout on the path.
Measure retransmissions per node; a sudden rise often points at a bad NIC, cable, switch port or overloaded link.

Recap

Handshake: SYN → SYN-ACK → ACK; graceful close with FIN; abort with RST.
States tell stories: CLOSE-WAIT = app leak, SYN-SENT = no answer, TIME-WAIT = many short connections.
Retransmissions mean loss and latency; ss -ti and nstat show them.
Accept-queue overflows mean the app can't keep up with incoming connections.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.