Prasanta Kumar Behera
Networking Deep Dive›06 · Services, kube-proxy & eBPF
Learning Hub / Kubernetes & Platform / Networking Deep Dive

Lesson 06 of 12 · Kubernetes Dataplane

Services, kube-proxy & eBPF

How a ClusterIP actually works on every node: kube-proxy's iptables, IPVS and nftables modes, eBPF replacements like Cilium, and the Service settings that change traffic paths and preserve client IPs.

Advanced
Key wordskube-proxyiptables modeIPVSnftables modeeBPFkube-proxy replacementexternalTrafficPolicysession affinity
client pod curl http://web CoreDNS web → 10.96.12.7 Service web ClusterIP 10.96.12.7:80 selector app=web kube-proxy / eBPF rules on every node EndpointSlice ready pod IPs pod web-1 10.244.1.5:8080 pod web-2 10.244.2.5:8080 pod web-3 10.244.3.5:8080 NodePort node:3xxxx LoadBalancer external IP Ingress HTTP host/path ways in from outside
Service → kube-proxy/eBPF rules → ready pod endpoints.

One API, several dataplanes

A Service is just an API object: a virtual IP plus a selector. Something on every node must turn it into packet forwarding. That's kube-proxy (in one of its modes), or an eBPF dataplane that replaces it.

A Service is a phone number for a team. How the call gets to a team member depends on the office's phone system: an old switchboard operator with a long paper list (iptables), a computerised exchange (IPVS or nftables), or smart phones that already know where to route before the call even leaves the desk (eBPF). The number never changes; only the machinery does.

The modes compared

Mode How it works At scale
iptables NAT chains per Service/endpoint; random probability rules Long chains; slower updates with many Services
IPVS Kernel L4 load balancer with hash tables; several balancing algorithms Faster lookups; still uses iptables for some parts
nftables Newer kube-proxy mode using nftables sets and maps Designed to scale better than iptables; the modern default direction
eBPF (e.g. Cilium) Load balancing in eBPF, often at socket connect time No per-Service chains; excellent visibility 
$ curl -s localhost:10249/proxyMode        # on a node running kube-proxy
iptables
$ sudo ipvsadm -Ln                           # if in IPVS mode
TCP  10.96.142.17:80 rr
  -> 10.244.1.7:80   Masq  1  0  0
  -> 10.244.2.4:80   Masq  1  0  0

Paths through a Service

ClusterIP (inside the cluster): the client's node picks a backend and DNATs to it (lesson 02).

NodePort / LoadBalancer, with the two traffic policies:

externalTrafficPolicy Path Client IP seen by the pod Balance
Cluster (default) Any node → possibly another node's pod (extra hop, SNAT) Node IP (lost) Even across all pods
Local Only to pods on the receiving node; nodes without pods fail the LB health check Preserved Depends on pods per node

internalTrafficPolicy: Local does the same for in-cluster traffic (e.g. talk to the node-local log agent).

Other Service settings that change behaviour:

  • sessionAffinity: ClientIP: stick a client to one pod (see Kubernetes Administration, lesson 08).
  • Topology-aware routing (trafficDistribution: PreferClose in newer versions): prefer endpoints in the same zone to cut cross-zone cost and latency.
  • Headless (clusterIP: None): no load balancing at all; DNS returns pod IPs (lesson 07).

eBPF: the replacement

With Cilium's kube-proxy replacement, Services are handled by eBPF programs. For pod-to-Service traffic, the translation can happen when the socket connects (the pod connects straight to a backend), so there's no per-packet NAT and no conntrack entry for the ClusterIP. It also gives consistent handling for NodePort, LoadBalancer and externalTrafficPolicy, plus flow visibility with Hubble.

$ cilium service list | head -4
ID   Frontend              Service Type   Backend
1    10.96.0.1:443         ClusterIP      1 => 172.18.0.2:6443
2    10.96.142.17:80       ClusterIP      1 => 10.244.1.7:80
                                          2 => 10.244.2.4:80

Try it: the same Service, three ways

  1. In your kind cluster, check the proxy mode and find the web Service's rules on a node (iptables or nftables).
  2. Recreate the cluster with networking.kubeProxyMode: ipvs in the kind config, and inspect the Service with ipvsadm -Ln (install ipvsadm in the node container if needed).
  3. Create a Cilium cluster with kube-proxy replacement enabled (kind with kubeProxyMode: none, then cilium install with kube-proxy replacement), and run cilium service list.
  4. Expose web as NodePort with externalTrafficPolicy: Local, curl each node's NodePort, and note which nodes answer.

Going deeper: Service dataplane operations

  • Endpoint changes propagate asynchronously to every node. During rollouts, readiness probes and a short preStop delay avoid sending traffic to terminating pods.
  • Large endpoint sets rely on EndpointSlices (100 endpoints per slice by default), which keeps updates cheap.
  • Health-check externalTrafficPolicy: Local Services correctly (healthCheckNodePort), or cloud LBs will send traffic to nodes without pods.
  • When migrating dataplanes (iptables → nftables or eBPF), leftover rules can conflict. Follow the CNI's migration guide and drain nodes one at a time.

Recap

  • Services are virtual IPs implemented per node by kube-proxy (iptables, IPVS, nftables) or an eBPF replacement.
  • iptables scales worst; IPVS, nftables and eBPF handle thousands of Services better.
  • externalTrafficPolicy: Local preserves client IPs at the cost of balance; Cluster balances but SNATs.
  • Topology-aware routing, session affinity and headless Services change traffic paths deliberately.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.