Prasanta Kumar Behera
Networking Deep Dive›09 · TLS: handshake & trust
Learning Hub / Kubernetes & Platform / Networking Deep Dive

Lesson 09 of 12 · Application Layer

TLS: handshake & trust

Understand and debug TLS: what happens in a TLS 1.3 handshake, how certificate chains and trust stores work, what SNI and ALPN do, and how to diagnose the errors you'll actually see.

Advanced
Key wordsTLS 1.3handshakecertificate chaintrust storeSNIALPNmTLSopenssl s_client

What TLS gives you

  1. Confidentiality: nobody on the path can read the traffic.
  2. Integrity: nobody can change it unnoticed.
  3. Authentication: the server (and with mTLS, the client) proves its identity with a certificate that chains to a CA the other side trusts.

Sending a secret note in class: TLS puts it in a locked box (encryption), with a seal that breaks if anyone opens it (integrity), and the box carries a stamp from the head teacher proving who sent it (the certificate). You only trust stamps from head teachers you know (the trust store).

The TLS 1.3 handshake, simplified

Client                                         Server
ClientHello  ─────────────────────────────────►   (supported versions/ciphers, key share,
  SNI: shop.example.com, ALPN: h2, http/1.1)
             ◄─────────────────────────────────  ServerHello (key share) … encrypted from here:
                                                  Certificate chain, CertificateVerify, Finished
             (client verifies: chain → trusted CA, name matches SNI, dates valid)
Finished     ─────────────────────────────────►
Application data (HTTP/2 because ALPN chose h2) ◄──────────►

One round trip before data can flow (TLS 1.2 needed two). Session resumption can make repeat connections even faster.

Chains and trust stores

A server presents a chain: its leaf certificate, then intermediate(s). The client builds a path from the leaf to a root CA in its trust store (the OS bundle, or a bundle you configure).

$ openssl s_client -connect shop.example.com:443 -servername shop.example.com -showcerts </dev/null 2>/dev/null | grep -E ' s:| i:'
 0 s:CN = shop.example.com
   i:C = US, O = Let's Encrypt, CN = R11
 1 s:C = US, O = Let's Encrypt, CN = R11
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1

(Intermediate names change over time.) Certificate 0 is the leaf, certificate 1 the intermediate; the root comes from the client's trust store.

Errors and what they mean

Error (wording varies by client) Meaning Check
certificate has expired Leaf or intermediate past notAfter Dates; renewal automation (cert-manager)
unable to get local issuer certificate Incomplete chain, or unknown private CA Server sends intermediates? Client trusts your CA?
certificate is valid for X, not Y / hostname mismatch Name not in SANs SANs vs the name the client used
self-signed certificate in certificate chain Private CA not trusted by the client Distribute the CA bundle (trust-manager, image, ConfigMap)
handshake failure / protocol version No common TLS version or cipher Old clients vs strict servers
certificate required / bad certificate mTLS: missing or rejected client certificate Client cert, key, and the CA the server trusts

SNI and ALPN

  • SNI: the client names the host in ClientHello; the ingress picks the matching certificate. Testing by IP without -servername/--resolve often returns a default certificate, a common false alarm.
  • ALPN: client and server agree on h2 or http/1.1 (and h3 uses QUIC instead, see the next lesson). gRPC requires h2 end to end.
$ openssl s_client -connect shop.example.com:443 -servername shop.example.com -alpn h2,http/1.1 </dev/null 2>/dev/null | grep ALPN
ALPN protocol: h2

mTLS in one paragraph

With mutual TLS, the server also asks for a client certificate and verifies it against a CA it trusts. Service meshes use mTLS between every pair of services, with short-lived certificates tied to workload identity. For manual tests:

$ curl --cacert ca.crt --cert client.crt --key client.key https://api.internal:8443/health

Try it: break TLS four ways

Using cert-manager's private CA from Kubernetes Security, lesson 02, and an ingress or gateway:

  1. curl https://shop.example.com (via --resolve) without --cacert: read the error. Add --cacert with your CA and retry.
  2. Request the host by a name not in the SANs and read the mismatch error.
  3. Issue a certificate with duration: 1h, wait for it (or use an already-expired test cert) and observe the expiry error.
  4. Configure a server that sends only the leaf (e.g. openssl s_server with just the leaf) and see unable to get local issuer certificate.
  5. Check which ALPN protocol your ingress negotiates.

Going deeper: TLS operations

  • Monitor certificate expiry from the outside (blackbox probes) as well as from cert-manager. Both can fail independently.
  • Keep private keys in Secrets with tight RBAC, or in HSM/KMS-backed setups for high-value endpoints.
  • Internal clients often lack your private CA: bake it into base images or mount a CA bundle (trust-manager), rather than turning verification off.
  • Where TLS terminates (edge, ingress, pod, mesh sidecar) is an architecture decision, and each hop is a place to decrypt and re-encrypt, log and debug.

Recap

  • TLS = confidentiality, integrity, authentication; TLS 1.3 needs one round trip.
  • Servers must send the full chain; clients need the root in their trust store; names must match SANs.
  • SNI picks the certificate; ALPN picks h2 vs http/1.1.
  • Debug with openssl s_client -servername … -showcerts and curl -v; mTLS adds client certificates.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.