Lesson 05 of 15 · Identity & Access
SSO, federation & MFA
Connect clusters to the enterprise identity provider, enforce MFA where it belongs, and design a break-glass path for when SSO itself is down, without leaving a permanent backdoor.
From one cluster to an organisation
Lesson 04 connected one cluster to one identity provider. In an organisation you have one corporate identity (Entra ID, Okta, Google Workspace…), many clusters, MFA and device policies, and a hard requirement: we must still be able to get in when SSO is down.
The school has one main office that knows every pupil and teacher (the corporate identity provider). Each building (cluster) trusts passes from that office. The office checks your face and your pass code (MFA). And the head teacher keeps one emergency key in a sealed envelope in a safe. If anyone ever opens it, an alarm goes off, and the lock is changed the next day.
Federation patterns
| Pattern | How it works | When |
|---|---|---|
| Direct | Each API server trusts the corporate IdP's issuer | Few clusters, IdP supports it well |
| Broker | Keycloak/Dex trusts the corporate IdP; clusters trust the broker | Many clusters, need consistent groups/claims |
| Cloud-native | EKS/GKE/AKS map cloud IAM identities (which federate to the IdP) | Managed Kubernetes |
| Login proxy | Pinniped or similar exchanges IdP login for cluster credentials | Mixed fleets |
Whatever the pattern, keep group names meaningful and owned by the identity team (k8s-platform-admins, k8s-team-orders-dev), so RBAC stays stable while membership changes in one place.
MFA and conditional access
MFA belongs at the IdP. Prefer phishing-resistant methods (WebAuthn / FIDO2 security keys, platform passkeys) over SMS or OTP for privileged groups. Add IdP policies such as:
- Require a managed device for production admin groups.
- Require re-authentication for privileged access every few hours.
- Block logins from unexpected locations.
Kubernetes sees only the token, but the token only exists if these checks passed.
Break-glass: planned emergency access
SSO fails sometimes: an IdP outage, a certificate expiry, a misconfigured flag. You need a path in that doesn't depend on SSO, without creating a permanent backdoor.
A pragmatic design:
- Separate emergency credentials per cluster, e.g. a client certificate for a dedicated identity (
breakglass-2026q3) bound to cluster-admin, with a modest lifetime. On kubeadm,kubeadm kubeconfig user --client-name=… --org=kubeadm:cluster-adminsgenerates one;super-admin.conf(system:masters) is the last resort. - Store it sealed: in a vault requiring two people to retrieve it, or offline.
- Alert on use: audit-log rule for that username → page the security on-call.
- Rotate after every use (issue a new one, and shorten or retire the old one by waiting out its expiry), and write an incident note.
- Test it quarterly. A break-glass key nobody has tried may not work when needed.
Certificates can't be revoked
A leaked break-glass certificate works until it expires. Keep lifetimes short enough to rotate regularly, store them as carefully as the CA key, and consider rotating the CA if a long-lived one leaks.
What auditors ask (and your answers)
| Question | Good answer |
|---|---|
| Who can access production clusters? | Members of these IdP groups; reviewed quarterly |
| How is MFA enforced? | By IdP policy for all Kubernetes groups; phishing-resistant for admins |
| How fast is access removed? | IdP disable + token lifetime (e.g. ≤ 15 min) |
| What about emergencies? | Sealed break-glass credentials, alerted and rotated after use |
| Where's the evidence? | API server audit logs (who did what) + IdP sign-in logs (who logged in, how) |
Try it: design, then rehearse
- Draw your organisation's identity flow for three clusters: IdP → (broker?) → API servers → RBAC groups.
- List the IdP groups you'd create and which ClusterRoles/Roles each gets.
- On a lab cluster, create a
breakglassclient certificate with a 1-day lifetime (CSR API from lesson 03), bind it to cluster-admin, and store it outside your normal kubeconfig. - Write the break-glass runbook: who may use it, how to retrieve it, what gets alerted, and how to rotate afterwards.
- "Break" your OIDC config (a wrong client ID), confirm OIDC users are locked out, then get in with the break-glass credential and fix it.
Going deeper: identity for platforms
- Give workloads federated identities too (ServiceAccount token → cloud IAM, Vault, or SPIFFE), so no static secrets are needed anywhere.
- Review access regularly with data, not memory: export RoleBindings and IdP group memberships and diff them.
- Separate human admin access from automation access (CI, GitOps controllers), with different identities, scopes and monitoring.
Recap
- One corporate identity, federated directly, via a broker, or via cloud IAM.
- MFA and device/location policies live at the IdP; prefer phishing-resistant methods for admins.
- Break-glass: separate, sealed, alerted, rotated after use, tested regularly.
- Keep audit evidence on both sides: API server audit logs and IdP sign-in logs.
This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.