RBAC design at scale · Prasanta Kumar Behera

Design RBAC for many teams and clusters: personas mapped to roles, aggregated ClusterRoles for custom resources, bindings managed in Git, the permissions that quietly grant admin, and regular access reviews.

RBAC grows messy without a design

Ten clusters × twenty teams × hand-made RoleBindings = nobody can answer "who can read production Secrets?". A small design fixes that: personas → roles → bindings in Git → regular reviews.

In a big school, you don't give each child a custom key ring. You define roles: pupil, class monitor, teacher, caretaker. Each role has a standard ring. Then you just record who has which role, in one register that the head checks every term.

1. Define personas

Persona	Scope	Role
Platform admin	All clusters	`cluster-admin` via an IdP group, ideally just-in-time
SRE on-call	All namespaces	Read everything except Secrets + restart/scale/cordon
Team developer	Own namespaces, non-prod	`edit`
Team developer	Own namespaces, prod	`view` (+ a narrow "restart deployment" role)
Auditor	All clusters	Read-only, including RBAC objects; no Secrets
CI deployer (bot)	Target namespaces	Only the resources its pipeline applies
GitOps controller	Cluster	What it reconciles, and nothing more

2. Reuse and extend built-in roles

Built-in view, edit and admin are aggregated: other ClusterRoles with the right label are merged into them automatically. When a platform add-on installs CRDs, give teams access by labelling a small ClusterRole:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: certificates-edit
  labels:
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
  - apiGroups: ["cert-manager.io"]
    resources: ["certificates", "issuers"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

Now everyone with edit in a namespace can manage cert-manager Certificates there, with no new bindings.

A narrow custom role for production on-call:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: deployment-restarter
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "patch"]      # 'kubectl rollout restart' is a patch

3. Bindings in Git

Keep one folder per cluster (or per environment) with the bindings for each team namespace, applied by GitOps:

rbac/
  prod/
    shop/rolebindings.yaml         # oidc:team-shop → view, deployment-restarter
    payments/rolebindings.yaml
  nonprod/
    shop/rolebindings.yaml         # oidc:team-shop → edit

Every access change is a pull request with an author, a reviewer and a reason. Drift (someone adding a binding by hand) is reverted by the GitOps controller.

4. Watch for escalation paths

RBAC permissions combine in non-obvious ways:

Workload creation (pods, deployments, jobs…) in a namespace ≈ everything that namespace's ServiceAccounts can do.
secrets get/list/watch = every credential in scope. Note list returns full objects.
bind / escalate on roles, and impersonate, allow granting or assuming more power.
nodes/proxy, pods/exec, pods/attach, serviceaccounts/token reach into running systems or mint tokens.
Admission webhooks and CRDs (cluster-scoped) can intercept or change everything.

5. Review regularly

$ kubectl who-can get secrets -n payments
ROLEBINDING               NAMESPACE  SUBJECT                    TYPE   SA-NAMESPACE
payments-admins           payments   oidc:team-payments-admins  Group
CLUSTERROLEBINDING        SUBJECT                  TYPE
oidc-platform-admins      oidc:platform-admins     Group
external-secrets          external-secrets         ServiceAccount  external-secrets

(Output simplified. kubectl who-can is a krew plugin; rbac-tool is another option.) Put the answer to "who can read production Secrets?" on a dashboard, and review it quarterly with each team.

Try it: design and verify

Create namespaces shop and payments, and bindings for groups oidc:team-shop (edit in shop) and oidc:sre (view everywhere + deployment-restarter).
Verify with impersonation: kubectl auth can-i --list -n payments --as=x --as-group=oidc:team-shop (should be almost nothing).
Install cert-manager's CRDs (lesson 02) and the certificates-edit aggregated role; confirm team-shop can now create Certificates in shop.
Give a test group create pods in payments, then show how that group could run a pod as payments' ServiceAccount. Write down why that matters.
Install kubectl who-can and list who can read Secrets in each namespace.

Going deeper: RBAC programmes

Just-in-time elevation: admins normally have view; a request/approval flow (IdP group with time-bound membership) grants cluster-admin for an hour.
Encode RBAC rules as policy tests (e.g. "no binding grants secrets:list to a developer group in prod") that run in CI against the Git repo.
Combine RBAC with admission policy (lesson 10): RBAC controls who may create a pod; policy controls what kind of pod may be created.

Recap

Start from personas, map them to a few roles, and bind IdP groups, not users.
Extend built-ins with aggregated ClusterRoles (labels) instead of new bindings.
Keep bindings in Git, applied by GitOps.
Know the escalation paths (workload creation, secrets, bind/escalate, impersonate, exec) and review access regularly.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.