Security: nodes, secrets, policy and supply chain

Layer GKE's defences: hardened nodes, encrypted secrets with your own keys, Secret Manager for app secrets, Pod Security and policy at admission, signed images with Binary Authorization, the security posture views, sandboxed Pods for untrusted code, and how security patches reach your nodes.

Layers

Layer	Controls
Organisation and project	Org policies, IAM, VPC Service Controls (lessons 03, 06)
Network	Private nodes, control-plane access, firewall policies, network policy (lessons 04, 05)
Nodes	Shielded nodes, COS, auto-upgrades, minimal node service account, metadata protection
Cluster data	Secrets encryption with Cloud KMS, audit logs
Workloads	Pod Security, policy engine, Workload Identity, resource limits
Supply chain	Artifact Registry scanning, signing, Binary Authorization
Detection	Security posture dashboard, Security Command Center, audit logs

Security is an onion. Each layer stops some attacks, and an attacker who gets through one meets the next. No single layer is perfect, which is exactly why there are several.

Nodes

Shielded GKE nodes: verified boot chain and integrity monitoring (secure boot is a per-pool setting; turn it on unless you load unsigned kernel modules).
Container-Optimized OS: read-only root filesystem, minimal packages, automatic security updates through node upgrades.
Auto-upgrade on: security patches arrive as new node versions within your release channel and maintenance windows.
Metadata protection: the GKE metadata server (Workload Identity) hides the node's credentials and sensitive instance metadata from Pods.
Confidential GKE nodes encrypt memory in use (on supported machine types) for workloads with that requirement.

Secrets

Turn on application-layer secrets encryption with a Cloud KMS key in a separate security project; rotate the key on a schedule.
Prefer Secret Manager for app secrets: the GKE Secret Manager add-on mounts them as files (Secrets Store CSI driver), or the External Secrets Operator syncs them into Kubernetes Secrets. Access is granted per workload through Workload Identity.
Keep plain Kubernetes Secrets out of Git; if they must be in Git, use sealed or encrypted formats.
RBAC: few people and controllers should be able to get or list Secrets.

Workloads: admission

Pod Security Admission: label namespaces pod-security.kubernetes.io/enforce=restricted (or baseline where a workload genuinely needs more); system namespaces stay exempt.
A policy engine for organisation rules: Policy Controller (Google's managed Gatekeeper, with ready-made bundles such as CIS and Pod Security), or Kyverno. Typical rules: images only from your Artifact Registry, required labels and owners, resource requests and limits, no latest tags, no public load balancers outside approved namespaces.
Autopilot already enforces a hardened baseline (no privileged Pods, no host namespaces), which is a good reason to use it where workloads allow.

Supply chain

Artifact Registry scanning finds vulnerable packages on push and keeps re-scanning.
Sign images in CI (Cloud KMS keys, Sigstore cosign) and create attestations after tests and scans pass.
Binary Authorization admits only images with the required attestations (per cluster or project policy), with a break-glass annotation that is logged.
Pin by digest in production manifests (lesson 08).

Seeing your posture

The GKE security posture dashboard reports workload misconfigurations (privileged containers, missing limits, host access) and OS and language-package vulnerabilities in running images.
Security Command Center aggregates findings across projects, including GKE-specific threat detection.
Security bulletins: GKE publishes them per vulnerability, with the patched versions per channel; subscribe to cluster notifications (Pub/Sub) so you know when a fix is rolling out.
Benchmark against the CIS GKE Benchmark (the parts you control: workloads, IAM, network, logging).

Untrusted code

GKE Sandbox runs Pods in gVisor, a user-space kernel, by selecting a RuntimeClass on a sandbox-enabled node pool. Use it for code you don't trust (customer plugins, CI jobs from forks, multi-tenant execution); expect some overhead and check compatibility for syscall-heavy apps.

Try it: block what shouldn't run

Create a KMS key ring and key, grant the GKE service agent encrypter/decrypter on it, and enable secrets encryption on a lab cluster.
Label a namespace restricted and try to run a privileged Pod and a Pod running as root; read the rejections.
Install Policy Controller (or Kyverno) with a rule allowing images only from your Artifact Registry; try nginx from Docker Hub.
Create a Secret Manager secret, grant a workload's principal access, and mount it with the Secret Manager add-on.
Open the security posture dashboard and fix one reported misconfiguration.

Going deeper: making it stick

Put policies in Git and roll them out with GitOps, first in audit/dry-run mode, then enforce once violations are fixed.
Measure: number of policy violations, images older than N days, time to patch after a security bulletin.
Run regular breakout exercises in a test cluster: what can a compromised Pod in each namespace reach?

Recap

Layer defences: organisation → network → nodes → data → workloads → supply chain → detection.
Nodes: Shielded, COS, auto-upgrade, metadata protection.
Secrets: KMS envelope encryption, Secret Manager through Workload Identity, tight RBAC.
Admission: Pod Security plus Policy Controller/Kyverno; Binary Authorization for signed images.
Watch the security posture dashboard and security bulletins; sandbox untrusted code with GKE Sandbox.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.