Lesson 09 of 18 · Operate
Upgrades & add-ons
Upgrade EKS safely and routinely: check upgrade insights, upgrade the control plane one minor at a time, then add-ons, then nodes (managed node groups or Karpenter drift), and know when a blue/green cluster is the better path.
A routine, not an event
EKS versions leave standard support roughly 14 months after release (lesson 01), so plan an upgrade every few months. Made routine and automated, it's boring. Postponed, it becomes a risky multi-version project.
Upgrading a cluster is like servicing a bus fleet while the buses keep running. First you update the control tower's manuals (control plane), then the shared tools every bus uses (add-ons), then you swap buses one at a time for newer ones (nodes), always keeping enough on the road.
1. Prepare
- Upgrade insights (
aws eks list-insights) flag deprecated API usage, add-on and kubelet compatibility issues for the next version. - Read the Kubernetes and EKS release notes (removed APIs, changed defaults).
- Check third-party add-ons (Karpenter, LBC, cert-manager, CNI plugins, service mesh) for compatibility with the target version, and upgrade them first if needed.
- Check PDBs and that every workload has at least 2 replicas where it matters.
- Rehearse on a non-production cluster first (upgrade waves; see Kubernetes Administration, lesson 25).
2. Control plane
In Terraform, bump the version and apply (one minor version at a time):
module "eks" {
source = "terraform-aws-modules/eks/aws"
cluster_version = "1.33" # was 1.32
# …
}
(Input names differ between major versions of the community module; newer releases call this input kubernetes_version. Use the name from the version you pin.) AWS upgrades the API servers and etcd with no API downtime for most clusters (clients may see brief connection resets; they retry).
3. Add-ons
Upgrade EKS managed add-ons to versions compatible with the new Kubernetes version (in Terraform, set addon_version or most_recent = true per add-on):
| Add-on | Watch out for |
|---|---|
| VPC CNI | Upgrade one minor version of the CNI at a time; check its release notes |
| CoreDNS | Replicas and PDB; brief DNS blips if under-provisioned |
| kube-proxy | Must match the cluster's minor version closely |
| EBS/EFS CSI, Pod Identity agent | Compatibility tables in the docs |
Then your Helm-installed controllers (LBC, Karpenter, cert-manager…), via GitOps.
4. Nodes
| Node type | How it rolls |
|---|---|
| Managed node groups | Update the version/AMI; EKS replaces nodes with a surge, draining each (respects PDBs) |
| Karpenter | Drift: change the EC2NodeClass AMI selection (e.g. to the new version's AMI); nodes are replaced within disruption budgets |
| Fargate | Recreate pods (e.g. rollout restart) to get new platform versions |
Watch pods moving, and stop if error rates rise.
Blue/green clusters
For risky changes (several versions behind, CNI swap, major redesign), build a new cluster from code at the target version, deploy everything via GitOps, move traffic gradually (weighted DNS or load balancer), then delete the old one. It costs more and needs stateful-data planning, but rollback is just moving traffic back.
Try it: an upgrade rehearsal (sandbox account)
- Create a cluster one minor version behind the latest available, with a managed node group and the core add-ons, from Terraform.
- Deploy a sample app with 3 replicas, a PDB, and a synthetic check (a loop curling it and logging failures).
- Run
aws eks list-insightsand read the findings. - Upgrade the control plane in Terraform, then the add-ons, then the node group; keep the synthetic check running and count failures.
- Write down the time each step took. That's your baseline for production planning.
Going deeper: upgrade automation
- Put the whole sequence in a pipeline with gates: insights clean → control plane → add-ons → nodes → synthetic checks, one environment wave at a time.
- Pin add-on and AMI versions explicitly in production, and bump them via pull requests (Renovate can help), so every change is reviewed.
- Track time behind latest per cluster as a platform metric.
- Don't forget client tools: kubectl, Helm and Terraform providers used by CI must support the new version.
Recap
- Upgrade regularly; one minor version at a time.
- Prepare (insights, release notes, third-party add-ons, PDBs) → control plane → add-ons → nodes.
- Managed node groups roll with surge; Karpenter rolls via drift.
- Blue/green clusters for risky jumps, when easy rollback is worth the cost.
This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.