Playbooks, challenges & practice · wrap-up
Cheat sheet & self-check
Every command from this section on one page.
Pre-flight
aws sts get-caller-identity | Right account and role? |
aws service-quotas get-service-quota --service-code ec2 --quota-code L-1216C47A | On-Demand standard vCPU quota |
terraform version / kubectl version --client / helm version | Tool versions match the repo's pins |
Per layer (network → cluster → platform)
terraform init -backend-config=backend-prod.hcl | Point at this layer's state |
terraform plan -var-file=prod.tfvars -out=prod.plan | Review: only expected creates |
terraform apply prod.plan | Apply exactly what was reviewed |
Verification gates
aws eks describe-cluster --name prod --query 'cluster.status' | ACTIVE |
aws eks update-kubeconfig --name prod --region eu-west-1 --alias prod | kubectl access |
aws eks list-addons --cluster-name prod / describe-addon … | Add-ons ACTIVE |
kubectl get nodes -o wide / kubectl get pods -A | Nodes Ready, system pods Running |
Everyday changes (through Git + pipeline)
NodePool / node group change in the platform or cluster layer | Add or reshape capacity |
aws_eks_access_entry + aws_eks_access_policy_association | Grant a team access |
aws eks describe-addon-versions --addon-name vpc-cni --kubernetes-version 1.32 | Compatible add-on versions |
Upgrade
pluto detect-helm -o wide / pluto detect-files -d manifests/ | Deprecated APIs before upgrading |
kubernetes_version = "1.33" → plan/apply (cluster layer) | Control plane first |
kubectl get nodes -L karpenter.sh/nodepool -o wide | Watch nodes roll to the new version |
Teardown (reverse order)
kubectl delete ingress,svc -A -l <app selector> (LB-backed) | Let controllers delete ALBs/NLBs |
kubectl delete nodepools --all | Let Karpenter terminate its nodes |
terraform destroy (platform → cluster → network) | Then Terraform |
Laptop
aws sso login --profile prod-readonly | Get short-lived credentials |
aws eks update-kubeconfig --name prod --region eu-west-1 --alias prod --profile prod-readonly | Write the kubeconfig entry |
kubectl config view --minify | See the exec plugin (aws eks get-token) |
kubectl auth whoami | Who does the cluster think I am? |
Pipelines
infra: OIDC role → terraform plan (PR) / apply (main, approved) | Infrastructure changes |
app: OIDC role → docker build → push to ECR (by digest) | Images |
GitOps repo commit → Argo CD sync | Deployments |
Quick checks
terraform plan | grep -E 'must be replaced|forces replacement' | Dangerous replacements |
terraform plan -detailed-exitcode (2 = changes) | Scheduled drift detection |
kubectl get nodeclaims / kubectl -n kube-system logs deploy/karpenter | Why no new nodes? |
kubectl describe pvc <p> / kubectl get pv -o wide | Volume AZ and binding |
kubectl -n kube-system logs deploy/aws-load-balancer-controller | tail | Why no ALB? |
Guard rails
lifecycle { prevent_destroy = true } | On the cluster and state-critical resources |
volumeBindingMode: WaitForFirstConsumer | EBS volumes created in the pod's AZ |
VPC endpoints: ecr.api, ecr.dkr, s3 (gateway), sts | Cut NAT traffic |
State
terraform force-unlock <LOCK_ID> | Release a stale lock (after confirming nothing is running) |
aws s3api list-object-versions --bucket <b> --prefix eks/prod/cluster/terraform.tfstate | Find earlier state versions |
terraform state pull > before-repair.tfstate | Backup before any repair |
import { to = … id = … } then plan/apply | Adopt existing resources |
Network & cluster
aws ec2 describe-network-interfaces --filters Name=vpc-id,Values=<vpc> --query 'NetworkInterfaces[].[NetworkInterfaceId,Description,Status]' | Leftover ENIs blocking destroy |
aws eks list-access-entries --cluster-name prod | Who can get in |
velero restore create --from-backup <name> | Restore app data/objects to a rebuilt cluster |
Terraform without AWS
terraform fmt -check && terraform validate | Syntax and references |
mock_provider "aws" {} in tests/*.tftest.hcl | Fake provider for tests (Terraform 1.7+) |
terraform test | Run plan-based tests with assertions |
Local stand-ins
docker run -d -p 4566:4566 localstack/localstack | LocalStack (check feature coverage for your version) |
backend "s3" { endpoints = { s3 = "http://localhost:4566" } use_path_style = true … } | State backend on LocalStack |
kind create cluster --name eks-sim | Practise the platform layer |
AWS Budgets alert + destroy after each session | When you do use real AWS |
