Autoscaling · Prasanta Kumar Behera

Match capacity to demand automatically: more pods (HPA), right-sized pods (VPA), more nodes (Cluster Autoscaler, Karpenter), and event-driven scaling to zero (KEDA). And how they fit together.

Three things you can scale

What	Tool	Trigger
Number of pods	HorizontalPodAutoscaler (HPA), KEDA	CPU, memory, custom or external metrics, events
Size of pods	VerticalPodAutoscaler (VPA)	Observed usage vs requests
Number of nodes	Cluster Autoscaler, Karpenter	Pods that can't be scheduled; under-used nodes

They work as a chain: load rises → HPA adds pods → pods don't fit → node autoscaler adds nodes → pods schedule. When load drops, the chain runs in reverse.

An ice-cream shop on a hot day. When the queue grows, the manager calls in more servers (HPA). If servers keep running out of scoops, they get bigger scoops (VPA). When there's no counter space left for more servers, the owner opens another counter (node autoscaler). KEDA is the manager who watches the queue outside and sends everyone home when it's empty.

HPA: more pods when busy

The HPA compares current usage with a target and adjusts replicas:

desired replicas = ceil( current replicas × current metric ÷ target )
e.g. 4 pods at 90% CPU, target 60%  →  ceil(4 × 90 ÷ 60) = 6 pods

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60      # percent of the pods' CPU *request*

Two prerequisites trip everyone up: metrics-server must be installed (lesson 23), and the pods must have CPU requests, because utilisation is a percentage of the request.

Try it: watch the HPA react (official example)

This is the Kubernetes documentation's HPA walkthrough, runnable on kind with metrics-server installed:

kubectl apply -f https://k8s.io/examples/application/php-apache.yaml (a small CPU-hungry web app with a CPU request)
kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10
In a second terminal, generate load: kubectl run -i --tty load-generator --rm --image=busybox:1.36 --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://php-apache; done"
Watch kubectl get hpa php-apache -w: utilisation climbs, then replicas increase.
Stop the load (Ctrl+C). Scale-down waits about 5 minutes (the default stabilisation window), so flapping traffic doesn't cause flapping pods.

VPA: right-sized pods

The VPA is an add-on (not built in). It watches real usage and recommends, or applies, better requests:

updateMode	Behaviour
`Off`	Recommendations only. Read them with `kubectl describe vpa`. The safest start
`Initial`	Applies recommendations only when pods are created
`Recreate` / `Auto`	Evicts pods to apply new requests

Start with Off on everything: it's a free right-sizing report.

Node autoscaling

Cluster Autoscaler works with node groups you define (cloud Auto Scaling groups, Cluster API MachineDeployments…). When pods are Pending for lack of resources, it grows the group that would fit them. When nodes are under-used and their pods can move elsewhere, it removes them.

Karpenter skips predefined groups. It looks at the Pending pods' requirements (CPU, memory, architecture, zone, Spot or on-demand) and launches a node that fits, then later consolidates onto fewer, cheaper nodes. It's widely used on AWS EKS (see Amazon EKS in Production).

Scale-down needs your cooperation

Node autoscalers can only remove a node if its pods can be evicted and rescheduled. Pods without a controller, restrictive PodDisruptionBudgets, local storage or "do not evict" annotations keep nodes alive (and the bill running).

KEDA: event-driven, including zero

Queue workers shouldn't scale on CPU; they should scale on work waiting. KEDA connects to event sources (Kafka, RabbitMQ, SQS, Prometheus queries, cron…) and drives replicas between 0 and N:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker
spec:
  scaleTargetRef:
    name: worker            # a Deployment
  minReplicaCount: 0
  maxReplicaCount: 20
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring:9090
        query: sum(queue_messages_waiting{queue="orders"})
        threshold: "100"

(Requires KEDA installed; the trigger fields depend on the scaler type.)

Going deeper: autoscaling that behaves

Tune HPA behavior (scale-up and scale-down policies, stabilisation windows) for spiky traffic. Fast up, slow down is the usual shape.
Pair the HPA with readiness probes and sensible start-up times. Pods that take 2 minutes to warm up make scaling feel broken.
Keep headroom: node provisioning takes minutes. Low-priority "placeholder" pods that get preempted are a common trick to keep spare capacity warm.
Autoscaling is a cost and reliability tool at once. Put max replicas and node limits in place, so a bug or an attack can't scale your bill without bound.

Recap

HPA scales pod count (needs metrics-server and requests); VPA right-sizes requests (start in Off mode).
Cluster Autoscaler / Karpenter add nodes for Pending pods and remove idle ones.
KEDA scales on events, down to zero.
Don't let the HPA and VPA fight over the same metric; keep scale-down safe with PDBs and controllers.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.