Autoscaling: Pods, nodes and compute classes

Three levels

Pods: HPA and VPA

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: shop
  namespace: shop
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: shop }
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 70 }
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300

• The HPA can also scale on Cloud Monitoring metrics (queue depth, requests per second from the load balancer) through the custom metrics adapter, or on Prometheus metrics from Managed Service for Prometheus.
• The VPA in updateMode: "Off" gives request recommendations without restarting anything: the safest way to rightsize. In Auto it recreates Pods with new requests.
• Don't let the HPA and VPA both act on CPU for the same workload.

Nodes: cluster autoscaler

The cluster autoscaler adds nodes when Pods are unschedulable because of their requests, and removes nodes whose Pods fit elsewhere:

Things that block scale-down: PDBs with no room, Pods with local storage, Pods without a controller, the safe-to-evict annotation, system Pods without PDBs.

Node auto-provisioning and compute classes

With node auto-provisioning (NAP), GKE creates node pools sized for the pending Pods (CPU, memory, GPUs, Spot, zones) within cluster-wide limits, and deletes them when empty. Compute classes describe preferences, for example:

apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
  name: cost-first
spec:
  priorities:
    - machineFamily: n4
      spot: true
    - machineFamily: n4
      spot: false
  nodePoolAutoCreation:
    enabled: true
  whenUnsatisfiable: ScaleUpAnyway

Workloads pick it with nodeSelector: { cloud.google.com/compute-class: cost-first }: Spot first, on-demand when Spot isn't available. The CRD's fields have changed across GKE versions; use the reference for yours.

Autopilot

In Autopilot there are no pools to size. Pods are scheduled, GKE adds the capacity and bills for the Pods' requests. Your levers are requests (rightsize them with VPA recommendations), the compute class (general purpose, Scale-Out, Balanced, Spot, GPU) and the HPA.

How fast is scale-up, really?

Pending Pod → autoscaler decision (seconds) → VM created and joined (often 1–2 minutes) → image pulled (seconds to minutes) → readiness. To shorten it:

• Image streaming and small images.
• Overprovisioning: a low-priority "balloon" Deployment that reserves headroom and gets evicted first, so real Pods start instantly while new nodes come up.
• Scale on leading metrics (queue depth, requests) instead of CPU, which reacts late.

Cost

• In Standard, cost follows nodes: bin-packing, the optimize-utilization profile, Spot pools and rightsized requests.
• In Autopilot, cost follows requests: rightsizing is the main lever.
• Committed use discounts for the steady baseline, Spot for the bursty part.
• Use GKE cost allocation (cost per namespace and label in billing exports) so teams see what they spend.

Recap

• HPA scales replicas, VPA rightsizes requests; don't let both act on CPU.
• The cluster autoscaler reacts to pending Pods, not CPU; NAP and compute classes choose machine shapes.
• Autopilot scales automatically and bills per request.
• Speed up scale-up with image streaming, overprovisioning and leading metrics; control cost with Spot, profiles, commitments and rightsizing.