Autoscaling with the HPA

Traffic is not flat. A results portal is quiet for months and then flooded on result day; a food-ordering app peaks at lunch and dinner. Setting replicas by hand means either paying for idle Pods or falling over at the peak. The HorizontalPodAutoscaler (HPA) watches a metric such as CPU use and changes a Deployment's replica count for you.

In this lesson you will install metrics-server, autoscale the menu Deployment, load it until it scales out, and learn how the HPA decides, how to calm it down, and which other autoscalers exist.

Step 1: metrics-server

The HPA needs current CPU and memory numbers for each Pod. They come from metrics-server, a small add-on that collects them from every kubelet. Install it:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

On kind, the kubelets use self-signed certificates, so metrics-server needs the test-only flag --kubelet-insecure-tls. Add it to the Deployment's arguments (never do this on a real cluster):

kubectl patch deployment metrics-server -n kube-system --type=json \
  -p '[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--kubelet-insecure-tls"}]'

On minikube, minikube addons enable metrics-server does all of this, and managed clouds usually include it. After a minute, kubectl top works:

kubectl top nodes
kubectl top pods -l app=menu

NAME                 CPU(cores)   CPU(%)   MEMORY(bytes)   MEMORY(%)
kind-control-plane   164m         4%       712Mi           9%
kind-worker          61m          1%       388Mi           5%
kind-worker2         58m          1%       402Mi           5%
NAME                    CPU(cores)   MEMORY(bytes)
menu-5c9d7b6f48-2xk7q   3m           6Mi
menu-5c9d7b6f48-8bvtn   2m           6Mi

metrics-server keeps only the latest values in memory. It exists for autoscaling and kubectl top, not for monitoring history; that is Prometheus's job (Logs, Metrics & Prometheus).

Step 2: requests are required

CPU utilization in an HPA means "percent of the container's CPU request". Without requests, the HPA has nothing to compare against and reports <unknown>. Give the menu container a request:

resources:
  requests:
    cpu: 100m
    memory: 64Mi
  limits:
    memory: 128Mi

With cpu: 100m, a Pod using 50m is at 50% utilization.

Step 3: create the HPA

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: menu
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: menu
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50

scaleTargetRef names what to scale, minReplicas and maxReplicas set the range, and the metric says "keep average CPU at about 50% of requests". The quick imperative version is kubectl autoscale deployment menu --cpu-percent=50 --min=2 --max=10.

kubectl apply -f menu-hpa.yaml
kubectl get hpa menu

horizontalpodautoscaler.autoscaling/menu created
NAME   REFERENCE         TARGETS       MINPODS   MAXPODS   REPLICAS   AGE
menu   Deployment/menu   cpu: 4%/50%   2         10        2          60s

TARGETS shows current/target. Remove replicas from the Deployment's YAML once an HPA manages it; otherwise every kubectl apply of the file resets the count and fights the autoscaler.

Step 4: generate load

In a second terminal, start a Pod that requests the page in a tight loop:

kubectl run load --image=busybox:1.37 --restart=Never -- sh -c 'while true; do wget -qO- http://menu > /dev/null; done'

Watch the HPA with kubectl get hpa menu -w. A few minutes later:

NAME   REFERENCE         TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
menu   Deployment/menu   cpu: 182%/50%   2         10        8          3m40s

Stop the load with kubectl delete pod load. CPU drops at once, but the replica count stays up for about five minutes before it comes down. That pause is deliberate, as you will see below.

How the HPA decides

Every 15 seconds the HPA controller computes:

desiredReplicas = ceil( currentReplicas × currentMetric / targetMetric )

With 2 replicas at 182% against a 50% target: 2 × 182 / 50 = 7.28, rounded up to 8. The result is kept between minReplicas and maxReplicas. If the ratio is within a tolerance of 10% of the target (for example 52% against 50%), nothing happens, which stops constant small changes. Since Kubernetes 1.37 that tolerance can be set per HPA, a feature that is now stable.

Behaviour: fast up, slow down

The behavior field controls how quickly the HPA moves. The defaults scale up quickly (doubling, or adding 4 Pods, every 15 seconds, whichever is more) and scale down cautiously: the HPA looks back over a stabilization window of 300 seconds and uses the highest recommendation from that period. That is why the Pods stayed for five minutes after the load stopped: a brief dip must not throw away capacity that will be needed again a moment later. You can tune it:

behavior:
  scaleDown:
    stabilizationWindowSeconds: 600
    policies:
      - type: Pods
        value: 1
        periodSeconds: 60
  scaleUp:
    stabilizationWindowSeconds: 0

This version waits ten minutes and then removes at most one Pod per minute, a sensible choice for apps that are slow to warm up.

Other metrics and other autoscalers

• Memory targets work the same way (name: memory), but many apps never give memory back, so memory-based scaling often scales up and never down. CPU or request rate is usually better.
• Custom and external metrics such as requests per second or queue length need an adapter that serves them to the HPA, for example a Prometheus adapter. KEDA is a popular project that scales on dozens of event sources (queues, Kafka lag, cron schedules) and can even scale to zero.
• The VerticalPodAutoscaler (VPA) is a separate add-on that recommends or sets better requests and limits for each container instead of changing the replica count. Do not let the HPA and VPA both act on CPU for the same Pods.
• Node autoscaling adds and removes machines. When Pods are Pending for lack of room, the Cluster Autoscaler or Karpenter (or a managed mode such as EKS Auto Mode, GKE Autopilot or AKS Automatic) adds nodes. Pod autoscaling and node autoscaling work together: the HPA asks for Pods, the node autoscaler makes room for them.

Common mistakes

• No CPU requests. TARGETS shows <unknown> and nothing scales.
• Keeping replicas in the Deployment file. Each deploy resets what the HPA chose.
• maxReplicas larger than the cluster can hold. Extra Pods stay Pending unless a node autoscaler adds capacity.
• Expecting instant scale-down. The 300-second window is a feature; change behavior only for a good reason.

Key points

• The HPA changes a workload's replica count to keep a metric, usually average CPU utilization, near a target.
• metrics-server supplies the CPU and memory numbers; utilization is measured against each container's request.
• The formula is ceil(currentReplicas × current / target), limited by minReplicas and maxReplicas, with a 10% default tolerance.
• By default the HPA scales up quickly and scales down only after a 300-second stabilization window; behavior tunes both.
• VPA adjusts requests, KEDA scales on events, and the Cluster Autoscaler or Karpenter adds nodes.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: results
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: results
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
    scaleDown:
      policies:
        - type: Percent
          value: 20
          periodSeconds: 60