Auto-Scaling (HPA) Simulator

Mental Model

Auto-scaling is the process of adjusting computing resources dynamically based on actual demand. In Kubernetes, the Horizontal Pod Autoscaler (HPA) monitors resource metrics (typically CPU utilization) and adjusts the replica count of a deployment to keep average resource utilization near a target threshold.

The core mental model is that auto-scaling is a feedback loop: external traffic increases CPU usage, the metrics server collects CPU samples, the HPA controller evaluates those samples against the target threshold, and the deployment controller adjusts the replica count. A startup delay means new replicas are not ready immediately, and cooldowns prevent the system from overreacting to noisy metric spikes.

Components and Why They Exist

Traffic Source

The traffic source generates realistic, fluctuating workloads that simulate real-world usage patterns, such as gradual morning ramps or sudden Black Friday spikes. It emits request messages to the ingress path, representing user or client requests hitting the application. Dragging the incoming traffic rate slider immediately changes the stream of request dots entering the service.

Application Service

The application service acts as the load balancer and routing layer, representing the Kubernetes Service that distributes traffic to running pods. It receives incoming requests and distributes them using a round-robin routing strategy across all active, healthy pods. Request dots fan out evenly across ready pods, while overload conditions cause excess requests to drop.

Metrics Server

Because HPA cannot make scaling decisions without cluster visibility, the metrics server aggregates utilization statistics across running replicas. It samples CPU and memory metrics from running pods and calculates the average utilization across the active replica set. The scraped average CPU utilization percentage updates on each polling tick as traffic fluctuates.

HPA Controller

The HPA controller implements the control loop that evaluates scraped metrics and decides whether to scale up, scale down, or stay stable. It computes the desired replica count using the ratio formula, checks cooldown constraints, and issues scaling commands to the deployment controller. Its panel displays the target replica calculation alongside the active scale-up and scale-down cooldown timers.

Deployment Controller

The deployment controller executes scaling decisions by managing the lifecycle of individual pod replicas. It creates new pods (which start in a pending state) or terminates existing pods to match the desired replica count. Pod cards on the canvas shift through Pending, Ready, Running, and Terminating states as the controller reconciles the replica pool.

Application Pods

The application pods execute the service workload and absorb incoming client requests once their containers pass readiness checks. Each pod consumes CPU and memory proportional to its assigned request load and transitions from Pending to Ready after the startup delay expires. Newly launched pods count down their boot timer before joining the active load-balancing rotation.

Edge Cases and Misconceptions

• Scaling is not instantaneous: New pods take time to initialize, pull container images, and pass readiness probes. During this startup delay, the existing pods must absorb the load, which can lead to temporary performance degradation or dropped requests.
• CPU is not the only metric: Scaling purely on CPU means the autoscaler will miss memory exhaustion issues (like memory leaks), queue backups, or downstream database bottlenecks.
• Aggressive scale-down is dangerous: Restoring capacity is expensive. Scaling down too quickly during a temporary dip in traffic can leave the system vulnerable to the next traffic wave, causing thrashing.
• Autoscaling has limits: Min and max replica boundaries protect against unbounded cost or resource exhaustion, but a max limit that is set too low will cause request drops under extreme traffic.
• Failed pods consume capacity: Pods that are crashing or restarting still consume cluster management resources and count toward the deployment’s replicas, even though they cannot serve traffic.