Health Checks and Kubernetes Probes Visualization

Health checks in Kubernetes prevent traffic from routing to broken containers and automatically restart dead application instances.

Mental Model

Kubernetes manages container health using a three-probe model representing distinct lifecycle phases. A Startup Probe gates all other checks, protecting slow-booting applications from premature restarts. Once the container is initialized, the Readiness Probe determines if it is ready to receive network traffic; if this probe fails (due to startup lag or missing dependencies), the pod is drained and removed from service endpoints. In parallel, the Liveness Probe checks if the container is deadlocked or permanently wedged; if it fails repeatedly, Kubernetes restarts the container to recover.

Components and Why They Exist

Client

The client represents external users sending requests to the application. It generates network traffic and targets the Kubernetes Service endpoint so request particles flow only toward healthy backends.

Service

The Kubernetes Service exposes a logical set of pods to clients while abstracting away individual pod IP addresses. It acts as an internal load balancer, routing incoming traffic exclusively to ready pods listed in the active Endpoints card.

Endpoints

The Endpoints object maintains the dynamic list of healthy, ready Pod IP addresses. It serves as the live routing registry where Pod A is added when readiness checks pass and removed immediately when readiness probes fail.

Kubelet

The kubelet is the local node agent responsible for managing container lifecycles on that machine. It periodically executes configured startup, readiness, and liveness probes against each container, evaluates the status codes, and triggers container restarts when liveness checks fail.

Startup Probe

The startup probe is a gating mechanism for applications that take a long time to boot or warm local caches. It delays liveness and readiness evaluation until the container finishes initializing, transitioning its status badge from gating to passed once boot completes.

Readiness Probe

The readiness probe ensures clients do not hit pods that are starting up, shutting down, or missing crucial dependencies. It queries the container readiness endpoint and, when downstream dependencies fail, marks the pod unready so Kubernetes drains its IP from the active Endpoints list.

Liveness Probe

The liveness probe automatically recovers containers that are deadlocked, stuck in infinite loops, or permanently unresponsive. It checks container responsiveness on each interval, incrementing its failure counter until it crosses the threshold and instructs the runtime to restart the pod.

Pod A / Pod B

Pod A and Pod B are the containerized application instances running your workload. Each pod processes requests routed by the Service, hosts the application runtime, and transitions across initializing, running, crashing, restarting, and recovered states in response to kubelet probes.

Dependency

The dependency represents external systems, such as databases or caches, required for the application to serve requests. It responds to application queries, and when this node goes down, readiness probes fail so broken pods stop receiving client traffic without triggering pointless container restarts.

Restart Manager

The restart manager coordinates container recovery actions when liveness failures or process crashes occur. It tears down failed container processes, increments the restart counter on the pod card, and schedules fresh container boots.

CrashLoopBackOff

CrashLoopBackOff protects the cluster and node from resource starvation during repeated crash cycles. This backoff controller enforces an exponential delay timer before each successive restart attempt, preventing a broken image or config from saturating node CPU.

Edge Cases and Misconceptions

• Misconception: Liveness probe failures are the best way to handle dependency outages: If a database is down, restarting the container will not fix it. Liveness probes should only check internal application state. Dependency outages should fail readiness probes to drain traffic, not liveness probes.
• Misconception: Readiness probes protect applications from boot-up crashes: Readiness probes only manage traffic routing. Startup probes are designed to shield container initialization.
• Edge Case: Cascading Failures due to Sudden Drains: If Pod A’s readiness probe fails, it is removed from the endpoints. The remaining pods must handle all the traffic. If the traffic is too heavy, the other pods might fail, leading to a cascading outage.