Almost every pod failure follows the same investigation sequence. Learn this once and most of the scenarios below are just "which step told you the answer."
CrashLoopBackOff
The container starts, exits, and Kubernetes keeps restarting it with increasing backoff. This is an application or config problem, not a scheduling problem — the pod did start.
Check the restart count and last state
kubectl get podsA rising RESTARTS count alongside CrashLoopBackOff status confirms the loop.
Read the logs from the previous crashed run
kubectl logs POD_NAME --previousThe current container has no logs yet if it just restarted — --previous shows what the last attempt actually printed before dying.
Check exit code and reason in describe
kubectl describe pod POD_NAMELook at the container's Last State: block — Exit Code 1 usually means an application error, 137 means it was SIGKILLed (often OOMKilled — see below), 0 means it exited cleanly, which usually points at a misconfigured command or entrypoint.
Common root causes
A missing environment variable or config file the app expects at startup, a failing dependency health check gating app boot, a wrong command/entrypoint override, or an unhandled exception in application startup code.
ImagePullBackOff
Kubernetes cannot pull the container image. The pod never actually starts.
Confirm the exact image and tag
kubectl describe pod POD_NAMECheck the Events section for the precise pull error — typo'd image name, missing tag, or an auth failure against a private registry all show distinct messages here.
Check for a private registry auth problem
If the event says unauthorized or pull access denied, the pod's imagePullSecrets is missing, wrong, or the
referenced Secret doesn't exist in this namespace — image pull secrets are namespace-scoped.
kubectl get secret REGISTRY_SECRET -n NAMESPACEConfirms the pull secret actually exists where the pod expects it.
Check the tag actually exists
ErrImagePull for a tag that was never pushed, or was deleted by a registry retention policy, is common right after
a botched CI release — confirm the tag exists in the registry directly.
Pending Pods
The pod is accepted but never gets scheduled onto a node.
Read the scheduling failure reason
kubectl describe pod POD_NAMEThe Events section names the exact reason: Insufficient cpu/memory, no nodes matching a nodeSelector or taint toleration, or a PVC that hasn't bound yet.
Check cluster capacity
kubectl describe nodes | grep -A5 AllocatedShows how much of each node's capacity is already requested — a Pending pod with Insufficient cpu/memory means the cluster (or that pod's allowed nodes) is genuinely full.
Check node selectors, taints and affinity
A pod requesting a node label nothing has, or lacking a toleration for every node's taint, will stay Pending indefinitely with no capacity problem at all — the scheduler simply has zero eligible nodes.
Readiness Probe Failures
The pod is Running but never becomes Ready, so it's excluded from Service endpoints — traffic never reaches it, but it isn't restarted either.
kubectl describe pod POD_NAMECheck the probe's configured path/port against events like Readiness probe failed: HTTP probe failed with statuscode: 503 — that tells you exactly what the probe is calling and what it got back.
Readiness vs. liveness
A failing readiness probe pulls the pod out of Service rotation without restarting it — correct for "temporarily can't take traffic" (still warming a cache, a dependency is briefly down). A failing liveness probe kills and restarts the container — correct for "this process is stuck and a restart is the only fix." Using the wrong one either restart-loops a pod that just needs a moment, or leaves a truly wedged process serving traffic forever.
Liveness Probe Failures
The pod restarts repeatedly because its liveness probe keeps failing, even though the process hasn't crashed.
kubectl describe pod POD_NAMECompare the probe's initialDelaySeconds and timeoutSeconds against how long the app actually takes to become responsive — a probe firing before the app is ready to answer is the most common cause.
Tip
If logs show the app is healthy right up until a SIGTERM, the liveness probe timeout is very likely too aggressive for a slow endpoint, not an actual application bug.
OOMKilled
kubectl describe pod POD_NAMELast State: Terminated, Reason: OOMKilled confirms the container exceeded its memory limit and the kernel killed it — this is a hard limit, not a warning.
kubectl top pod POD_NAME --containersShows current usage against the configured limit, if the pod is up long enough to sample.
Either raise the container's memory limits, or fix an actual leak — check whether usage climbs steadily over the
pod's lifetime (a leak) or spikes sharply under load (undersized limit for legitimate peak usage).
DNS Issues
kubectl run -it --rm debug --image=busybox --restart=Never -- nslookup my-serviceRuns a throwaway pod to test whether cluster DNS resolves a service name at all.
kubectl get pods -n kube-system -l k8s-app=kube-dnsConfirms CoreDNS pods are actually Running — cluster-wide DNS failures usually trace back here.
Confirm the query uses the right form: my-service resolves within the same namespace; a pod in a different
namespace needs my-service.NAMESPACE.svc.cluster.local.
Service Connectivity
A Service exists but nothing can reach it through it.
kubectl get endpoints SERVICE_NAMEEmpty endpoints means the Service's selector doesn't match any Ready pod — check the selector against the pod's actual labels, and confirm the pod is passing its readiness probe.
kubectl describe svc SERVICE_NAMEConfirms the selector and the port/targetPort mapping — a common mistake is targetPort not matching the container's actual listening port.
Ingress Errors
kubectl describe ingress INGRESS_NAMEShows whether the ingress has resolved a backend at all, and any controller-reported errors.
kubectl logs -n ingress-nginx deploy/ingress-nginx-controllerThe controller's own logs show the specific upstream error — connection refused, timeout, or a TLS mismatch — behind a generic 502/504 at the edge.
See HTTP 502 Bad Gateway for the general proxy-to-upstream diagnostic sequence, which applies directly here.
PVC Problems
kubectl get pvcA claim stuck Pending usually means no StorageClass matches what it's asking for, or the provisioner can't satisfy the requested size/access mode.
kubectl describe pvc PVC_NAMEEvents show the provisioner's own rejection reason — insufficient capacity in the backing storage, an access mode the storage class doesn't support, or a missing default StorageClass entirely.
ReadWriteOnce and multiple pods
A PVC with ReadWriteOnce can only be mounted read-write by pods on a single node at a time. A pod stuck Pending
after a rollout to a new node — while an old pod using the same PVC hasn't fully terminated — is a common,
confusing variant of this, not a storage capacity problem.