Goutham K

TIL: a Terminating pod can still get traffic for a bit

I assumed once a pod showed Terminating, Services had already dropped it. Not quite — there’s a window where the container is winding down and the load balancer / kube-proxy path may still be sending it requests.

On delete, the pod gets a deletionTimestamp and kubelet starts the graceful shutdown (terminationGracePeriodSeconds, preStop hooks, then SIGTERM). Endpoint / EndpointSlice removal is racing that shutdown, not guaranteeing ‘zero traffic before SIGTERM’. If your app keeps accepting connections until it dies, those late requests fail or get cut mid-flight.

The usual fix is to fail readiness as soon as shutdown starts — or use a preStop sleep / hook so endpoint propagation can finish before the process stops listening:

spec:
  containers:
    - name: api
      lifecycle:
        preStop:
          exec:
            command: ["/bin/sh", "-c", "sleep 5"]
  terminationGracePeriodSeconds: 30

Readiness going False is what pulls you out of Service endpoints. Terminating alone is not a traffic barrier.

So ‘pod is Terminating’ means the control plane started deletion — not that the mesh, the NLB, or kube-proxy already agreed to leave it alone.