Graceful Shutdown Explained: SIGTERM, Draining, Deploys

Graceful shutdown is the practice of letting a running service finish the work it has already accepted before the process exits. Instead of dying the instant it is told to stop, the service stops taking new requests, completes or hands off in-flight ones, closes connections and flushes buffers cleanly, and only then exits. It is the difference between a deploy that nobody notices and one that sprays errors at every user who happened to be mid-request.

Why abrupt shutdowns cause errors

Modern infrastructure stops processes constantly. Every rolling deploy replaces old instances with new ones. Autoscalers remove capacity when traffic drops. Orchestrators reschedule workloads when a node is drained for maintenance. Spot or preemptible instances get reclaimed. In all of these cases, something tells your process to stop while it may be in the middle of real work.

If the process exits immediately, several things go wrong at once:

• In-flight HTTP requests fail. Clients see connection resets or 502 errors from the load balancer.
• Background jobs are cut off halfway. A job that charged a card but had not yet recorded the charge leaves inconsistent state behind.
• Buffered data is lost. Logs, metrics, and batched writes still sitting in memory never reach their destination.
• Database transactions are left open until the server notices the dead connection and rolls them back.

None of these are exotic. They are the default outcome when shutdown is not designed.

The signal sequence

On Linux and other Unix-like systems, a polite stop request arrives as the SIGTERM signal. Unlike SIGKILL, SIGTERM can be caught: your program can register a handler and decide what to do. Orchestrators and process managers follow a common pattern:

1. Send SIGTERM to the process.
2. Wait for a grace period.
3. If the process is still running when the period ends, send SIGKILL, which cannot be caught and ends the process immediately.

Kubernetes, for example, sends SIGTERM to the main process in each container and then waits for the pod’s termination grace period before force-killing it. Docker’s stop command behaves the same way with its own timeout. The grace period is your budget. Everything a graceful shutdown does has to fit inside it.

A common trap is that the signal never reaches your code. If a container’s entrypoint is a shell script that launches your app as a child process, the shell may receive SIGTERM and not forward it. Using the exec form of the entrypoint, or a minimal init process, keeps your application as the process that actually receives signals. Our explainer on Dockerfile CMD vs ENTRYPOINT covers the exec and shell forms in more detail.

The steps of a graceful shutdown

A well-behaved service handles SIGTERM in a predictable order.

1. Stop receiving new work

First, make sure no new traffic arrives. For a web service this has two halves. The service should stop accepting new connections on its listening socket, and the load balancer in front of it should stop routing requests to it.

The second half is the subtle one. Removing an instance from a load balancer or a service’s endpoint list is asynchronous. For a short window after SIGTERM, traffic can still be routed to a process that has begun shutting down. Many teams handle this by failing the readiness check immediately and then waiting a few seconds before closing the listener, so the routing layer has time to notice. Our guide to liveness vs readiness probes explains why readiness, not liveness, is the right signal here.

For queue consumers, “stop receiving” means stop pulling new messages while leaving already-claimed messages in progress.

2. Drain in-flight work

Next, let current requests and jobs finish. Most HTTP server libraries offer a shutdown method that closes idle keep-alive connections and waits for active requests to complete. Long-lived connections such as WebSockets or server-sent events need an explicit plan: notify clients that the server is going away so they reconnect elsewhere, then close.

Work that cannot finish in time should be handed back rather than abandoned. A queue consumer can release or negatively acknowledge an unfinished message so another worker picks it up. That only works safely if processing is designed so that running a message twice is harmless, which is why idempotency and graceful shutdown go hand in hand.

3. Release resources

Once work has drained, close database connections and connection pools, flush log and metric buffers, commit or roll back any open transactions, and release locks or leases held in external systems. Closing these explicitly is faster and cleaner than letting remote systems time out a dead peer.

4. Exit with the right code

Finally, exit. An exit code of zero tells the orchestrator the shutdown went as planned. If draining hit its internal deadline and work had to be abandoned, logging that fact clearly makes later debugging much easier.

Timeouts: budget the grace period

Every step needs a deadline, and the deadlines must add up to less than the platform’s grace period. Suppose the platform allows 30 seconds. A reasonable budget might be a few seconds of readiness delay, around 20 seconds for draining, and a few seconds for cleanup, leaving margin before SIGKILL lands.

Set the drain timeout based on your real request profile. A service whose slowest legitimate requests take a minute needs a longer grace period configured at the platform level, not a drain step that silently gets killed halfway through. Conversely, a very long grace period slows down every deploy and every scale-in, so keep it as short as your workload allows.

Graceful vs abrupt shutdown

	Abrupt shutdown	Graceful shutdown
In-flight requests	Dropped with errors	Completed or retried elsewhere
New traffic during stop	May still arrive and fail	Routed away first
Background jobs	Cut off mid-execution	Finished or returned to the queue
Buffered logs and metrics	Lost	Flushed
Deploy visibility to users	Error spikes	Typically invisible

Where graceful shutdown fits

Graceful shutdown is one piece of a larger reliability story. Rolling, blue-green, and canary deployments all assume that old instances can leave without breaking requests; graceful shutdown is what makes that assumption true. Autoscaling based on load, such as the Kubernetes HPA, removes instances routinely, so every scale-in is effectively a small deploy. The disposability principle from the twelve-factor app methodology states the goal directly: processes should start fast and shut down gracefully on SIGTERM.

The best way to know whether your service gets this right is to test it. Run a steady stream of requests against a staging environment, trigger a rolling restart, and look for errors. If any appear, trace which of the steps above was skipped.

The takeaway

Graceful shutdown means treating SIGTERM as a request to wrap up, not an instruction to vanish. Stop receiving new work, give the routing layer time to notice, drain in-flight requests and jobs, hand back anything unfinished, release resources, and exit, all within the platform’s grace period. Get it right and deploys, scale-ins, and node maintenance become invisible to users.