Articles

Deployment Rollback Strategies: Roll Back vs Forward

Rolling back reverts to the last known-good deploy; rolling forward ships a fix on top of the bad one. How to choose, and why database changes complicate both.

The Lycoris Team The Lycoris Team · · 4 min read
Abstract illustration representing DevOps workflows

A rollback reverts a running system to its last known-good version after a deploy causes a problem. Rolling forward takes the opposite approach: instead of reverting, you ship a new deploy that fixes the issue on top of the broken one. Both are legitimate responses to a bad release, and most teams need both strategies available, because which one is faster and safer depends entirely on what actually broke and how the deploy was structured.

Why you need a plan before the incident

The worst time to design a rollback strategy is during an active incident, with a broken deploy in production and people asking when it’ll be fixed. Whether reverting is even possible — and how long it takes — is decided by decisions made long before the bad deploy: how deployments are packaged, whether infrastructure is treated as immutable, and whether database changes are kept separate from application changes. Teams that only discover their rollback story doesn’t work while the site is down have, in effect, no rollback story.

Rolling back

A clean rollback assumes the previous version is still available, deployable, and compatible with the current state of everything it depends on — meaning the previous application code still works correctly against the current database schema, external service contracts, and any in-flight data. That assumption is easy to break.

Rolling back works best when deployments are immutable artifacts (a container image, a versioned build) rather than in-place file changes on a server, since immutable artifacts guarantee the “previous version” is a complete, known-good snapshot rather than whatever files happen to be left after a partial revert. Teams using GitOps get this close to automatically: the previous known-good state is just the prior commit in the repository that describes the desired deployed state, and reverting is often as simple as reverting that commit and letting the reconciliation loop apply it.

The case where rollback gets complicated — sometimes impossible — is a database migration that isn’t backward compatible. If a deploy added a column and the new code depends on it, rolling back the application code alone leaves the schema in a state the old code was never written to handle. See database migrations for why migrations are usually written to be applied in reversible, additive steps specifically so this scenario doesn’t happen — expand the schema in one release, migrate the code to use it, and only remove the old column in a later release once nothing depends on it anymore.

Rolling forward

Sometimes reverting isn’t practical — the bad deploy already included a data migration that ran and can’t be cleanly undone, or the fix is small and well understood, and writing, testing, and shipping it forward is genuinely faster than reconstructing the previous state. Rolling forward means you fix the specific problem with a new, small, targeted deploy rather than discarding everything in the release.

The risk with rolling forward is compounding: a fast, under-tested “forward fix” pushed under incident pressure can introduce a second bug on top of the first. It’s the right call when the fix is well-scoped and the team has confidence in it — not a default reflex to avoid the discomfort of admitting a release needs to be pulled.

Feature flags as a third option

A well-placed feature flag sidesteps this decision entirely for a large class of problems: if the broken behavior is gated behind a flag, disabling the flag reverts the behavior instantly without touching the deployed code or the database at all. This is the fastest possible mitigation — no build, no deploy pipeline, often no more than an API call or a config change — and it’s why many teams deliberately wrap risky new functionality in a flag even when they don’t expect to need it. The tradeoff is that flags only cover what was actually flagged; a bug in code that runs unconditionally has no flag to flip off.

Rollback vs roll-forward at a glance

Roll backRoll forward
What happensRevert to last known-good versionShip a new fix on top of the bad deploy
SpeedFast, if the previous version is deployableDepends on how quickly the fix can be written and verified
Best forClean regressions with no incompatible data changesMigrations that already ran, or small well-understood fixes
Main riskSchema/data incompatibility with the reverted codeA rushed fix introduces a second bug
Requires in advanceImmutable, versioned deploy artifactsConfidence in fast, safe test-and-ship cycles

Reducing how often you need either one

Gradual rollout strategies exist specifically to shrink the blast radius before a full rollback is ever necessary. A canary deployment exposes a new version to a small slice of traffic first, so a bad release is caught and reverted while it’s only affecting a fraction of users, rather than everyone. Blue-green deployments make the rollback itself close to instantaneous by keeping the previous environment fully running and simply switching traffic back to it, with no rebuild required. Neither approach eliminates the need for a rollback plan, but both reduce how often a full incident-level rollback is the only option left.

The takeaway

Rolling back and rolling forward are both legitimate responses to a bad deploy, and the right choice depends on what broke: a clean application regression usually favors rolling back, while an already-applied data migration or a small, well-understood bug usually favors rolling forward. The deciding factor in how fast either one goes is preparation done well before the incident — immutable deploy artifacts, backward-compatible migrations, and feature flags around risky code — not decisions made in the middle of it.

The Lycoris Team The Lycoris Team · · 4 min read

What Is an Internal Developer Platform (IDP)?

An internal developer platform packages infrastructure into self-service tools so developers ship without filing tickets or learning Kubernetes.

#DevOps #Cloud #Developer Tools