When apply fails¶

A plan can fail half-way through. What the engine does next is the part worth knowing before you need it.

atlantide apply --on-failure rollback   # default: undo what this run completed
atlantide apply --on-failure halt       # stop where it stands, undo nothing

The three phases¶

Apply runs in phases, and only the first one is reversible.

Phase	What runs	Order	Reversible
1	CREATE, UPDATE, REPLACE	dependencies first, in parallel	yes
1b	the destroy half of a create-before-destroy REPLACE	dependents first	no — terminal
2	DELETE	dependents first	no — terminal

Phases 1b and 2 are terminal by design: recreating a destroyed resource would lose its identity and its outputs, so a failure there is reported rather than compensated. Committed stack outputs are persisted only after all three phases succeed.

The compensation saga¶

Under the default --on-failure rollback, every node that completes in phase 1 records an undo. On failure they run in reverse completion order — not reverse graph order, because what matters is what this run actually did.

What was applied	What the undo does
CREATE	provider.delete, then drop the state row. The resource carries the outputs of its create — notably its id — so the delete acts on that resource directly rather than locating it by attributes, which could match an unrelated resource sharing them.
UPDATE	provider.update back to the prior values, then restore the prior state row verbatim, still sealed.
REPLACE	Undo the create half, then restore the prior resource and its recorded outputs.

The saga catches BaseException, not Exception: an interrupt arrives as a cancellation, and a narrower clause would skip compensation on a Ctrl-C part-way through an apply that has already created resources.

When rollback deliberately does not run¶

A run that lost its lease is not rolled back. A compensation is a provider call and a state write, and a run that no longer holds the lock may be racing whoever took it — undoing "its" creates could destroy theirs. Leaving the resources in place is the lesser harm. The report says so explicitly rather than staying quiet about it.

This is the one blocker today. Recover with atlantide refresh.

When rollback itself fails¶

An undo is a provider call followed by a state write, so a failure between the two leaves state describing a resource that is no longer there. Because the diff is symbolic, that row's stored hash would still match config — and the next plan would report NOOP, hiding the divergence permanently.

The engine forecloses that. Before running a node's compensation it poisons the row, clearing its input_hash so the next plan cannot Merkle-skip it. The undo's own state write clears the poison when it succeeds, so being early costs nothing, while poisoning afterwards would only survive an exception — a process killed mid-rollback would leave the row untouched and the divergence silent.

When compensations fail, the original error and the rollback errors are raised together, as a group, so every leaf is rendered. The apply failure is never replaced by the rollback failure.

Interrupts¶

The first Ctrl-C cancels the run and follows the failure path above: under the default, completed nodes are compensated on the way out. It exits 130.

A second Ctrl-C abandons that cleanup. The compensation is shielded against it, so the press cannot cancel an undo between its provider call and its state write — the exact window poisoning exists to make visible — but the CLI's hard exit still wins. State may then no longer describe what exists: run atlantide refresh before applying again.

The run's lock is deliberately not released on that second press. An abandoned run may still have provider calls in flight, so the lease is left to lapse, or to be cleared explicitly with atlantide state unlock, rather than handed straight to a second writer.

The re-diff before anything runs¶

Apply does not execute the changeset that plan printed. Once it holds the lease it re-diffs against freshly loaded state — it must, or a resource another run created in the meantime would still be planned as a CREATE and get built twice.

So the plan a human approved and the plan about to execute can legitimately differ, and the gap between them is exactly where an unreviewed destroy fits. When they differ, the run stops with a PlanDriftError naming what appeared and what disappeared, rather than reconciling it silently.

After a failed run¶

atlantide refresh          # what actually exists, vs what state records
atlantide state list       # what state records
atlantide state unlock     # who holds the lease

refresh is the recovery command in every case above. It is read-only unless given --write, and a resource the provider cannot find is reported but kept unless --prune is also given.

See also Troubleshooting for the specific errors these paths raise.