How it works

This section covers the engine in three passes, shortest first.

  • This page — the mental model, the pipeline on one screen, and which command runs which stages.
  • Walkthrough — the same pipeline run end to end against real files. It needs no credentials, so you can follow it without an AWS account.
  • The Merkle hash — the idea the rest of the engine rests on, examined in detail with real numbers.
  • When apply fails — the phases, what gets rolled back, what cannot be, and how the engine keeps a failed rollback from going silent.

New to the vocabulary? The glossary defines every term below, in the order the pipeline meets them.


The mental model

Atlantide runs a Python file in a sandbox that has no clock, no randomness, no environment and no network. The resources the file creates become a content-hashed dependency graph, called the IR.

Each node in that graph is hashed together with its dependencies, producing an input_hash per node. Those hashes are compared against the ones recorded in a state database, and the difference between the two sets is the plan.

Apply then walks the graph in topological order, in parallel, calling providers only for nodes whose hash changed.

Two consequences follow, and both are worth holding onto:

plan needs no credentials. Everything up to the diff is pure computation over the config and the recorded state.

The diff compares desired inputs against recorded inputs, not against the live world. Detecting infrastructure that has changed underneath you is a separate command, refresh. The walkthrough shows precisely where that boundary sits, and what it costs.


The whole pipeline on one page

infra.py
   │
   │  ─── pure: no I/O, no credentials, byte-reproducible ───
   │
   ├─ 1  Atlas-lang        validate_source + Interpreter.run   lang/validate.py, lang/interp.py
   ├─ 2  Lower             lower(registry, providers)          ir/lower.py
   ├─ 3  Canonicalize      RFC 8785 + sha256                   ir/hash.py
   ├─ 4  Graph             build_graph (Tarjan) + Kahn         graph/build.py, graph/order.py
   ├─ 5  Merkle            merkle_hashes -> input_hash/node    ir/merkle.py
   │
   │  ─── touches the world ───
   │
   ├─ 6  Diff              vs prior hashes from state          reconcile/diff.py
   ├─ 7  Plan              guard + policies                    reconcile/planner.py, engine/planner.py
   └─ 8  Apply             lease -> re-diff -> parallel DAG    engine/locking.py, reconcile/executor.py

The layering is enforced rather than documented. pyproject.toml declares import-linter contracts for the order cli > engine > reconcile > (policy | providers | state | secrets | lang) > graph > ir > core, along with three further rules: core imports no sibling package, and both providers and secrets depend only on core.

Those contracts run in CI beside mypy --strict and a 90% coverage floor. An import that crosses a layer sideways fails the build.


Command reference

Command Runs stages Needs credentials Writes state
validate 1–5 no no
graph 1–4 no no
build 1–5 no no
verify no no
plan 1–7 no no
apply 1–8 yes yes
deploy 6–8 (from artifact) yes yes
refresh provider read only yes only with --write
import 1–5, then provider read only yes yes (records; creates nothing)
destroy 6–8, empty desired graph yes yes
init — (compiles the config it scaffolds) no no

The mutating commands share a set of flags: --confirm/-y, --state, --region, --parallelism/-p, --on-failure rollback|halt (defaulting to rollback), --target/-t, --replace, --json, --detailed-exitcode and --audit-log. What --on-failure actually does is When apply fails.

Defaults come from atlantide.toml, which is looked up from the current directory upwards. It sets the config path, the state backend, the secrets provider, parallelism, AWS region, profile and endpoint, component pins, inputs, and any [profile.<name>] overlays selected with -P. See Configuration.