Building on AWS¶

This guide builds a multi-environment AWS stack one resource at a time, and introduces each authoring feature at the point where it is first needed.

The finished configuration has a shared VPC in a common stack, and dev and prod stacks that each hold an S3 bucket, an SQS queue, an IAM role with an inline policy, a Lambda function that reads a secret, and a security group attached to the shared VPC. That is fifteen resources across two providers, wired together entirely by references.

The end state is examples/aws/example-one.py in the repository, so you can compare your file against it at any point.

If you have not met the engine before, the walkthrough covers the same pipeline against local files in more depth. This guide assumes that material.

Before you start¶

mkdir aws-demo && cd aws-demo

plan requires nothing further. apply requires AWS credentials resolvable in any of the usual ways — environment variables, ~/.aws/credentials, or an instance role — together with permission to create the resource types used below.

1. One bucket, one stack¶

infra.py:

from atlantide.core import Stack, output
from atlantide.providers.aws import Region, S3Bucket

with Stack("dev", region=Region.EuNorth1, name_prefix="atlantide", tags={"env": "dev"}):
    assets = S3Bucket("assets", bucket="atlantide-assets-dev-7f3a91c2")
    output("assets_arn", assets.arn)

atlantide validate infra.py

ok infra.py — 1 resource(s), no cycles

atlantide plan infra.py --state infra.db

state: infra.db
dev ────────────────────────────────────────────────────────────────────────────
  + create  aws.S3Bucket:assets

Plan: 1 to add

Outputs:
  dev:assets_arn = (known after apply)

Three things are happening in six lines:

S3Bucket blocks public access and enables SSE-S3 encryption by default. A public or unencrypted bucket should be a decision recorded in the config, not an oversight.

2. A queue, a role, and a policy wired by references¶

Add an SQS queue, an IAM role, and an inline policy granting the role access to both:

from atlantide.core import Stack, output
from atlantide.providers.aws import (
    IamPolicy, IamRole, Region, S3Bucket, ServicePrincipal, SqsQueue, allow,
)

with Stack("dev", region=Region.EuNorth1, name_prefix="atlantide", tags={"env": "dev"}):
    assets = S3Bucket("assets", bucket="atlantide-assets-dev-7f3a91c2")
    jobs = SqsQueue("jobs", fifo=True)

    worker = IamRole("worker", assumed_by=[ServicePrincipal.Ec2, ServicePrincipal.Lambda])
    IamPolicy(
        "worker-policy",
        role_arn=worker.arn,
        policy_name="worker-access",
        statements=[
            allow(S3Bucket.Action.GetObject, S3Bucket.Action.PutObject, on=assets.objects_arn),
            allow(SqsQueue.Action.SendMessage, on=jobs.arn),
        ],
    )
    output("assets_arn", assets.arn)

atlantide plan infra.py --state infra.db

state: infra.db
dev ────────────────────────────────────────────────────────────────────────────
  + create  aws.IamPolicy:worker-policy
  + create  aws.IamRole:worker
  + create  aws.S3Bucket:assets
  + create  aws.SqsQueue:jobs

Plan: 4 to add

Nothing in the config declares an ordering, yet one exists. Ask the engine for it:

atlantide graph infra.py --format mermaid

The policy depends on all three resources because it reads all three: worker.arn, assets.objects_arn and jobs.arn. None of those ARNs exists yet, so each read returns a lazy Ref, and each Ref becomes an edge in the graph.

The role, bucket and queue do not depend on one another, so they are created in parallel. The policy waits for all three.

Two details of the policy are worth noting. allow() builds the policy document, so there is no hand-written JSON, and the actions are enum members such as S3Bucket.Action.GetObject — a misspelled action fails before anything runs.

The statement also uses assets.objects_arn rather than assets.arn, because object-level permissions require the /* form. The resource exposes both, so the correct one is chosen by name instead of by remembering to append a suffix.

3. A Lambda, fingerprinted from disk¶

Create the handler:

def handler(event, context):
    return {"received": len(event.get("Records", []))}

Then add the function, and give it the role:

processor = LambdaFunction(
    "processor",
    role_arn=worker.arn,
    handler="app.handler",
    code_path="processor",
    signing_secret=SecretRef("app/signing-key-dev"),
)

code_path points at a local directory. The directory is zipped with sorted entries and fixed timestamps, then fingerprinted into code_sha256 at config-evaluation time.

Two properties follow. Two checkouts of the same source produce the same digest, and editing app.py changes the digest — which is what makes the next plan show an update.

Now plan:

atlantide plan infra.py --state infra.db

state: infra.db
error: undefined secret(s): dev:aws.LambdaFunction:processor.signing_secret -> 
'app/signing-key-dev'

The plan refuses to proceed. SecretRef("app/signing-key-dev") names a secret that does not exist in the store, and an apply would therefore fail part-way through, having already created some resources. Atlantide checks the entire secret set during planning to avoid that.

Set it:

atlantide secret set app/signing-key-dev

set secret 'app/signing-key-dev'

Omitting the value prompts for it with the input hidden. Secrets are held in a local AES-256-GCM encrypted keyfile store by default; [secrets] in atlantide.toml can point at SSM Parameter Store or the environment instead.

atlantide plan infra.py --state infra.db

state: infra.db
dev ────────────────────────────────────────────────────────────────────────────
  + create  aws.IamPolicy:worker-policy
  + create  aws.IamRole:worker
  + create  aws.LambdaFunction:processor
  + create  aws.S3Bucket:assets
  + create  aws.SqsQueue:jobs

Plan: 5 to add

The secret's value never appears in the config, the IR or the state file — only its name travels. The plaintext is resolved in memory at apply time. State keeps a salted digest of it, which is enough to detect a rotated secret without storing the secret itself.

4. A shared VPC, and a second environment¶

One VPC that both environments build on. Put it in its own stack:

with Stack("common", region=Region.EuNorth1, name_prefix="atlantide", tags={"env": "common"}):
    network = Vpc("network", cidr_block="10.0.0.0/16")
    vpc_id = output("vpc_id", network.vpc_id)

output() returns a typed handle. Consuming that variable, rather than a string such as "common:vpc_id", turns a typo into an undefined-variable error at config time instead of a mismatch discovered during planning.

Now wrap the environment body in a loop and hang a security group off the shared VPC:

for env in ["dev", "prod"]:
    with Stack(env, region=Region.EuNorth1, name_prefix="atlantide", tags={"env": env}):
        ...
        edge = SecurityGroup("edge", group_name=f"edge-{env}", vpc_id=vpc_id)

This is ordinary Python — a for loop over stacks. Because common lives in the same config, the engine inlines the output handle into a real dependency edge and applies common before dev and prod. Those two are independent of each other, so they apply in parallel.

5. Deterministic names, and a value decided at apply¶

Two more lines in the environment body:

build = Id("build-id", byte_length=4)

bucket_name = f"atlantide-assets-{env}-{uuid5('atlantide-buckets', env)[:8]}"
assets = S3Bucket(
    "assets",
    bucket=bucket_name,
    versioning=(env == "prod"),
    tags={"build_id": build.result, "config_hash": sha256_hex(env)[:12]},
)

The two lines look similar but do opposite things:

The bucket name is therefore stable across every run and every machine, while the build id is stable across every run after the first.

The bucket reads build.result, so the engine orders the random resource before the AWS bucket — a single graph spanning two providers.

Note also versioning=(env == "prod"). Per-environment differences are ordinary Python expressions; there is no separate templating layer.

6. Policies, at the top of the file¶

enforce("require-tags", keys=["env"])
enforce("deny-destroy-in-protected", stacks=["prod"])

Policies run at the end of planning, against the finished changeset, so they see the actions rather than only the config. A mandatory violation blocks apply before anything is touched.

To see the first one working, drop the env tag from the stacks and plan:

policy DENY require-tags: dev:aws.S3Bucket:assets is missing tag(s): env
policy DENY require-tags: dev:aws.SqsQueue:jobs is missing tag(s): env
policy DENY require-tags: dev:aws.SecurityGroup:edge is missing tag(s): env
policy DENY require-tags: dev:aws.LambdaFunction:processor is missing tag(s): 
env
...
10 mandatory policy violation(s) block apply

Every offending resource is named. Because stack tags merge into every resource declared in the body, restoring tags={"env": env} on the two stacks satisfies all ten violations at once, without repeating a tag per resource.

The second policy, deny-destroy-in-protected, fails any plan that would delete or replace something in prod. That covers removing a resource from the file, and also editing an immutable() field, since a replacement is a destroy followed by a create.

7. The finished config¶

Exports for each environment, then the whole thing:

output("assets_arn", assets.arn)
output("assets_bucket", assets.bucket)
output("jobs_url", jobs.url)
output("worker_role_arn", worker.arn)
output("processor_arn", processor.arn)
output("build_id", build.result)
output("edge_sg_id", edge.group_id)
output("region", Region.EuNorth1)

atlantide plan infra.py --state infra.db

state: infra.db
common ─────────────────────────────────────────────────────────────────────────
  + create  aws.Vpc:network
dev ────────────────────────────────────────────────────────────────────────────
  + create  aws.IamPolicy:worker-policy
  + create  aws.IamRole:worker
  + create  aws.LambdaFunction:processor
  + create  aws.S3Bucket:assets
  + create  aws.SecurityGroup:edge
  + create  aws.SqsQueue:jobs
  + create  random.Id:build-id
prod ───────────────────────────────────────────────────────────────────────────
  + create  aws.IamPolicy:worker-policy
  + create  aws.IamRole:worker
  + create  aws.LambdaFunction:processor
  + create  aws.S3Bucket:assets
  + create  aws.SecurityGroup:edge
  + create  aws.SqsQueue:jobs
  + create  random.Id:build-id

Plan: 15 to add

Outputs:
  common:vpc_id = (known after apply)
  dev:assets_arn = (known after apply)
  dev:assets_bucket = 'atlantide-assets-dev-b89f6cc5'
  dev:jobs_url = (known after apply)
  dev:worker_role_arn = (known after apply)
  dev:processor_arn = (known after apply)
  dev:build_id = (known after apply)
  dev:edge_sg_id = (known after apply)
  dev:region = 'eu-north-1'
  prod:assets_arn = (known after apply)
  prod:assets_bucket = 'atlantide-assets-prod-445ca963'
  ...

Note which outputs already have values. assets_bucket and region are known now, because both are computed at config time. Everything else reads (known after apply) because it does not exist yet.

The whole graph:

One VPC feeds two independent environment subgraphs. Nothing in the config stated that ordering; it follows from which resources read which values.

8. Apply, and what happens after¶

atlantide secret set app/signing-key-prod    # prod needs its own
atlantide apply infra.py --state infra.db

Apply takes a lease over the reachable graph, re-diffs under that lock, and reconciles independent nodes in parallel. State is written per node as each one succeeds, so a crash part-way through still leaves an accurate record. On failure, the default --on-failure rollback undoes the nodes that had completed.

Run it a second time and every node is a no-op, with no AWS API calls at all: the engine compares hashes and stops. On a graph this size, that is the difference between a re-run costing nothing and one costing hundreds of calls.

Afterwards:

atlantide output --state infra.db        # what the apply exported
atlantide state list --state infra.db    # what is recorded
atlantide refresh --state infra.db       # has anything drifted?
atlantide destroy --state infra.db       # tear it down, dependents first

Making it a project¶

Put the defaults in an atlantide.toml beside the config, and the flags are no longer needed:

config      = "infra.py"
state       = "infra.db"
aws_region  = "eu-north-1"
aws_profile = "default"

atlantide plan       # no flags needed

For a team, move state to a shared backend — S3 with DynamoDB locking, or Postgres — so that concurrent runs cannot overwrite each other. That, along with per-environment [profile.<name>] overlays, is covered in Remote state and Configuration.

Where to go next¶

• The Merkle hash — why the second apply costs nothing, and which nodes a change propagates to.
• Authoring — output combinators for values not known until apply, explicit ordering, and renaming a resource without destroying it.
• Components — packaging this stack as a reusable construct, publishable from a git repository and pinned by commit.
• Providers — the full AWS resource list, and how to write a provider.
• A static website — examples/aws/example-three.py builds a private S3 origin behind CloudFront with an Origin Access Control, in four resources.