State

Where Anvil keeps track of what's deployed, and what to do when a deploy gets stuck.

Anvil records every resource it has deployed, with its settings and outputs, in a state file. Each deploy compares your program with that file to work out what to create, change or delete. Anvil uses Pulumi's state format, stored in your own AWS account: nothing about your infrastructure leaves it.

Where state lives

Each stage has its own S3 bucket, <stage>-anvil-state-<project>-<id>, created the first time you deploy that stage. It's private, encrypted and versioned. State can contain secrets and resource details, so treat access to the bucket like access to the infrastructure itself.

What's kept

defaults.state.retainVersions (default 3) is how many deploys' state is kept: the latest deploy and the two before it. After every deploy Anvil deletes older deploys' copies. Configure it on the App.

• Deploys, not saves. The state file is saved after every resource change, so one deploy saves it hundreds of times. Those in-between saves are deleted a day after they're replaced. The current state is never deleted.
• Deploy summaries are kept forever. They're about 2 KB each, so years of deploys stay small. They are your record of when each deploy ran and what it changed. Infrastructure changes are also in CloudTrail.
• Cleanup is automatic. Anvil sets S3 lifecycle rules on the bucket when the stage is created and re-applies them on every deploy. S3 runs them about once a day, so deletions land a day or two after they're due. Anvil owns these rules: any you add to the state bucket by hand are replaced on the next deploy.

Compression

With defaults.state.compressState on (the default), state is saved gzipped. Every save during a deploy is uploaded to S3, so a smaller file makes deploys noticeably faster: a site's state of a few megabytes typically shrinks to tens of kilobytes. Turning it off or on is safe, since state is read in either format and rewritten in the new one on the next save. The only cost is that you can't read the file directly in the S3 console; download and unzip it first.

Locks

A deploy, preview or refresh locks the stage so two can't change it at once. The lock is released when it finishes, successfully or not. A deploy that's killed instead (Ctrl+C, a closed terminal, a crash) can leave its lock behind, and the next command fails with:

error: the stack is currently locked by 1 lock(s). Either wait for the other process(es) to end or delete the lock file with `pulumi cancel`.

First make sure nothing is actually running against the stage: no other terminal, and no CI job. Then release it:

anvil unlock --stage <stage>

Let deploys run to the end where you can. Stopping one halfway is the usual cause of both of these problems.

Interrupted deploys

If a deploy stops partway, some changes may have been sent to AWS without being recorded. The next deploy warns about pending operations and lists the resources that were mid-change. Refresh to check them against AWS, then deploy again:

anvil refresh --stage <stage>
anvil deploy --stage <stage>

refresh only updates the state to match what's really in your account: it doesn't create, change or delete anything in AWS. The deploy afterwards finishes whatever the interrupted one didn't.

Restoring an earlier deploy's state

Rarely needed: for example, if the current state is damaged. The full state from the end of each of the last retainVersions deploys is in .pulumi/history/, and you can put one back:

# 1. Find the checkpoint for the deploy you want (newest last)
aws s3 ls s3://<stage>-anvil-state-<project>-<id>/.pulumi/history/<project>/<stage>/ | grep checkpoint

# 2. Copy it over the current state file
aws s3 cp s3://<bucket>/.pulumi/history/<project>/<stage>/<stage>-<time>.checkpoint.json.gz \
          s3://<bucket>/.pulumi/stacks/<project>/<stage>.json.gz

# 3. Reconcile it with what's actually in your account
anvil refresh --stage <stage>

Use .checkpoint.json and <stage>.json instead if compressState is off. Run nothing else against the stage while you do this. Restoring state doesn't change your resources: the refresh brings the restored file up to date with what's actually deployed.