DSQL

Serverless Postgres on Aurora DSQL. IAM auth over TLS, schemas and roles declared in code, and optional multi-region clusters.

Database · anvil:aws:DSQL

Set stream to send every committed insert, update and delete, in every table, to a KinesisStream. Lambdas read it as consumers of that stream, each filtering to the tables it cares about.

const indexer = new anvil.aws.Lambda('indexer', {
  runtime: 'nodejs24.x',
  handler: 'functions/indexer/index.handler',
});

const changes = new anvil.aws.KinesisStream('changes', {
  retentionHours: 168,
  consumers: [{
    consumerName: 'indexer',
    lambda: { arn: indexer.arn, lambdaRoleArn: indexer.roleArn },
    grantRead: true,
    filter: { tables: ['app.users'] },
  }],
});

const db = new anvil.aws.DSQL('db', {
  schemas: [/* ... */],
  stream: { kinesisArn: changes.arn },
});
indexer = anvil.aws.Lambda("indexer",
    runtime="nodejs24.x",
    handler="functions/indexer/index.handler",
)

changes = anvil.aws.KinesisStream("changes",
    retention_hours=168,
    consumers=[{
        "consumer_name": "indexer",
        "lambda_": {"arn": indexer.arn, "lambda_role_arn": indexer.role_arn},
        "grant_read": True,
        "filter": {"tables": ["app.users"]},
    }],
)

db = anvil.aws.DSQL("db",
    schemas=[...],
    stream={"kinesis_arn": changes.arn},
)
changes, err := anvilaws.NewKinesisStream(ctx.PulumiCtx(), "changes", &anvilaws.KinesisStreamArgs{
    RetentionHours: pulumi.Int(168),
    Consumers: anvilaws.KinesisConsumerArgsArray{
        &anvilaws.KinesisConsumerArgsArgs{
            ConsumerName: "indexer",
            Lambda: &anvilaws.KinesisLambdaConsumerArgsArgs{
                Arn:           indexer.Arn,
                LambdaRoleArn: indexer.RoleArn,
            },
            GrantRead: pulumi.Bool(true),
            Filter: &anvilaws.KinesisConsumerFilterArgsArgs{
                Tables: pulumi.StringArray{pulumi.String("app.users")},
            },
        },
    },
})
if err != nil {
    return err
}

db, err := anvilaws.NewDSQL(ctx.PulumiCtx(), "db", &anvilaws.DSQLArgs{
    Schemas: anvilaws.DSQLSchemaArray{ /* ... */ },
    Stream: &anvilaws.DSQLStreamArgsArgs{
        KinesisArn: changes.Arn,
    },
})

What deploy does

AWS has no infrastructure resource for a DSQL CDC stream, so the Anvil CLI manages it around the rest of your stack. Use anvil deploy; a plain pulumi up won't create it.

  • Anvil creates the IAM role DSQL writes with. Only this cluster can assume it, and it can only write to kinesisArn.
  • anvil deploy creates the CDC stream after everything else is deployed, and waits until it's active. The first time takes a few minutes.
  • anvil preview shows whether the stream will be created or deleted.
  • anvil destroy deletes the stream before anything else.

Before you turn it on

  • The Kinesis stream must be in the stack's region. For a multi-region cluster that's the stack region's cluster, which captures writes made in both regions.
  • Records must be allowed up to 2048 KiB, since a DSQL row can be 2 MiB. KinesisStream defaults to this; deploy fails early if the stream is smaller.
  • Events are unordered and delivered at least once. Consumers must handle repeats and must not rely on the order changes arrive in.
  • Set retentionHours: 168 on the stream. A failing consumer then has a week to recover before unread changes expire, instead of the default 24 hours.

Changing kinesisArn or roleArn replaces the CDC stream, and removing stream deletes it. DSQL can't update a stream in place, and changes made while there is none are never captured.

Good to know

  • If deploy warns that the stream is IMPAIRED, DSQL can't write to Kinesis. Check the role's permissions, and kmsKeyArn if the stream uses your own KMS key.
  • A stream that has FAILED is replaced on the next deploy. Streams Anvil didn't create are never touched.