Use artifacts from S3

dbt-tools can read manifest.json and related artifacts directly from an Amazon S3 bucket. This is common in CI/CD pipelines that upload dbt artifacts after a run.

Minimal example

export DBT_TOOLS_S3_REGION=us-east-1
npx @dbt-tools/cli status \
  --dbt-target s3://my-bucket/dbt/prod/latest \
  --json

Required objects

The S3 prefix must contain:

Object key (relative to prefix)	Required
manifest.json	Yes
run_results.json	Yes (for run-related commands)
catalog.json	No
sources.json	No

For example, if --dbt-target is s3://my-bucket/dbt/prod/latest, dbt-tools reads:

s3://my-bucket/dbt/prod/latest/manifest.json
s3://my-bucket/dbt/prod/latest/run_results.json

Environment variables

Variable	Required	Description
DBT_TOOLS_S3_REGION	Yes	AWS region of the bucket
DBT_TOOLS_S3_ENDPOINT	No	Custom S3-compatible endpoint URL

Standard AWS credential variables also apply: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, and AWS_PROFILE. dbt-tools inherits the standard AWS SDK credential chain.

With a custom endpoint

For S3-compatible storage (MinIO, Ceph, DigitalOcean Spaces, etc.):

export DBT_TOOLS_S3_REGION=us-east-1
export DBT_TOOLS_S3_ENDPOINT=https://s3.example.internal
npx @dbt-tools/cli status \
  --dbt-target s3://my-bucket/dbt/prod/latest \
  --json

With named AWS profile

export AWS_PROFILE=my-read-only-profile
export DBT_TOOLS_S3_REGION=us-east-1
npx @dbt-tools/cli status \
  --dbt-target s3://my-bucket/dbt/prod/latest \
  --json

Recommended IAM permissions

Grant the IAM role or user used by dbt-tools read-only access to the artifact prefix only:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": ["arn:aws:s3:::my-bucket", "arn:aws:s3:::my-bucket/dbt/prod/latest/*"]
    }
  ]
}

Do not grant s3:PutObject, s3:DeleteObject, or bucket-level write permissions. dbt-tools only reads artifacts.

Using in GitHub Actions

- name: Check dbt artifact health from S3
  env:
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    DBT_TOOLS_S3_REGION: us-east-1
  run: |
    npx @dbt-tools/cli status \
      --dbt-target s3://my-bucket/dbt/prod/latest \
      --json

Troubleshooting

Symptom	Likely cause	Fix
NoSuchKey error	Object does not exist at the prefix	Confirm the prefix contains manifest.json
AccessDenied	IAM policy missing s3:GetObject on the prefix	Review and update IAM permissions
Auth error in CI	Credentials not passed to the step	Set AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY as secrets
Slow cold reads	Large artifacts or slow network	Consider caching artifacts as pipeline artifacts rather than re-reading from S3 each time

Related

• GCS — Google Cloud Storage
• Credentials — credential precedence and least-privilege guidance
• GitHub Actions — full CI example
• Configuration reference