Production patterns · Prasanta Kumar Behera

Run Terraform like production software: linting and security scanning, built-in tests, policy as code on plans, reviewed pipelines, drift detection, and the habits that keep large codebases healthy.

Infrastructure code deserves engineering discipline

Terraform changes production directly. Treat it like application code: review, lint, test, scan, gate, monitor.

Before a new ride opens at a theme park, it gets a checklist inspection (linting and scanning), test runs with sandbags instead of people (tests), a safety officer's sign-off against the rulebook (policy as code), and daily checks once it's open (drift detection). Terraform code deserves the same.

1. Static quality gates

Tool	Catches
`terraform fmt`, `validate`	Formatting, syntax, invalid references
tflint	Provider-specific mistakes (invalid instance types, deprecated arguments), naming conventions
checkov / trivy config	Security misconfigurations: public buckets, open security groups, unencrypted storage, missing logging

Run them in pre-commit hooks (fast feedback) and CI (enforced).

2. Tests with terraform test

# tests/bucket.tftest.hcl
variables {
  name = "acme-dev-logs"
}

run "bucket_is_named_and_private" {
  command = plan

  assert {
    condition     = aws_s3_bucket.this.bucket == "acme-dev-logs"
    error_message = "Bucket name must come from the name variable"
  }

  assert {
    condition     = aws_s3_bucket_public_access_block.this.block_public_acls == true
    error_message = "Public ACLs must be blocked"
  }
}

$ terraform test
tests/bucket.tftest.hcl... in progress
  run "bucket_is_named_and_private"... pass
tests/bucket.tftest.hcl... pass
Success! 1 passed, 0 failed.

command = plan tests are fast and free.
command = apply tests create and destroy real resources (in a sandbox account); use them for critical modules.
Terratest (Go) suits complex end-to-end tests (e.g. "deploy the module, then make an HTTP request to it").

3. Policy as code on the plan

# policy/no_cluster_deletion.rego
package main

deny[msg] {
  rc := input.resource_changes[_]
  rc.type == "aws_eks_cluster"
  rc.change.actions[_] == "delete"
  msg := sprintf("Refusing to delete EKS cluster %s", [rc.address])
}

$ terraform show -json tfplan > plan.json
$ conftest test plan.json -p policy/
FAIL - plan.json - main - Refusing to delete EKS cluster module.eks.aws_eks_cluster.this[0]

Good candidates: no deletions of stateful or critical resources without an explicit label, required tags, allowed regions and instance families, encryption everywhere.

4. Pipelines and review

Plan on PR, apply on merge, with separate identities and approvals (see Amazon EKS in Production, lesson 10).
Reviewers read the plan, not just the diff. The plan is what will happen.
One apply per state at a time; locking always on.

5. Drift detection and hygiene

Nightly plan -detailed-exitcode per state; alert on exit code 2 with the plan attached.
Restrict console write access so drift is rare and intentional (break-glass).
Keep providers and modules current with automated update PRs (Renovate/Dependabot), tested in dev first.

Try it: a quality pipeline for your module

Take the bucket (or files) module from lesson 04 and add a tests/ folder with two plan-mode tests.
Add tflint and checkov; introduce a deliberate problem (e.g. block_public_policy = false) and see which tool catches it.
Write a Conftest policy that denies any resource missing an Owner tag, and run it against a plan JSON.
Put fmt, validate, tflint, checkov, terraform test and Conftest into a GitHub Actions workflow.
Make a change by hand (outside Terraform) and catch it with a drift job run.

Going deeper: large Terraform estates

Standardise repository layout and module conventions early; inconsistency is the main cost at scale.
Measure plan time and state size; split states that grow slow.
Keep exceptions to policies explicit (a label or allow-list with an owner and expiry), never by disabling a rule globally.
OpenTofu and Terraform share most workflows. If you evaluate a switch, test providers, state encryption options and CI tooling on a copy first.

Recap

Lint and scan (fmt, validate, tflint, checkov/trivy) in pre-commit and CI.
terraform test for fast plan-mode assertions; apply-mode or Terratest for critical modules.
Policy as code (Conftest/OPA) on the plan JSON: deletions, tags, regions, encryption.
Reviewed pipelines, locking, and nightly drift detection.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.