Lesson 03 of 7 · Level 1 — The model
Promotion between environments
Move a release from dev to staging to production entirely through Git: what exactly gets promoted, automated pull requests per environment, automatic updates for dev, approval and checks for production, promotion tooling, and rolling back by reverting.
Promote artifacts, not code
Build once; promote the same image digest (and chart version) from environment to environment. The only thing that changes between environments is configuration: replicas, resources, endpoints, feature flags. A rebuild for production would mean production runs an image nobody tested.
A school play: the same actors rehearse in the classroom, then in the hall, then perform on the big night. You don't swap in new actors for the final show; you move the same cast to a bigger stage.
The flow
CI: build + push digest D
└─► PR: dev ← D (auto-merge) → agent deploys to dev → tests, smoke checks
└─► PR: staging ← D (auto-merge or 1 approval) → agent deploys → soak, SLO checks
└─► PR: prod ← D (approval + checks) → agent deploys → monitoring, rollback ready
Each arrow is a pull request against the config repository that changes one line (the digest) in one environment's overlay or values file. The PR description links to the build, the test results and the previous environment's health.
Automating the first steps
| Tool | What it does | Typical scope |
|---|---|---|
| CI job (GitHub Actions, Jenkins) | Opens the promotion PR after a successful build (see "GitHub Actions — Level by Level", lesson 04) | Dev (and further with gates) |
| Argo CD Image Updater / Flux image automation | Watches the registry and writes new tags/digests back to Git | Dev, preview environments |
| Renovate | Opens PRs for new image or chart versions across repos | Third-party charts and images |
| Kargo (and similar promotion tools) | Models stages and promotes freight (image + chart + config) between them with verification | Multi-stage pipelines at scale |
Keep production changes as explicit, reviewed PRs even when everything before it is automated.
Gates before production
A promotion PR to prod should only be mergeable when:
- The same digest has been healthy in staging for an agreed soak time.
- Automated checks pass: smoke/integration tests, policy checks on the rendered manifests (Kyverno/conftest), vulnerability status.
- SLOs in staging are green (error rate, latency).
- The right approvers (CODEOWNERS for
envs/prod/) have approved. - It isn't inside a change freeze window (enforce with a check, not a memo).
Progressive delivery (lesson 07) adds automated checks during the production rollout as well.
Rolling back
$ git log --oneline -- apps/orders-api/envs/prod/
7c1d2e3 orders-api: promote 1.4.0 to prod
5b4a3f2 orders-api: promote 1.3.2 to prod
$ git revert 7c1d2e3 && git push # through a fast-track PR if prod is protected
The agent sees the previous digest in Git and deploys it. Avoid kubectl rollout undo in GitOps-managed environments: the agent would put the "bad" version straight back, because Git still says so.
Keep environments from drifting
If staging and prod overlays each accumulate their own hand edits, promotions stop meaning much. Keep overlays minimal, review diff between environments regularly, and put anything common into the base.
Try it: three environments
- Extend the layout from lesson 02 with
dev,stagingandprodoverlays, each deployed by the agent to its own namespace on a kind cluster. - Promote a new digest to dev via a CI-created PR (or by hand), then to staging, then to prod with a required approval.
- Roll back prod with
git revertand watch the agent restore the previous version. - Try
kubectl rollout undoinstead and see self-heal reapply Git's version.
Recap
- Build once, promote the digest; environments differ only by configuration.
- Each promotion is a PR to one environment in the config repo; automate dev, gate prod.
- Tools: CI-created PRs, Image Updater/Flux automation, Renovate, Kargo.
- Roll back with git revert, not kubectl.
This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.