Lesson 02 of 6 · CI/CD pipelines
The same pipeline, GitOps style with Argo CD
Rebuild project 1's pipeline the GitOps way: Jenkins still tests and builds the image, but instead of running kubectl it commits the new image tag to a GitOps repository, and Argo CD inside the cluster pulls and syncs it. Compare both designs and know when to use which.
The brief
"Now do the same with Argo CD. What changes, and why is it better or worse?"
You reuse the cluster, registry and Jenkins from project 1 (or any cluster and CI). Only the deploy step changes.
In project 1, the factory robot walked into the shop and rearranged the shelves itself. Now the robot only updates the shop's display plan (the GitOps repo). The shop has its own assistant (Argo CD) who reads the plan every few minutes and arranges the shelves to match. The robot never needs the shop's keys.
Resources needed
| Resource | Detail |
|---|---|
| Cluster from project 1 | Or any Kubernetes cluster |
| Registry | GHCR as before (regcred in namespace demo) |
| Jenkins from project 1 | Or GitHub Actions (CI only builds and commits) |
| App repo | hello-k8s: code, Dockerfile, Jenkinsfile |
| GitOps repo | hello-gitops: Kustomize base + overlay |
| SSH deploy keys | One read-only key for Argo CD, one write key for Jenkins on the GitOps repo |
| Tools | argocd CLI; kustomize on the Jenkins VM |
The flow
- Push code to the app repo → webhook → Jenkins.
- Jenkins tests, builds and pushes
hello-k8s:<sha>. - Jenkins clones the GitOps repo, runs
kustomize edit set image, commits and pushes. - Argo CD notices the new commit (polling every few minutes, or a webhook).
- Argo CD syncs: the Deployment gets the new image; pods roll.
- You check in the Argo CD UI/CLI and with curl.
Step 1: install Argo CD
$ kubectl create namespace argocd
$ kubectl apply -n argocd --server-side --force-conflicts \
-f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
$ kubectl -n argocd rollout status deploy/argocd-server
$ kubectl -n argocd patch svc argocd-server -p '{"spec":{"type":"NodePort"}}' # lab access
$ kubectl -n argocd get svc argocd-server # note the HTTPS NodePort
$ argocd admin initial-password -n argocd
$ argocd login <node-public-ip>:<https-nodeport> --username admin --insecure
(Allow that NodePort from your IP in the security group. In real setups, expose Argo CD through an Ingress with TLS and SSO; see GitOps with Argo CD, lessons 02 and 07.)
Step 2: the GitOps repository
hello-gitops/
└── apps/hello/
├── base/
│ ├── deployment.yaml # as in project 1, image: hello (placeholder name)
│ ├── service.yaml
│ └── kustomization.yaml
└── overlays/prod/
└── kustomization.yaml
# apps/hello/base/kustomization.yaml
resources: [ deployment.yaml, service.yaml ]
# apps/hello/overlays/prod/kustomization.yaml
namespace: demo
resources: [ ../../base ]
images:
- name: hello
newName: ghcr.io/<github-user>/hello-k8s
newTag: "3f9c2a1"
In base/deployment.yaml, set image: hello (the name Kustomize replaces).
Step 3: connect Argo CD and create the Application
$ ssh-keygen -t ed25519 -N '' -f argocd_ro # add argocd_ro.pub as a read-only deploy key on hello-gitops
$ argocd repo add git@github.com:<github-user>/hello-gitops.git --ssh-private-key-path ./argocd_ro
# hello-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: hello
namespace: argocd
spec:
project: default
source:
repoURL: git@github.com:<github-user>/hello-gitops.git
targetRevision: main
path: apps/hello/overlays/prod
destination:
server: https://kubernetes.default.svc
namespace: demo
syncPolicy:
automated: { prune: true, selfHeal: true }
syncOptions: [ CreateNamespace=true ]
$ kubectl apply -f hello-app.yaml
$ argocd app get hello # Synced, Healthy
If project 1's Jenkins already created objects in demo, Argo CD takes them over (same names). Remove the Deploy stage from project 1's pipeline first so two tools don't fight.
Step 4: change Jenkins to commit instead of deploy
Add a write deploy key to hello-gitops, store the private key in Jenkins as an "SSH Username with private key" credential (gitops-key), install kustomize on the Jenkins VM, and replace the Deploy stage:
stage('Update GitOps repo') {
steps {
withCredentials([sshUserPrivateKey(credentialsId: 'gitops-key', keyFileVariable: 'KEY')]) {
sh '''
export GIT_SSH_COMMAND="ssh -i $KEY -o StrictHostKeyChecking=accept-new"
rm -rf hello-gitops && git clone git@github.com:<github-user>/hello-gitops.git
cd hello-gitops/apps/hello/overlays/prod
kustomize edit set image hello=$IMAGE:$TAG
git config user.name "jenkins" && git config user.email "jenkins@example.com"
git commit -am "hello: deploy $TAG" && git push origin main
'''
}
}
}
Jenkins no longer has any Kubernetes credentials: delete the kubeconfig-demo credential and the jenkins-deployer token.
Step 5: check the app
$ git commit -am "hello v3" && git push # in the app repo
# Jenkins: build → push → commit to hello-gitops
$ argocd app get hello # new revision, Synced, Healthy
$ argocd app history hello
$ kubectl -n demo get deploy hello -o jsonpath='{.spec.template.spec.containers[0].image}'
$ curl -s http://<node-public-ip>:30080 | head -5
Argo CD polls every few minutes by default; add a GitHub webhook to Argo CD (https://<argocd>/api/webhook) for instant syncs.
Self-heal demo: kubectl -n demo scale deploy hello --replicas=5 → Argo CD sets it back to 2.
Rollback: git revert the tag commit in hello-gitops and push.
Jenkins-push vs GitOps-pull
| Project 1 (Jenkins deploys) | Project 2 (Argo CD syncs) | |
|---|---|---|
| Cluster credentials in CI | Yes (limited SA) | No |
| Drift correction | No | Yes (self-heal) |
| What's deployed where | Jenkins history | Git history of the GitOps repo |
| Rollback | Re-run/undo | git revert |
| Moving parts | Fewer | One more (Argo CD) |
| Multi-cluster | CI needs access to each | Each cluster (or one Argo CD) pulls |
Interview answer in one line: "Jenkins for building and testing; Argo CD for deploying, so the cluster pulls its desired state from Git and CI never holds cluster credentials."
Command summary
kubectl create namespace argocd
kubectl apply -n argocd --server-side --force-conflicts -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl -n argocd patch svc argocd-server -p '{"spec":{"type":"NodePort"}}'
argocd admin initial-password -n argocd ; argocd login <ip>:<port> --insecure
ssh-keygen -t ed25519 -N '' -f argocd_ro ; argocd repo add git@github.com:<you>/hello-gitops.git --ssh-private-key-path ./argocd_ro
kubectl apply -f hello-app.yaml ; argocd app get hello
# Jenkins update stage
kustomize edit set image hello=$IMAGE:$TAG ; git commit -am "hello: deploy $TAG" ; git push
# Check / roll back
argocd app history hello ; kubectl -n demo get pods ; curl http://<node-ip>:30080
git revert <sha> && git push # in hello-gitops
Recap
- CI builds and commits a tag; Argo CD pulls and syncs; CI never touches the cluster.
- Two repos: app (code) and GitOps (desired state); read-only key for Argo CD, write key for CI.
- Self-heal fixes drift; git revert rolls back; Git history shows what's deployed.
This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.