A GitOps fleet on kind · Prasanta Kumar Behera

Run a small fleet on one machine: a management kind cluster with Argo CD and three workload clusters. Register them with labels, let an ApplicationSet deploy add-ons and an app to every matching cluster, roll a change out canary-first, and add a new cluster that configures itself.

One management cluster, three workload clusters, one ApplicationSet.

The brief

"You'll manage dozens of edge clusters. Show me how a new cluster gets its platform add-ons automatically, and how you'd roll a change out safely."

One head office (the mgmt cluster) and three shops (workload clusters). Each shop wears stickers ("canary", "stable"). The head office has one instruction card: "every shop with a sticker gets the standard shelves and the hello stand". A new shop with a sticker is set up automatically, and new ideas are tried in the canary shop first.

Resources needed

Resource	Detail
Linux machine	16 GB RAM recommended (four kind clusters)
Docker, kind, kubectl, argocd CLI	Local tools
A Git repo (`fleet`)	`addons/` and `apps/` folders (public repo is simplest for the lab)

Step 1: clusters

$ for c in mgmt edge-01 edge-02 edge-03; do kind create cluster --name $c; done
$ kubectl config use-context kind-mgmt
$ kubectl create namespace argocd
$ kubectl apply -n argocd --server-side --force-conflicts -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
$ kubectl -n argocd rollout status deploy/argocd-server
$ kubectl -n argocd port-forward svc/argocd-server 8080:443 &
$ argocd login localhost:8080 --username admin --password "$(argocd admin initial-password -n argocd | head -1)" --insecure

Step 2: register the workload clusters

$ for c in edge-01 edge-02 edge-03; do
    IP=$(docker inspect -f '{{.NetworkSettings.Networks.kind.IPAddress}}' $c-control-plane)
    kind get kubeconfig --name $c | sed "s#https://127.0.0.1:[0-9]*#https://$IP:6443#" > $c.kubeconfig
    argocd cluster add kind-$c --name $c --kubeconfig ./$c.kubeconfig --yes
  done
$ argocd cluster list

Label the cluster Secrets (Argo CD stores each cluster as a Secret in argocd):

$ kubectl -n argocd get secrets -l argocd.argoproj.io/secret-type=cluster
$ kubectl -n argocd label secret <edge-01-secret> env=edge wave=canary
$ kubectl -n argocd label secret <edge-02-secret> env=edge wave=stable
$ kubectl -n argocd label secret <edge-03-secret> env=edge wave=stable

Step 3: the fleet repo and ApplicationSets

fleet/
├── addons/monitoring/      # e.g. a small node-exporter DaemonSet or a Helm chart reference
└── apps/hello/             # the hello app from project 1 (public image)

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata: { name: hello, namespace: argocd }
spec:
  goTemplate: true
  goTemplateOptions: [ "missingkey=error" ]
  generators:
    - clusters:
        selector: { matchLabels: { env: edge } }
  template:
    metadata:
      name: 'hello-{{.name}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/<you>/fleet.git
        targetRevision: '{{if eq (index .metadata.labels "wave") "canary"}}main{{else}}stable{{end}}'
        path: apps/hello
      destination: { server: '{{.server}}', namespace: demo }
      syncPolicy:
        automated: { prune: true, selfHeal: true }
        syncOptions: [ CreateNamespace=true ]

Create a stable branch (or tag) in the fleet repo pointing at the last good commit. Canary clusters follow main; stable clusters follow stable. Create a second ApplicationSet for addons/monitoring the same way (without waves, or with its own).

$ kubectl apply -f appset-hello.yaml
$ argocd app list          # hello-edge-01, hello-edge-02, hello-edge-03

Step 4: roll out a change, canary first

Change apps/hello on main (new replica count or image tag), push.
Only edge-01 (canary) syncs it; check it: kubectl --context kind-edge-01 -n demo get deploy hello.
Promote: move stable to the new commit (git push origin main:stable, or merge); edge-02/03 sync.
Roll back: move stable back.

Step 5: a cluster that configures itself

$ kind create cluster --name edge-04
# register as in step 2, then:
$ kubectl -n argocd label secret <edge-04-secret> env=edge wave=stable
$ argocd app list          # hello-edge-04 appears and syncs

Also try: stop edge-02's container for a few minutes (docker stop edge-02-control-plane), push a change, start it again, and watch it catch up.

Interview talking points

Hub-and-spoke vs Argo CD per cluster, and what happens when the hub is down (see GitOps with Argo CD, lesson 05).
Labels as the fleet interface; ApplicationSet generators.
Waves and promotion through Git; blast radius.
Edge realities: disconnected sites, pull-based agents, per-site credentials (see Edge Kubernetes & Zero-Touch Provisioning, lesson 10).

Command summary

for c in mgmt edge-01 edge-02 edge-03; do kind create cluster --name $c; done
kubectl apply -n argocd --server-side --force-conflicts -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
argocd login localhost:8080 --insecure
docker inspect -f '{{.NetworkSettings.Networks.kind.IPAddress}}' <c>-control-plane
kind get kubeconfig --name <c> | sed "s#https://127.0.0.1:[0-9]*#https://$IP:6443#" > <c>.kubeconfig
argocd cluster add kind-<c> --name <c> --kubeconfig ./<c>.kubeconfig --yes
kubectl -n argocd label secret <cluster-secret> env=edge wave=canary|stable
kubectl apply -f appset-hello.yaml ; argocd app list
git push origin main:stable      # promote to stable clusters

Recap

mgmt cluster with Argo CD; workload clusters registered via their container IPs.
Labels on cluster Secrets + ApplicationSet cluster generator = automatic add-ons and apps.
Waves via different revisions per label; promote and roll back through Git.
New cluster + labels → configured automatically.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.