Multi-cluster management · Prasanta Kumar Behera

Manage many clusters from Git: registering clusters with Argo CD, hub-and-spoke vs Argo CD per cluster, and ApplicationSets that generate Applications from cluster labels, Git folders or combinations, so a new cluster gets its platform add-ons automatically.

Registering clusters

$ argocd cluster add edge-01-admin@edge-01 --name edge-01
$ argocd cluster list

argocd cluster add creates a ServiceAccount with broad permissions in the target cluster and stores its credentials as a Secret in the argocd namespace. Declaratively, the same thing is a Secret with the label argocd.argoproj.io/secret-type: cluster:

apiVersion: v1
kind: Secret
metadata:
  name: edge-01
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: cluster
    env: prod
    region: eu
stringData:
  name: edge-01
  server: https://edge-01.example.com:6443
  config: |
    {
      "bearerToken": "<token>",
      "tlsClientConfig": { "caData": "<base64 CA>" }
    }

Keep such Secrets out of plain Git (External Secrets, Sealed Secrets or SOPS), or use cloud-native auth where available (e.g. IAM-based awsAuthConfig for EKS).

One robot now looks after many LEGO tables in different rooms. Instead of writing a separate note for every table, you write a note template: "Every table with a red sticker gets a castle and a moat." Put a red sticker on a new table, and the robot builds the castle there too.

ApplicationSets

Cluster generator: platform add-ons on every prod cluster:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: monitoring
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: [ "missingkey=error" ]
  generators:
    - clusters:
        selector:
          matchLabels:
            env: prod
  template:
    metadata:
      name: 'monitoring-{{.name}}'
    spec:
      project: platform
      source:
        repoURL: https://git.example.com/platform/gitops.git
        targetRevision: main
        path: 'addons/monitoring/overlays/{{index .metadata.labels "region"}}'
      destination:
        server: '{{.server}}'
        namespace: monitoring
      syncPolicy:
        automated: { prune: true, selfHeal: true }
        syncOptions: [ CreateNamespace=true ]

Git directory generator: one app per folder:

  generators:
    - git:
        repoURL: https://git.example.com/platform/gitops.git
        revision: main
        directories:
          - path: addons/*

Matrix: every add-on folder × every prod cluster:

  generators:
    - matrix:
        generators:
          - git:
              repoURL: https://git.example.com/platform/gitops.git
              revision: main
              directories:
                - path: addons/*
          - clusters:
              selector:
                matchLabels: { env: prod }

Other generators: list, SCM provider (one app per repo), pull request (preview environments), merge, cluster decision resource and plugin.

Hub-and-spoke or Argo CD per cluster?

	Hub (one Argo CD, many clusters)	Per cluster
Visibility	One UI for everything	One UI per cluster (or aggregate elsewhere)
Credentials	Hub holds access to all clusters	Stay local
Failure/blast radius	Hub down = no syncs anywhere	Independent
Edge / poor links	Needs reachable API servers	Works disconnected (pulls when online)

Large fleets often combine them, e.g. a hub for data-centre clusters and local Argo CD on edge sites that bootstraps from the same repo (see Edge Kubernetes & Zero-Touch Provisioning).

Try it: a fleet in kind

Create three kind clusters (mgmt, edge-01, edge-02). Install Argo CD on mgmt.
Register the edge clusters. (kind API servers listen on 127.0.0.1; for Argo CD inside kind to reach them, use each cluster's container IP on the kind Docker network in the server URL, and insecure: true in the lab config only.)
Label them env=prod and region=eu/us; create the monitoring ApplicationSet (a simple nginx folder is fine as the "add-on").
Add a fourth cluster with the same labels; confirm it gets the add-on without new YAML.
Build a matrix ApplicationSet for two add-on folders × all prod clusters, and count the generated Applications.

Going deeper: fleet safety

With ApplicationSets, one commit changes every matching cluster. Use environments (dev → canary clusters → prod), a canary label, or ApplicationSet progressive syncs (a RollingSync strategy; check its maturity and how to enable it in your version) to roll out in waves.
Set goTemplateOptions: ["missingkey=error"] so a missing label fails loudly instead of rendering an empty path.
Control what happens when a cluster leaves the selector: the ApplicationSet deletes the generated Application, and with the finalizer its resources. Consider preserveResourcesOnDeletion and the ApplicationSet policy (create-only, create-update) for safety.
Scale the application controller with sharding when managing many clusters.

Recap

Clusters are registered as Secrets in argocd (CLI or declarative), with labels that describe them.
ApplicationSets generate Applications from cluster, Git, list, SCM/PR and matrix/merge generators.
New cluster + right labels = the right add-ons automatically.
Choose hub, per-cluster or a mix; roll fleet changes out progressively.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.