Prasanta Kumar Behera
Kubernetes Administration — Level by Level›09 · Build a cluster with kubeadm
Learning Hub / Kubernetes & Platform / Kubernetes Administration — Level by Level

Lesson 09 of 32 · Level 2 — Operator

Build a cluster with kubeadm

Build a real multi-node cluster from plain Linux machines with kubeadm, and understand every step it takes, from preflight checks to certificates to joining nodes.

Practitioner
Key wordskubeadmcontainerdcontrol-plane endpointbootstrap tokenjoinCNIPKI
cp1 · kubeadm init preflight swap, ports, runtime 1 certs + kubeconfigs /etc/kubernetes/pki 2 static pods apiserver, etcd, scheduler, cm 3 token + addons CoreDNS, kube-proxy 4 Install CNI Flannel / Calico / Cilium then workers w1 kubeadm join w2 kubeadm join token + CA hash nodes turn Ready once the CNI runs
kubeadm init on the first node, a network plugin, then workers join.

What kubeadm is (and isn't)

kubeadm turns a few Linux machines into a Kubernetes cluster. It generates certificates, writes the control-plane static pod manifests, starts etcd, and gives you join commands. It does not install a container runtime, a network plugin, a load balancer or monitoring. Those are your choices.

Almost every "vanilla" cluster, and many products underneath (including kind and EKS Anywhere), use kubeadm. Understanding it is understanding how clusters are born.

Building a cluster is like opening a new school. kubeadm is the principal on day one: prints the ID cards (certificates), sets up the head office (control plane), and hands teachers a joining letter (the join command) so they can start work. But the principal doesn't build the corridors between classrooms. That's the network plugin you add yourself.

The lab: three Linux machines

kind can't teach you kubeadm (it hides it), so bring three Linux machines. How you create them is up to you; use whatever you have:

  • VMs on a Linux host: KVM/libvirt (virt-manager, virt-install, or Terraform, see Terraform & Infrastructure as Code, lesson 06)
  • VMs on Windows or macOS: Hyper-V, VirtualBox, VMware Workstation/Fusion, or a similar tool
  • Cloud instances: three small VMs in one network (with a firewall allowing traffic between them)
  • Physical machines: three spare Linux boxes or mini PCs on the same network
Machine Role Minimum
cp1 Control plane 2 vCPUs, 2 GB RAM, 15 GB disk
w1, w2 Workers 2 vCPUs, 2 GB RAM, 15 GB disk

Requirements, whatever the platform:

  • Ubuntu 22.04 or 24.04 (the commands below use apt; other distributions work with their package manager).
  • Fixed IP addresses (static, or DHCP reservations) that can reach each other on all ports (or at least the ports in the Kubernetes docs), plus internet access for packages and images.
  • Unique hostnames, MAC addresses and product_uuid on every machine. If you clone VMs, check this (cat /sys/class/dmi/id/product_uuid, ip link) and regenerate /etc/machine-id if needed; duplicates cause confusing failures.
  • SSH access to all three.

Quick check on each machine:

$ hostnamectl --static && ip -4 addr show | grep inet && cat /sys/class/dmi/id/product_uuid

Step 1: prepare every node

Run this on all three nodes.

Swap off and kernel settings:

$ sudo swapoff -a
$ sudo sed -i '/ swap / s/^/#/' /etc/fstab
$ printf 'overlay\nbr_netfilter\n' | sudo tee /etc/modules-load.d/k8s.conf
$ sudo modprobe overlay && sudo modprobe br_netfilter
$ printf 'net.bridge.bridge-nf-call-iptables = 1\nnet.bridge.bridge-nf-call-ip6tables = 1\nnet.ipv4.ip_forward = 1\n' | sudo tee /etc/sysctl.d/k8s.conf
$ sudo sysctl --system

Container runtime (containerd) with the systemd cgroup driver:

$ sudo apt-get update && sudo apt-get install -y containerd
$ sudo mkdir -p /etc/containerd
$ containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
$ sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
$ sudo systemctl restart containerd

kubeadm, kubelet and kubectl from the official package repository (replace v1.31 with the minor version you want):

$ sudo apt-get install -y apt-transport-https ca-certificates curl gpg
$ sudo mkdir -p -m 755 /etc/apt/keyrings
$ curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
$ echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
$ sudo apt-get update && sudo apt-get install -y kubelet kubeadm kubectl
$ sudo apt-mark hold kubelet kubeadm kubectl

Why each step matters

  • Swap off: by default the kubelet refuses to start with swap enabled, because memory limits and eviction assume no swap.
  • br_netfilter + sysctl: lets iptables see bridged pod traffic and lets the node route packets between pods.
  • SystemdCgroup = true: the kubelet and the runtime must agree on the cgroup driver.
  • apt-mark hold: Kubernetes upgrades must be deliberate (lesson 12), never a side effect of apt upgrade.

Step 2: initialise the control plane

On cp1 only. The --control-plane-endpoint should be a DNS name or load-balancer address that will always point at your API servers. For the lab, add cp1's IP to /etc/hosts on every node as k8s-api.

$ sudo kubeadm init \
    --control-plane-endpoint k8s-api:6443 \
    --pod-network-cidr 10.244.0.0/16 \
    --upload-certs
[preflight] Running pre-flight checks
[certs] Generating "ca" certificate and key
...
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!
...
You can now join any number of control-plane nodes by running the following command on each as root:
  kubeadm join k8s-api:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:<hash> --control-plane --certificate-key <key>
...
Then you can join any number of worker nodes by running the following on each as root:
  kubeadm join k8s-api:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:<hash>

Set up kubectl for your user, exactly as the output tells you:

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

What kubeadm init actually did

Phase What happened
preflight Checked CPUs, memory, swap, ports, runtime
certs Created the cluster CA and every certificate in /etc/kubernetes/pki
kubeconfig Wrote kubeconfigs for admin, kubelet, controller-manager, scheduler
control-plane / etcd Wrote static pod manifests into /etc/kubernetes/manifests
upload-certs Stored control-plane certificates (encrypted) in a Secret, for joining more control-plane nodes
bootstrap-token Created a short-lived token so new nodes can join
addons Installed CoreDNS and kube-proxy

Step 3: install a network plugin

$ kubectl get nodes
NAME   STATUS     ROLES           AGE   VERSION
cp1    NotReady   control-plane   2m    v1.31.x
$ kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
$ kubectl get nodes
NAME   STATUS   ROLES           AGE   VERSION
cp1    Ready    control-plane   3m    v1.31.x

Flannel's default pod network is 10.244.0.0/16, which is why we passed that CIDR to kubeadm init. Calico and Cilium are common production choices (see Networking Deep Dive).

Step 4: join the workers

On w1 and w2, run the worker kubeadm join command from the init output. Lost it, or it's been more than 24 hours? Generate a fresh one on cp1:

$ kubeadm token create --print-join-command
kubeadm join k8s-api:6443 --token 7q1p2x.9dk3m5n8b2v4c6z0 --discovery-token-ca-cert-hash sha256:<hash>

$ kubectl get nodes -o wide
NAME   STATUS   ROLES           VERSION    INTERNAL-IP    CONTAINER-RUNTIME
cp1    Ready    control-plane   v1.31.x    10.10.0.10     containerd://1.7.x
w1     Ready    <none>          v1.31.x    10.10.0.11     containerd://1.7.x
w2     Ready    <none>          v1.31.x    10.10.0.12     containerd://1.7.x

The discovery hash protects you

--discovery-token-ca-cert-hash lets the joining node verify it's talking to your cluster's CA, not an impostor. Never skip it with --discovery-token-unsafe-skip-ca-verification outside a throwaway lab.

Certificates you now own

$ sudo kubeadm certs check-expiration
CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY
admin.conf                 Sep 27, 2027 09:12 UTC   364d            ca
apiserver                  Sep 27, 2027 09:12 UTC   364d            ca
apiserver-etcd-client      Sep 27, 2027 09:12 UTC   364d            etcd-ca
...
CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME
ca                      Sep 25, 2036 09:12 UTC   9y
etcd-ca                 Sep 25, 2036 09:12 UTC   9y

Leaf certificates last one year. kubeadm upgrade apply renews them, which is one more reason to upgrade regularly. A cluster left alone for a year will have its API server certificate expire, and everything stops talking.

Try it: break and rebuild a node

  1. On w2: sudo kubeadm reset -f, then remove leftover CNI config with sudo rm -rf /etc/cni/net.d.
  2. On cp1: kubectl get nodes (w2 goes NotReady), then kubectl delete node w2.
  3. Generate a new join command and re-join w2. Watch it return to Ready.

Going deeper: production kubeadm

  • Drive kubeadm from a config file (kubeadm config print init-defaults > kubeadm.yaml) and keep it in Git. That's reviewable and repeatable.
  • For HA, put a load balancer or keepalived VIP in front of the API servers before running init, and point --control-plane-endpoint at it (lesson 22).
  • Air-gapped installs: kubeadm config images list shows the images to pre-load into your internal registry, and imageRepository in the config points kubeadm at it.
  • Track certificate expiry in monitoring (for example, the API server's client-certificate expiry metrics, or a simple cron around kubeadm certs check-expiration).

Recap

  • Prepare every node: swap off, kernel modules and sysctls, containerd with systemd cgroups, pinned kube packages.
  • kubeadm init on the first control plane → install a CNI → kubeadm join workers.
  • Use a stable control-plane endpoint from day one.
  • Next: build nodes from images instead of by hand (lesson 10), and give bare-metal Services real IPs with MetalLB (lesson 11).
  • kubeadm leaf certificates last one year: upgrade or renew before they expire.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.