EKS Anywhere on bare metal

What EKS Anywhere is

Amazon EKS Anywhere (EKS-A) is an open-source tool for creating and operating Kubernetes clusters on your own infrastructure (bare metal, vSphere, and others) using the EKS Distro Kubernetes builds and Cluster API underneath. On bare metal it uses Tinkerbell (lesson 05) to provision servers. The software is free to use; AWS sells an optional support subscription, which some features (such as curated packages) require.

The creation flow (bare metal)

admin machine (Linux, Docker, eksctl anywhere)
  1. start a kind bootstrap cluster
  2. run Tinkerbell stack + Cluster API controllers in it
  3. power-cycle servers via BMC (Rufio) → PXE → HookOS → stream OS image → reboot
  4. kubeadm-based control plane + workers form the cluster
  5. pivot: move CAPI/Tinkerbell management into the new cluster
  6. delete the bootstrap cluster → the cluster is self-managed

The admin machine must be on the same layer-2 network as the servers during creation (DHCP/PXE), and reach their BMCs.

Inputs: hardware CSV and cluster spec

hostname,bmc_ip,bmc_username,bmc_password,mac,ip_address,netmask,gateway,nameservers,labels,disk
site042-cp1,10.42.10.11,admin,*****,3c:ec:ef:00:00:01,10.42.0.11,255.255.255.0,10.42.0.1,10.42.0.1,type=cp,/dev/sda
site042-cp2,10.42.10.12,admin,*****,3c:ec:ef:00:00:02,10.42.0.12,255.255.255.0,10.42.0.1,10.42.0.1,type=cp,/dev/sda
site042-cp3,10.42.10.13,admin,*****,3c:ec:ef:00:00:03,10.42.0.13,255.255.255.0,10.42.0.1,10.42.0.1,type=cp,/dev/sda

apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
  name: site042
spec:
  kubernetesVersion: "1.31"
  clusterNetwork:
    cniConfig:
      cilium: {}
    pods:
      cidrBlocks: [ "192.168.0.0/16" ]
    services:
      cidrBlocks: [ "10.96.0.0/12" ]
  controlPlaneConfiguration:
    count: 3
    endpoint:
      host: "10.42.0.10"              # API VIP (kube-vip), unused IP on the node network
    machineGroupRef:
      kind: TinkerbellMachineConfig
      name: site042-cp
  datacenterRef:
    kind: TinkerbellDatacenterConfig
    name: site042
  # no workerNodeGroupConfigurations → a compact cluster: control-plane nodes run workloads
---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: TinkerbellDatacenterConfig
metadata:
  name: site042
spec:
  tinkerbellIP: "10.42.0.9"           # unused IP for the Tinkerbell stack
  osImageURL: "http://10.42.0.5:8080/ubuntu-2204-kube-1-31.gz"
---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: TinkerbellMachineConfig
metadata:
  name: site042-cp
spec:
  hardwareSelector:
    type: cp
  osFamily: ubuntu
  users:
    - name: ec2-user
      sshAuthorizedKeys: [ "ssh-ed25519 AAAA… ops@example" ]

Field names and where osImageURL lives have changed between EKS-A releases; generate a fresh config with your version's generate clusterconfig and compare. For a single-node cluster, set count: 1 with no worker groups.

Management and workload clusters

• A management cluster can create and upgrade other (workload) clusters, via eksctl anywhere with its kubeconfig, or by applying cluster objects (including via GitOps).
• For many small sites, decide between self-managed clusters per site (each manages itself) and a management cluster that must reach each site's hardware network, a real constraint at the edge (see lesson 02).

Upgrades

• eksctl anywhere upgrade plan cluster shows what would change; upgrade cluster applies a new kubernetesVersion or EKS-A version (bundles of component versions).
• The default is a rolling replacement: new machines are provisioned before old ones are removed, so keep spare hardware matching each selector (or reduce surge where supported).
• Recent versions support in-place upgrades on bare metal (upgradeRolloutStrategy), which avoid spare hardware; check the docs for your version and its constraints.
• Kubernetes minor versions go one at a time; read the EKS-A release notes and test on a lab site first.

Recap

• EKS Anywhere = EKS Distro + Cluster API + (on bare metal) Tinkerbell, driven by eksctl anywhere.
• Creation: bootstrap kind cluster on an admin machine (same L2) → provision → pivot → self-managed.
• Inputs: hardware CSV (labels decide roles) + cluster spec (Cluster, TinkerbellDatacenterConfig, TinkerbellMachineConfig).
• Compact (no workers) and single-node layouts are supported.
• Upgrades: rolling (spare hardware) or in-place (recent versions), one minor at a time.