Cluster Design — Architect Track›Requirements & Proof · Cheat sheet & self-check

Requirements & Proof · wrap-up

Cheat sheet & self-check

Every command from this section on one page.

01 · Requirements → NFRs

Availability math (30-day month)

99.9% → 43.2 min/month downtime43,200 min × 0.001
99.95% → 21.6 min/month43,200 × 0.0005
99.99% → 4.32 min/month43,200 × 0.0001
Serial: A = A1 × A20.999 × 0.999 ≈ 0.998
Parallel (independent): A = 1 − (1 − A1)(1 − A2)two 99% → 99.99% (only if truly independent)

The NFR sheet

ID · category · requirement · metric · target · measurement · ownerOne measurable row per requirement
Categories: availability, latency, throughput, scalability, durability (RPO/RTO), security, operability, costCover them all

02 · PoC from scratch

Baseline tools

sonobuoy run --mode=certified-conformanceKubernetes conformance tests
kube-burner init -c <config>Control-plane / object churn load
iperf3 -c <pod-ip> -P 4 -t 30Pod-to-pod network throughput
fio --name=etcd --rw=write --ioengine=sync --fdatasync=1 --bs=2300 --size=22m --directory=/var/lib/etcd-testfsync latency like etcd's WAL
k6 run load.jsApplication load test

PoC discipline

Success criteria written before startingPass/fail per criterion
Weighted scoring matrixCriteria × weight × score per option
Time-box (e.g. 3–4 weeks)A PoC is not a pilot

03 · Proving stability

Stability tests

Soak: realistic load for 48–72 hLeaks, slow degradation, log/disk growth
Peak + burst profile (k6 stages)Behaviour at NFR-03 load
Kill a node / pod / etcd member under loadRecovery time and error budget spent
Minor upgrade under loadNo failed requests (NFR-07)

Deliverables

Evidence packMethod, environment, raw results, graphs, issues found and fixed
Exec one-pagerRecommendation, confidence, risks, cost, next steps
Risk registerRisk · likelihood · impact · mitigation · owner

04 · Sizing & capacity math

The chain

Demand = Σ (replicas × requests) at peak (+ daemonsets per node)What workloads reserve
Allocatable = capacity − system/kube reserved − eviction thresholdWhat a node can offer
Usable per node = allocatable × target utilisation (e.g. 70%)Room for bin-packing and bursts
Nodes = ceil(demand ÷ usable per node), for CPU and memory; take the largerBase count
Survive losing 1 of R racks: nodes × R ÷ (R − 1)Failure-domain headroom

Limits to check

max pods per node (kubelet default 110)Pod density
pod CIDR per node (e.g. /24 → 256 addresses)IPAM (lesson 06)
Upstream tested limits: 5,000 nodes, 150,000 pods per cluster, 110 pods per nodeFar above most designs, but real