Method · wrap-up
Cheat sheet & self-check
Every command from this section on one page.
The eight parts
Situation | Impact, who/what/when, timeline |
Claim | The first hypothesis and the signal that suggested it |
Pivot | The evidence that changed direction |
Elimination | What was ruled out, and how |
Root cause | Mechanism + contributing factors (not 'human error') |
Fix | What stopped the bleeding |
Prevention | Changes that remove the class of failure |
Honesty | What went badly, what's still unknown |
Action items
Specific · owned · dated · verifiable | Good |
'Be more careful', 'improve monitoring' | Not action items |
Roles
Incident Commander (IC) | Coordinates, decides, delegates; doesn't debug |
Operations / technical lead | Leads investigation and mitigation |
Communications lead | Stakeholder and customer updates |
Scribe | Timeline of actions and findings |
Rhythm
Declare early; downgrade later | Cheap to cancel, expensive to be late |
Status update every 30 min (SEV1) / 60 min (SEV2) | Even when there's no news |
Update = impact · what we know · what we're doing · next update time | Fixed structure |
Antipattern → safer pattern
Liveness probe checks the database | Liveness = process health only; readiness may check dependencies |
imagePullPolicy: Always + single registry | Digests + IfNotPresent, registry mirrors/pull-through cache |
App calls the Kubernetes API per request | Cache/watch; data path independent of the control plane |
Config pushed to all clusters at once | Staged rollout with health gates |
Deep synchronous chains | Timeouts, async where possible, degrade gracefully |
