In partnership with
Automating the routine can quietly de-skill your best responders. This is how to keep the edge on purpose.
In this drop
The point: Automating triage removes the small daily reps that kept your team sharp, so the next novel incident finds them rusty.
Why it matters: The cost is invisible on your dashboards. It only shows up on the one incident that matters most, weeks later.
Try this next week: Run one live incident drill the automation cannot help with. A novel fault, no runbook, your best people, cold.
The point
A while back I watched one of the best incident teams I have worked with get quietly worse over about six weeks, and everything we did to them was, on paper, an improvement. We automated triage. The routine incidents got caught and handled, fewer people were woken, the numbers looked wonderful.
Then a genuinely novel fault arrived, the kind that is in no runbook. The team that used to eat those stalled. They reached for the automation, which had never seen it either, and the humans were slow and tentative. Nothing about them had got worse as people. We had taken away their reps.
People in dangerous work drill the basics far more than seems necessary, because the reps buy down the risk on the day it is real. Every routine incident used to be a small rep. We automated the boring reps away, and the edge went with them.
Reality check
What changes everything: the number to watch is not how well you handle the incidents you automated. It is how well you handle the ones you could not.
One proof
Lisanne Bainbridge named this in 1983, in a paper called the Ironies of Automation: the more you automate a process, the more the operator's manual skill decays, so the human is least practised at exactly the moment automation fails and hands the problem back. Field note, from one team I worked with and not an industry benchmark: once we automated the routine triage, the share of incidents the humans actually handled fell by roughly a third within a quarter, and the first genuinely novel fault after that took them noticeably longer than their old form. The people had not changed. Their reps had.
Where this breaks
This breaks if you read it as an argument against automation. It is not. Automation is worth having. The failure is treating it as free and never replacing the reps it absorbs. Keep the automation; add the drills back deliberately.
Try this next week
Schedule one live incident drill, and make it one the automation cannot help with.
Pick a novel fault that is not in the runbook. Get your best responders in a room, real or virtual, and let them work it cold: no script, no automated summary.
Notice how rusty they have got. That is the signal. Then make the drill a habit, because the routine work is no longer sharpening them for you.
Three links I am watching
Lisanne Bainbridge, 'Ironies of Automation' (Automatica, 1983): the original and still the clearest statement of the skill-decay problem.
Google SRE, the 'Wheel of Misfortune' disaster role-play: a lightweight way to give a team reps on incidents that have not happened yet.
Signal Drop 21, 'You're Going To Have To Suffer Today': the companion on rehearsing the bad night before it arrives.
One question for you
When did a novel incident last find your team rusty, and had automation quietly stopped giving them the reps that used to keep them sharp?
Allan
|
Everything in one place New here? Start with the free chapter. | ||||
More from Metrics & Mayhem
|
Stop doomscrolling for tech news
TLDR is the free daily email that does the reading for you. Each issue summarizes the most interesting stories in startups, tech, and programming, curated by ex-Google and Anthropic engineers, so you get the signal without the noise.
It's one email instead of 12 open tabs. A 5-minute read with your morning coffee, and you walk in already caught up.
Tech is just the start. We also cover AI, marketing, dev, and more, so you can follow the topics that matter to your work.
Free, daily, and read by 7M+ subscribers. Subscribe and get your mornings back.