Normal Accidents. Some failures are built into the design.
Front
Normal Accidents
Some failures are built into the design
Failures built into a system rather than caused by somebody's error.
Back
Charles Perrow studied Three Mile Island and concluded the accident was not a fluke. When a system is complex, so that parts interact in ways nobody predicted, and tightly coupled, so that there is no slack to absorb a fault, serious accidents stop being surprises. Not frequent. Expected. The design produces them.
- Adding a safety device adds complexity, and complexity adds new interactions. Some safety systems have gone on to cause the exact accident they were fitted to prevent.
- The practical answer is slack and simplicity: buffers, spare capacity, and fewer hidden dependencies. Efficiency and resilience pull against each other.
Clearfield and Tilcsik, Meltdown (2018)
Keep exploring
Because you read Normal Accidents.
The Investigation
Ask what failed, not who failed
The method of learning from an accident instead of punishing it.
The Backup
Three copies, two kinds, one elsewhere
The rule that turns a lost device into a nuisance instead of a disaster.
Antibiotic Resistance
A cure being used up in slow motion
Bacteria learning to survive the medicines that used to kill them.