§09
systems design
None of the failures catalogued below were caused by a person being bad at their job. Each was a system that made the wrong outcome easy and the right one effortful. These are the standards that already existed when it happened.
- A component gets the narrowest authority that lets it work.
- Least privilege was published in 1975, by Saltzer and Schroeder, in the same paper that gave us fail-safe defaults and complete mediation. Every breach below where one compromised credential reached everything is a shop that had fifty years to read it. principle of least privilege
- One control is a single point of failure wearing a hat.
- Reason's Swiss cheese model is the honest picture: every layer has holes, and an accident is the day the holes line up. The design question is never whether a layer can fail, it is how many have to fail together. the Swiss cheese model
- Do not remove the fence until you know why it was built.
- Chesterton's rule, from 1929, and the most frequently ignored line in software. The dormant code at Knight Capital was a fence nobody could explain and nobody removed. Both halves of that sentence did damage. Chesterton's fence
- Be conservative in what you send, and stop being liberal in what you accept.
- Postel's robustness principle held the early internet together and then quietly became a liability: accepting malformed input generously is how ambiguity becomes an attack surface. The IETF wrote the correction down in 2023. RFC 9413
- Complex systems run in a degraded state as their normal condition.
- Richard Cook's eighteen points are the shortest useful thing written about failure. The one worth memorising: catastrophe requires multiple failures, and single-point failures are not enough. how complex systems fail