§09

systems design

None of the failures catalogued below were caused by a person being bad at their job. Each was a system that made the wrong outcome easy and the right one effortful. These are the standards that already existed when it happened.

A component gets the narrowest authority that lets it work.
Least privilege was published in 1975, by Saltzer and Schroeder, in the same paper that gave us fail-safe defaults and complete mediation. Every breach below where one compromised credential reached everything is a shop that had fifty years to read it. principle of least privilege
One control is a single point of failure wearing a hat.
Reason's Swiss cheese model is the honest picture: every layer has holes, and an accident is the day the holes line up. The design question is never whether a layer can fail, it is how many have to fail together. the Swiss cheese model
Do not remove the fence until you know why it was built.
Chesterton's rule, from 1929, and the most frequently ignored line in software. The dormant code at Knight Capital was a fence nobody could explain and nobody removed. Both halves of that sentence did damage. Chesterton's fence
Be conservative in what you send, and stop being liberal in what you accept.
Postel's robustness principle held the early internet together and then quietly became a liability: accepting malformed input generously is how ambiguity becomes an attack surface. The IETF wrote the correction down in 2023. RFC 9413
Complex systems run in a degraded state as their normal condition.
Richard Cook's eighteen points are the shortest useful thing written about failure. The one worth memorising: catastrophe requires multiple failures, and single-point failures are not enough. how complex systems fail