subscribe for upcoming articles and periodical summaries of the most read ones

Random Thoughts on Leadership & Technology

Building Codes Are Written in Ash

fire-code

Every time I am in London the sheer number of fire exit signs always grabs my attention. The fire code is the only book in your office written entirely by the dead.

It does not read that way. It reads like the most boring document ever committed to paper - door swing directions, flame spread indices, maximum travel distance to an exit, the required width of a corridor measured in inches. No names. No photographs. No flowers. Just load ratings and stair pitch, set in a typeface chosen by a committee that hated joy.

And yet nearly every clause is a headstone with the name filed off.

The provenance of boring

Exit doors must swing outward in the direction of egress. That sentence is 146 garment workers in Manhattan, March 1911, pressed against doors that opened inward and were locked to stop pilferage. The Triangle Shirtwaist fire did not teach us anything about combustion. It taught us about the physics of a crowd, which is that a panicked human being is not a person but a fluid, and fluids do not pull.

Panic hardware - the horizontal bar that opens a door when a body simply falls against it - is the Iroquois Theatre, Chicago, two days after Christmas 1903, roughly six hundred people dead in under fifteen minutes in a building advertised as absolutely fireproof. The device was later commercialised by a partnership including a hardware salesman named Carl Prinzler, who had a ticket to that matinee and did not use it. His company's name still appears on doors in your building. The lettering is small. It fits nicely above a crash bar that exists because somebody who nearly died decided that pulling a handle was too much to ask of the dying.

Revolving doors must be flanked by conventional outward-swinging doors. That is Cocoanut Grove, Boston, November 1942, where a nightclub's single fashionable revolving entrance jammed with bodies and 492 people did not get out.

Sprinklers in small assembly occupancies, crowd managers, restrictions on cheap acoustic foam - The Station, Rhode Island, 2003, one hundred dead in roughly ninety seconds of flame front.

Combustible cladding restrictions on high rise residential - Grenfell Tower, 2017, seventy two people, a building wrapped in something that performed beautifully on a spreadsheet.

The pattern is monotonous once you see it. The code is not a set of opinions about buildings. It is a scar tissue map of the last hundred and fifty years, transcribed into the passive voice by people who wanted very badly for it to sound like engineering rather than grief.

Your runbook has the same author

Here is the part that ruins your afternoon.

The change freeze. The two-approver rule. The deploy gate that runs for forty minutes and fails on something cosmetic. The line in the runbook that says, in tone of quiet menace, do not run this with the force flag even if it seems fine. The requirement that migrations ship in a separate release from the code that depends on them. The prohibition on holding both production database credentials and deploy authority in the same pair of hands. The insistence that a rollback plan be written down before launch and not, as tradition prefers, improvised at 2 am by whoever answers the page.

Every one of those has a body count. It is denominated in dollars and minutes rather than people, which makes it easier to forget but no less real.

Knight Capital lost something in the neighbourhood of 440 million dollars in forty five minutes in 2012 because a deploy reached seven of eight servers and dormant code woke up on the eighth. That is the entire ancestry of your annoying rule about verifying deployment across every host in the fleet. Amazon took a large portion of the internet offline in 2017 when an engineer ran a legitimate playbook command with a fat-fingered argument and removed more capacity than intended. The remediation was not a stern memo. It was tooling that refuses to remove capacity below a threshold, which is to say a door that closes itself. GitLab deleted a production database directory in 2017 and then discovered that five separate backup mechanisms had all quietly failed, which is why somebody on your team is required, tediously, to restore from backup on a schedule and prove it worked.

Runbooks, review checklists, deploy gates, the weirdly specific alert threshold, the field in the incident template nobody likes filling in. Identical provenance. Different ash.

Enter the reformer

There is a genre of engineer, and more dangerously a genre of executive, who encounters a checklist the way a tourist encounters a fence in an empty field - as an obstacle with no history.

She arrives with a mandate to reduce friction. Reduce friction is a beautiful phrase. It means, when you hold it up to the light, that she intends to sand down the scar tissue and observe what happens. She will point out, correctly, that the gate adds forty minutes. She will note that we are a high-trust team, which is what people say roughly eleven months before an outage teaches them what trust costs. She will observe that no incident has occurred in eighteen months, treating the absence of fire as evidence against the smoke detector.

Friction is what a brake feels like from the perspective of the wheel.

The reformer is not stupid and this is not a defence of every rule ever written. It is an observation about asymmetry. The cost of a gate is visible, recurring, and paid by the person who resents it. The cost of removing the gate is invisible, occasional, catastrophic, and paid by somebody else - frequently somebody who has not been hired yet. Institutional memory has a half-life of roughly eighteen months, which is also the median tenure of the person who wrote the runbook. The rule outlives the reason, then the reason's absence is used as an argument against the rule.

A rule without a story is a rule without a lawyer.

Keep the incident attached to the rule

The fix is unglamorous and nearly free. Every rule carries its origin in the text. Not in a wiki three links away that returns a permissions error. In the line itself.

Not this - all migrations require a separate release.

This - all migrations require a separate release. See the incident of 14 March 2023, four hours of write downtime, customer data reconciled by hand for nine days.

Name rules after their incidents, not their mechanisms. Nobody remembers change control policy section three subsection b. Everybody remembers the Tuesday Cascade rule, and more importantly, everybody knows how to find out what the Tuesday Cascade was. Postmortems should emit rules and rules should point back at postmortems, bidirectionally, so that the graph is walkable in both directions by a person with a grudge and forty minutes to kill.

Then the crucial move, the one that separates this from mere conservatism.

The point of the story is that it makes the rule arguable

Not every scar corresponds to a wound you still have.

Some rules protect a system that was decommissioned during the reorg two before last. Some encode a failure mode that the platform team made structurally impossible in 2021. Some were never scars at all - they are the fossilised embarrassment of a vice president who once got asked a hard question in a board meeting and responded with process. Cargo cult is real. Ritual accumulates. Fire codes get pruned too, by committees, on cycles, with hearings, precisely because the alternative is a document that eventually forbids gas lighting in buildings that no longer have gas.

So the reason to keep the incident attached is not that it renders the rule sacred. It is that it renders the rule falsifiable.

An undocumented rule cannot be defended and cannot be killed. It can only be obeyed by the anxious and deleted by the confident, and neither of them will have evidence. A documented rule can be interrogated - does this failure mode still exist, is this control still the cheapest place to catch it, has the thing we were afraid of been engineered out of existence. Sometimes the honest answer is yes, remove it, we built a wall where we used to post a warning sign. That is a good day. That is what pruning looks like when it is done by adults.

Which suggests the ranking. A rule is a bet on human memory and it pays out badly under stress. A constraint is a bet on physics and it pays out always. Fire codes learned this the hard way and graduated from please keep this door closed to a door that closes itself, unbidden, whether or not anyone is watching or sober or new. Where you can convert a checklist item into a self-closing door, do it, and retire the checklist item with full honours and a link to the fire that made it.

The museum you already work in

Any organisation that survives long enough becomes a museum of its own worst days. The artefacts are all there - the freeze calendar, the mandatory second reviewer, the timeout value that is suspiciously not a round number. The only real question is whether the exhibits have labels.

Without labels, the building is just a maze of inconvenient doors, and eventually someone helpful props one open to improve airflow.

So write the ash into the margin. Put the date in the comment. Name the rule after the fire. Not to make it holy, and not to weaponise other people's bad Tuesdays against anyone who wants to move quickly, but because a rule that remembers why it exists is a rule that can be honestly retired, and a rule that has forgotten will simply be deleted by the next person who finds it annoying.

They will be very polite about it. They will call it reducing friction. And they will be standing in the same room, in the same building, holding the same door.