Engineering notesView as Markdown ↗

Freeze the baseline, then burn it down

In plain terms

Some rules for Biloh's code were already broken in many places when the rule was written. Rather than keep a check that always fails and gets ignored, the known breaches are listed and fenced off. Any new breach fails straight away, and the list can only shrink. One list went from 209 entries to none.

A hard-fail check that has failed since the day it was written enforces nothing — everyone learns to ignore it. Convert it to a committed, provenance-stamped list of today's offenders: the check goes green at the current debt and red the moment a new offender appears, the list can only shrink, and spec-tested honesty invariants stop it drifting into fiction.

The decorative check

Every codebase accumulates rules it believes in but does not meet: "every route validates input", "every tool has tests", "every handler is instrumented". Write that rule as a hard-fail check against a codebase with 92 existing violations and you get a check that is red on day one, red every day after, and therefore ignored — including the day violation 93 arrives. A permanently-failing check is not strictness; it is decoration.

The conversion

Freeze today's measured debt into a committed baseline file:

{
  "seededAt": "2026-08-24",
  "seededFromCommit": "abc1234",
  "seededCount": 92,
  "offenders": ["app/api/route-one", "app/api/route-two", "..."]
}

Then rewrite the check: an offender in the list passes (fenced debt); an offender not in the list fails, immediately, with an exact message contract:

FREEZE: route "<path>" has no input validation and is not in the
frozen baseline. Validate it in this commit — the baseline only shrinks.

The check is now green at today's debt and red the moment anyone adds a new violation. Enforcement starts today, not after the backlog is cleared.

The honesty invariants

A frozen list rots unless it is policed. Spec-test these invariants — and demonstrate each red before trusting it green:

  • Entries are unique and sorted — diffs stay reviewable, duplicates can't pad the count.
  • The count never exceeds seededCount — the list may only shrink from its seeded size.
  • Every entry still offends. When an offender is fixed, its entry must be removed in the same commit — otherwise the list silently becomes a list of things that used to be true, and the "shrink-only" claim is fiction.
  • The exact failure message is asserted, so the check cannot degrade into a vague warning.

When the measurement itself improves

Tightening detection can reveal that the seeded count was wrong. On this platform, sharpening a coverage scan from a substring match to an exact quoted-name match exposed six tools that had been counted as covered but were not. The honest move is to correct the seeded count upward with a written note in the file recording why — not to quietly absorb the difference. A baseline you cannot trust the provenance of is worse than none; the correction note is what keeps "the list only shrinks" a claim about reality.

Debt fenced is not debt forgiven

Each frozen baseline gets a named burn-down work package in the backlog. The freeze buys enforcement now; the work package owns getting the list to zero. On this platform the per-tool test-coverage baseline went from 209 seeded offenders to an empty list, at which point the check became the unconditional rule it always claimed to be — "no tool ships without tests" — and moved into the pre-push hook, where a breach cannot even leave a developer's machine.

Where this differs from a lint suppression file

Superficially this is .eslintignore with ceremony. The differences are exactly the parts that matter: provenance (who measured this, from which commit), shrink-only enforcement (a spec test, not a convention), the still-offends invariant (fixed entries must leave), and a named owner for the burn-down. Suppression files have none of these, which is why they only ever grow.

Related

Last updated 2026-08-28