# Freeze the baseline, then burn it down

> A hard-fail check that has failed since the day it was written enforces nothing — everyone learns to ignore it. Convert it to a committed, provenance-stamped list of today's offenders: the check goes green at the current debt and red the moment a new offender appears, the list can only shrink, and spec-tested honesty invariants stop it drifting into fiction.

URL: https://biloh.com.au/docs/engineering-notes/freeze-the-baseline-then-burn-it-down
Category: Engineering notes | Audience: builder | Updated: 2026-08-28

## The decorative check

Every codebase accumulates rules it believes in but does not meet: "every route validates input", "every tool has tests", "every handler is instrumented". Write that rule as a hard-fail check against a codebase with 92 existing violations and you get a check that is red on day one, red every day after, and therefore **ignored** — including the day violation 93 arrives. A permanently-failing check is not strictness; it is decoration.

## The conversion

Freeze today's measured debt into a committed baseline file:

```json
{
  "seededAt": "2026-08-24",
  "seededFromCommit": "abc1234",
  "seededCount": 92,
  "offenders": ["app/api/route-one", "app/api/route-two", "..."]
}
```

Then rewrite the check: an offender **in the list** passes (fenced debt); an offender **not in the list** fails, immediately, with an exact message contract:

```
FREEZE: route "<path>" has no input validation and is not in the
frozen baseline. Validate it in this commit — the baseline only shrinks.
```

The check is now green at today's debt and red the moment anyone adds a new violation. Enforcement starts today, not after the backlog is cleared.

## The honesty invariants

A frozen list rots unless it is policed. Spec-test these invariants — and demonstrate each red before trusting it green:

- **Entries are unique and sorted** — diffs stay reviewable, duplicates can't pad the count.
- **The count never exceeds `seededCount`** — the list may only shrink from its seeded size.
- **Every entry still offends.** When an offender is fixed, its entry must be removed in the same commit — otherwise the list silently becomes a list of things that used to be true, and the "shrink-only" claim is fiction.
- **The exact failure message is asserted**, so the check cannot degrade into a vague warning.

## When the measurement itself improves

Tightening detection can reveal that the seeded count was wrong. On this platform, sharpening a coverage scan from a substring match to an exact quoted-name match exposed six tools that had been counted as covered but were not. The honest move is to **correct the seeded count upward with a written note in the file** recording why — not to quietly absorb the difference. A baseline you cannot trust the provenance of is worse than none; the correction note is what keeps "the list only shrinks" a claim about reality.

## Debt fenced is not debt forgiven

Each frozen baseline gets a named burn-down work package in the backlog. The freeze buys enforcement now; the work package owns getting the list to zero. On this platform the per-tool test-coverage baseline went from 209 seeded offenders to an empty list, at which point the check became the unconditional rule it always claimed to be — "no tool ships without tests" — and moved into the pre-push hook, where a breach cannot even leave a developer's machine.

## Where this differs from a lint suppression file

Superficially this is `.eslintignore` with ceremony. The differences are exactly the parts that matter: provenance (who measured this, from which commit), shrink-only enforcement (a spec test, not a convention), the still-offends invariant (fixed entries must leave), and a named owner for the burn-down. Suppression files have none of these, which is why they only ever grow.

## Related

- [A failure ceiling that only goes down](/docs/engineering-notes/a-failure-ceiling-that-only-goes-down)
- [Five ways a test suite lies to you](/docs/engineering-notes/five-ways-a-test-suite-lies)
- [One home, locked in both directions](/docs/engineering-notes/one-home-locked-both-ways)