The third kind of failing test
Triage a failing test and it lands in one of three buckets: the test is stale (the behaviour legitimately changed — fix the test), the code is defective (fix the code), or — the awkward one — the test is right and the feature doesn't exist yet. The assertion describes behaviour the product owner has explicitly ruled correct, but building it is a real work package, not an afternoon.
Delete that test and you delete a ratified decision. Mark it .skip and you hide a decision inside a mechanism built for flaky tests. Leave it red and it poisons the suite's signal. None of these survive an honest zero-failing-tests policy.
The convention
Move the spec — assertions preserved verbatim — into a dedicated quarantine directory excluded from the suite run. Each file carries a header naming:
- the work package that owns building the behaviour,
- the ruling it encodes (who decided, and what they decided), and
- the evidence the ruling was based on.
The failing test stops counting against the suite, but nothing about it has been weakened: it is now a specification waiting for its build. When the work package lands, its definition of done is the quarantined spec passes as written and moves back into the suite. The team calls this building red-first from a ratified contract — the spec exists, measured red, before the first line of the feature.
The quarantine is itself under test
A convention like this rots into a landfill unless something enforces it. A structural integrity lock — an ordinary test in the main suite — asserts:
- Immutability: quarantined assertions match the verbatim text recorded when they entered. Softening a spec inside quarantine is a build failure.
- Ownership: every file's header names a work package. Anonymous debt is refused.
- A hard cap on the number of quarantined specs. The cap forces a conversation ("build one out, or raise the cap deliberately") instead of unbounded quiet growth.
- No vacuous specs: a quarantined file with no real assertions fails the lock — a spec that asserts nothing specifies nothing.
The cap and the immutability rule are what distinguish this from a graveyard: the directory can only shrink through builds, never through erosion.
A worked example
A billing-cadence defect surfaced as a failing test: a job completing late, after its period's consolidated invoice had already been drafted, was silently appended to that draft. The owner ruled the opposite: a drafted invoice is stable once drafted; the late job rolls to the next period automatically. The ruling became three quarantined specs — the drafted invoice is byte-identical before and after the late completion, the late line lands on the next period's draft with its exact cents, and the job is billed exactly once — each asserting exact database-level amounts. The work package that builds the routing inherits its acceptance tests ready-made, and cannot be declared done until specs it never wrote go green.
Why not just a backlog ticket?
A ticket describes intent in prose; a quarantined spec is the acceptance test, with exact assertions, already agreed. Prose drifts and gets re-litigated; an immutable spec with a named ruling does not. The backlog ticket still exists — it points at the spec.