Engineering notesView as Markdown ↗

A public promise is a test, not a sentence

In plain terms

Biloh publishes a short list of things it will not do. Each one is checked by an automatic test every time the software changes. If someone builds a feature that would break a promise, the change is stopped until either the feature or the promise is fixed. A promise that is not fully true yet is not published.

Biloh's list of what it will not do lives in one file, and every entry names the behaviour tests that break when the promise does. A promise that fails its check against the code is kept off every public page until it is true, and rewording one needs a protected review.

Why tie a public claim to a test?

A sentence on a website can go out of date the day someone ships a feature. If a platform says "this never happens" and then builds the thing that makes it happen, two truths compete: the old page and the new code. An AI assistant reading the page has no way to tell which is current. Biloh closes that gap by making each claim a test.

Where do the promises live?

In one file, lib/docs/guardrails.ts. Each entry has the sentence, the tests that enforce it (enforced_by), the date a person last re-read it, and a published flag. Every public surface — the capability pages, the /llms.txt section headed "What Biloh will not do", the Markdown copies and the reviewer runbook — renders from that one list, so changing a sentence changes it everywhere in the same deploy. A style guard refuses a staged page that types a promise out by hand instead of rendering it, so copies never drift.

What does a tripwire look like?

Each promise has a behaviour test that goes red when a change breaks it, each proven by a red fixture — a planted file with the forbidden thing in it:

  • Money movement. A scan of the codebase finds no call that creates a transfer, a payout or a refund at the payment provider. A planted transfer call fails the test.
  • One business per connection. No tenant-scoped tool declares a field that names another business.
  • Where records are stored. The hosting regions are pinned, and adding a new database or storage package fails the test until a person checks where it keeps data.
  • The change log. Every write route and write tool must reach an audit writer, be listed as writing no business data with a written reason, or be named in a list of paths proven by a behavioural test. An entry on that last list counts only while the named test exists: renaming or deleting the test turns the path back into a gap.

What happens before a promise is published?

A promise is checked against the code first. Until it holds everywhere, it carries published: false with the evidence written beside it, and no public page shows it. The change-log promise went through exactly that path: it was held back, the missing records were built for every write route and tool, the tripwire reached zero gaps, and only then was the flag turned on.

Who can change a promise?

lib/docs/guardrails.ts is in the merge steward's constitutional set (lib/platform-work/merge-steward.ts), so an agent cannot merge a reworded or retired promise on its own; the pull request has to quote the old and new sentence side by side. Every new build spec answers one question first: does this change make a promise untrue, and if so, will the feature change, will the sentence change, or will the promise be retired?

What locks it?

tests/structural/docs-guardrails.test.ts holds the catalogue checks (every enforced_by path exists, sentences stay short and plain, published sentences render exactly once and unpublished ones never) and the tripwires above, including the G-10 change-log check.

Related: One home, locked in both directions.

Under the hood

Source files this page is checked against:

  • lib/docs/guardrails.ts
  • lib/docs/guardrail-scan.ts
  • lib/audit/exemptions.ts
  • lib/platform-work/merge-steward.ts
  • tests/structural/docs-guardrails.test.ts

Raw markdown: /docs-md/engineering-notes/a-public-promise-is-a-test

Last updated 2026-11-12