Engineering notesView as Markdown ↗

Every record needs a visible bucket

In plain terms

Every job your board loads appears in one of its sections, including missed and overdue ones, so none quietly goes missing. Overdue work shows whatever dates you are looking at. Where the board leaves something out on purpose, it says how many it hid and lets you show them.

A dashboard that sorts rows into named buckets with independent predicates will eventually drop a row into none of them. It stays in the database, it counts in the totals, and it renders nowhere. Treat the buckets as a partition, assert that their union equals what you fetched, and remember that an open obligation is not a window slice.

At a glance

Who it's for:
Engineers building on Biloh
Checked against the code:
2026-10-29

If a view sorts records into named buckets — today, upcoming, done — and each bucket is an independent predicate, then sooner or later a record satisfies none of them. It is still in the database. It is still counted in the header totals. It renders nowhere. Users call this "the job disappeared"; the logs show nothing wrong at all.

The remedy is to stop thinking of buckets as filters and start treating them as a partition: every fetched record lands in at least one, and a test asserts that union so the next status added to the enum cannot silently fall through.

How the gap opens

The buckets usually start honest. In one dispatch board they were:

unassigned  → no contractor and status = scheduled
today       → date = today and not finished and not cancelled
upcoming    → date > today and has a contractor
completed   → status in (completed, approved, invoiced, paid)
cancelled   → status in (cancelled, declined)

Read those five predicates against a job whose status is missed. It is not in the completed set and not in the cancelled set, so it is not terminal. Its date is in the past, so it fails today and fails upcoming. It is assigned, so it is not unassigned. No bucket. The same hole swallows an assigned job whose date has passed while it is still in a working state — precisely the two categories an operator most needs to see.

Nothing was broken in isolation. Each predicate is defensible; the set of them is not exhaustive. That is the failure mode: exhaustiveness is a property of the collection, and nobody owns the collection.

The cheapest detector you already have

The bug announced itself in a contradiction that was on screen the whole time: the summary strip said 41 records with 2 assigned, while the buckets contained 36 unique rows and the upcoming list was empty. Totals were computed over everything fetched; sections rendered only what bucketed. When those two disagree, the difference is the invisible set.

So make it an assertion rather than an observation:

const inBuckets = new Set(Object.values(view.buckets).flat().map(r => r.id));
expect(fetched.filter(r => !inBuckets.has(r.id))).toEqual([]);   // zero orphans
expect(view.stats.overdue).toBe(view.buckets.overdue.length);    // counts match rows

Seed the fixture with one record per status value, across a past date, today, and a future date. That is a dozen rows and it pins the partition permanently: add a status to the enum without giving it a home and the test names the orphan.

The second-order trap: an obligation is not a window slice

Adding the missing bucket — overdue: any non-terminal record dated before today, plus anything explicitly marked missed — made the tests pass. The deploy went green. Then the screen said Overdue: 0 while dozens genuinely were.

The reason is worth internalising. The view fetched a date window, and the window's default start is today. Overdue records are, by definition, before that start, so they were never fetched at all. The bucket was correct and empty.

Every test had passed because every fixture chose an explicit window that happened to contain the past dates. The product's default window never does. The fixture and the default configuration were different products.

The fix follows from naming the concept properly: overdue work is an open obligation, not a slice of the current period. It gets its own unbounded-below query, merged into the windowed result and de-duplicated by id:

// windowed rows for the period on screen, plus every open obligation, regardless of window
const overdueRows = await q.lt("scheduled_date", tenantToday)
                           .not("status", "in", TERMINAL_STATUSES);
const merged = dedupeById([...windowRows, ...overdueRows]);

Volume is self-limiting in practice: unresolved past work is a backlog the user is supposed to be clearing, and if it is large that is information, not noise.

When hiding a record is the right call, count it

Sometimes a record genuinely should not be on screen by default. On the same board, switching a recurring line off cancels its future visits — machine leftovers nobody booked or cancelled by hand, and clutter in a list an operator reads every morning. The board now leaves those out by default.

That is not a hole in the partition, because the omission is counted and reversible: the view returns how many it left out (line_deactivation_leftovers_hidden), the list says "N hidden" and offers Show, and the agent read takes a flag to include them. Only that one machine origin is hidden; a visit cancelled by an operator, by the system for a reason, or with no recorded reason always renders. The rule generalises: a view may choose not to show something, but it must say how much it chose not to show — a filter with no count is a black hole with a better excuse.

const windowRows = includeLeftovers ? allRows : allRows.filter(r => origin(r) !== "line_deactivated");
const hidden = allRows.length - windowRows.length;   // surfaced, never silent

Checklist

  • Enumerate the buckets against the status enum, not against the happy path.
  • Assert union(buckets) == fetched, and assert summary counts against rendered rows.
  • Ask of every bucket: is this a slice of a period, or a standing obligation? Obligations ignore the window.
  • Fixtures must use the product's default parameters at least once, not only convenient ones.
  • A record left out on purpose is counted on screen, with a way to show it.
  • Render the urgent bucket first, and style it so an empty one is obviously empty rather than absent.

Locked by tests/unit/scheduling/build-schedule-view-overdue.test.ts, which asserts every seeded job lands in at least one bucket, so a new status with no home fails by name.

Related

Under the hood

Source files this page is checked against:

  • lib/scheduling/attention-chips.ts
  • lib/scheduling/build-schedule-view.ts

Raw markdown: /docs-md/engineering-notes/every-record-needs-a-visible-bucket

Last updated 2026-10-29