Backfilling tests on untested code
Tests that fail when the behaviour breaks, not when it is refactored
Find the behaviour boundaries in code that has never been exercised, write tests against them, and have the run prove that each test actually detects a change.
What actually goes wrong today
Stated plainly, because the approach below only makes sense as an answer to something specific.
- 01
The code most likely to break is precisely the code nobody felt confident enough to test, so the coverage number is high where it does not matter and absent where it does.
- 02
Generated tests without a plan assert the implementation rather than the behaviour: they pass forever and fail the moment someone refactors correctly, which teaches the team that tests are an obstacle.
- 03
Deciding whether a branch is a contract or an implementation detail requires knowing the call sites. That is research, and it is the step a test-generation script skips entirely.
- 04
A test that never fails is worse than no test, because it is counted.
How the hierarchy and the router do it
Concretely, stage by stage: which mode the stage uses, who owns it, and why the router prices it the way it does.
- 01
Research finds the real behaviour boundaries from call sites and data shapes rather than from the function signature, because the signature tells you what is possible and the call sites tell you what is promised.
- 02
A debate stage argues whether a specific branch is behaviour or incidental, and the chair records the ruling for the cases that were contested.
- 03
The plan names test files and the cases in them, which is what stops the implementation from writing the tests for itself.
- 04
Developers write tests and run them. The review-loop has QA challenge the tests: a test that survives an intentional change to the code under test is not a test, and the loop sends it back to be rewritten.
- 05
Per-file work is bounded and tool-driven, so it sits on a standard tier; the scope decision sits above it. The tier band per role is what keeps a coverage sweep from being billed at frontier prices throughout.
What you actually end up with
Artifacts, not assurances. Every one of these is something you can open, diff, review or throw away.
Test files that fail when the behaviour changes and survive a refactor, which is the only property that makes a suite worth keeping.
A written record of which behaviours are deliberately uncovered, so the gap is a decision instead of an accident.
A cost profile that scales with the size of the code under test rather than with the size of the model.
Which part of the company is involved
Six of the thirteen roles, four of the eight departments. The rest of the office stays idle, and that is what keeps the run affordable.
Finds behaviour boundaries from call sites and data shapes, because the signature says what is possible and the call sites say what is promised.
Argues whether a given branch is contract or incidental — the question that decides whether a test is worth keeping.
Names the test files and the cases in them, which is what stops the implementation from writing tests for itself.
Writes the tests and runs them. Bounded, tool-driven work, so it sits on a standard tier rather than the top of the walk.
Challenges each test: a test that survives an intentional change to the code under test is not a test, and it goes back.
Modes used
Intake runs single, research runs parallel, contested decisions run as a debate, and anything that produces files ends in a review-loop.
Where the money goes
Reading and summarising sit on cheap bands. The plan, the implementation turns and the review escalate, because those are the turns where being wrong is the expensive mistake.
What is recorded
Per-turn cost, the route decision and the models the router rejected, the files touched, and the debate ruling including the position that lost.
Two other jobs in the same shape
Cutting model cost on routine work
Pay frontier prices for the decisions, not for the summaries
Move the routine turns of an agent workflow onto the cheapest models that can do them, and keep the expensive ones where being wrong is expensive — with a recorded reason for every route decision.
Read itLegacy codebase modernisation
Understand the seams before you start cutting along them
A migration of a service nobody fully remembers — a framework upgrade, a module split, a runtime change — cut into reviewable steps that each land as real files and are checked by someone other than the author.
Read itRun this in mock mode first
Node 24, one install, no API keys. The pipeline, the stages, the file writes and the review loop all run against scripted employees, so you can watch the shape of the work before it costs anything.
Nothing in the orchestrator authenticates, so bind it to localhost. The tools are confined to the run workspace, but that workspace is a normal directory on the machine and the orchestrator is not inside an operating-system sandbox — give it a checkout you would be willing to lose.