Verification

Claiming a project is complete is cheap. This page is the actual evidence, suite by suite, including what each one can and cannot prove — and the method behind every number, so you can re-derive it rather than trust it.

The suites, and what each cannot prove

Suite	What it actually does	What it cannot prove
Core vocabulary tests packages/core	Unit tests over the shared contracts: style presets round-trip, colour parsing accepts six-digit hex and nothing else, every pattern carries a label because the editor renders all of them.	Nothing about runtime behaviour. This suite cannot tell you that the office does anything at all.
Engine and server tests apps/server, 16 files	Real runs against the scripted provider and a scratch workspace: real pipelines, the real router, the real tool loop. Proves files land on disk, the budget halts a run, cancellation is safe, debates produce a verdict, the review loop sends work back to the builders, and a run on one floor leaves another floor's directory untouched. Also the router's score and walk arithmetic, plugin manifest validation and the host, the skills loader and selector, the fifteen tools, live org-chart editing, floor-layout geometry, and the catalog, discovery, quality, health and benchmark readers with an injected fetch.	Nothing about a live provider's real responses, nothing about the browser, and nothing about security. A fully green run says the machine behaves as described; it does not say the machine is safe to expose.
Protocol smoke scripts/smoke-ws.mjs	Drives a running office as a real client over the socket: full office state on connect, two complete runs, streamed deltas reconstructing each turn's final text, replay from the persisted event log, direct messages, planning turns, hire and fire and seat moves, per-floor skills and budget staying independent, plugin contributions and the install gate, floor growth, and a malformed request body being a 400 rather than a 500.	Anything visual. It asserts wire behaviour only, never that a pixel was drawn. It also mutates state: it opens and closes floors and normalises them back, and it needs a writable workspace and a running server.
Client store harness apps/web/.verify/smoke.ts	Drives the real client store with one synthetic frame per server-event variant, and asserts the rules a typecheck cannot see: an org update for another floor is ignored rather than applied to the one on screen, a plan reply is queued and consumed exactly once, streamed text survives into the finished turn, an optimistic echo is reconciled rather than duplicated, and every event was counted. Also checks that every material role resolves to a styled material for each shipped preset.	Nothing about rendering. The store is exercised headlessly; React, three.js and the actual draw calls are outside it.
Failure-path checks scripts/check-failure-paths.mjs	Asserts the degradations the documentation promises rather than assuming them: an unusable database path still yields a working non-persistent store that says why and is logged rather than swallowed, a missing skills directory yields an empty catalogue instead of throwing, and one malformed skill file is skipped with its reason while the valid ones still load.	That no other failure is silent. It covers the paths someone thought to write down.
CSS class check scripts/check-css.mjs	Compares every class name the source uses against the selectors the stylesheet defines, in both directions, with a known-dynamic prefix list. Both directions matter: an unknown class is a typo, and a dead rule is a retired component nobody removed.	That anything looks good. Class-name agreement is a lint, not a design review.
Block kit and anchor contract blender/scripts/verify-blocks-glb.mjs, 03_office_furniture.py	Cross-checks the block sidecar against the built GLB: 24 modules, sizes, anchors, heights and material roles, 91 seat anchors and 91 desk anchors; and the furniture script's own contract check of 21 seats, 13 desks and 7 room anchors, which collects problems and exits non-zero rather than using a language assert.	That a room anchor a block declares actually exists. The verifier reads seat and desk nodes and never inspects a block's declared room, which is why the two phantom room anchors described in Known gaps pass it. It also says nothing about whether the office looks good.
Production bundle served by the orchestrator	The web app is built and served from the orchestrator's own static handler, alongside the office asset; a rebuild is picked up on refresh without restarting the server.	Almost everything, as evidence, because no automated suite asserts it. This is a build-and-look check. The code path is real; the verification is by hand.

The numbers, and how each was determined

Every figure below is stated with its method. Where a count is derived from a single declaration, it will move when that declaration changes; where it is asserted by a suite, a change breaks the build.

Number	Value	How it was determined
Roles per floor	13	Asserted by the protocol smoke test's org-chart section, which rebuilds the chart and checks the shipped roster is intact at 13 roles. It is also the length of the shipped roles table.
Departments	8	Read from the shipped departments table: executive, design, research, technology, frontend, backend, platform, quality. Asserted nowhere. The header comment in the same file used to say "seven departments" and has since been corrected to eight; the table was always the truth.
Stage kinds	12	The StageKind union in the shared run contract: intake, plan, research, debate, workshop, design, architect, build, review, test, integrate, report.
Pipelines, and their stage counts	3 — 10, 7, 3	The three exported pipelines in the shipped company: product-build, code-change and quick-answer. Exactly one stage in all three is optional (product-build's research).
Built-in tools	15	The tools suite asserts the registry's own name list matches the shipped TOOL_IDS list and then names all fifteen: think, todo_write, list_dir, read_file, search_files, glob, grep, write_file, edit_file, apply_patch, run_shell, git, web_search, web_fetch, recall.
Skills	15	The skills suite asserts the loader loads all fifteen and that the loaded ids match the shipped id list.
Plugin contributes keys	8	The contribution type and the manifest validator, which reads only those eight: providers, models, skills, roleTemplates, pipelines, routingRules, uiPanels, toolNames.
Office asset contract	21 seats, 13 desks, 7 rooms	The furniture script counts objects by reserved name prefix and exits non-zero if the counts differ, printing the names it found. The protocol smoke test independently asserts that the core office reports 21 seats, read out of the built asset rather than written down.
Block kit	24 modules, 91 seat anchors, 91 desk anchors, 41 materials	Printed by the block verifier, which exits non-zero on a mismatch. It asserts that seat anchors equal desk anchors, that every material maps to a theme role, and a UV-versus-world scale floor. It does not assert the module count and says nothing about room anchors, which is how the two phantom rooms below survive.
Server events	31	Counted from the members of the ServerEvent union in the shared events contract. One of the 31, memory.updated, is declared and never emitted.
Client commands	33 declared, 26 served	Counted from the members of the ClientCommand union in the same file, then compared against the socket's command switch, which implements 26 arms: the plugin-management commands and several others are reached over HTTP instead.
Protocol checks	202	Measured, not estimated: a clean run of the smoke suite against a scripted office prints its own total, and it printed PASS — 202 checks, 0 failures. The source contains 203 call sites; two of them are the two arms of a single try/catch, so exactly one of those two executes. One site is an unconditional pass in an else-branch, so the total flatters the coverage very slightly.
Server tests	not quoted	The suite cannot run in the sandbox this documentation is written in, because it spawns children with piped stdio, and the project's own documents give six different totals for it (415, 437, 557, 558, 634 and 709) across README.md, development.md, sandbox.md and the remediation log. Rather than pick one, this page declines to state a number. The core suite is consistent everywhere at 14 tests.
Core tests	14	Measured the same way: 14 pass, 0 fail, 0 skipped. This is the one test figure the project's documents agree on.
Client harness checks	391	Measured: node apps/web/.verify/smoke.ts prints its own total and reported 391 of 391. It was 129 before the office, walking and material work landed.
Failure-path checks	10	Measured: the script prints each check and a total; 10 of 10 passed.
Stylesheet classes	559 across 62 source files	Printed by the CSS check itself, alongside its known-dynamic prefix list and its verdicts: no unknown class names, no dead CSS.

The corrections this page exists to make

The earlier version of this page and the project's own release notes carried numbers that were simply wrong. Rather than quietly replacing them, here they are with the truth beside them:

Claim that was wrong	The truth
"14 client commands and 24 server events"	33 client commands and 31 server events, counted from the two unions. The project's own docs/wire-protocol.md still prints the old pair, and an intermediate revision of this page printed 31 and 27; both were wrong when written, because the earlier 31 already counted a command that did not exist yet.
"11 plugin contribution surfaces, all wired"	There is no such list. The real counts are 8 contributes keys, 10 plugin permissions, and 10 fields in the host's own active-contribution record. The confusion is easy to see once the three tables are side by side, and the sentence that mixed them should never have been written.
"12 floor checks"	11, in the client harness's floor section.
"11 runtime tests"	16, in the runtime suite.
"Seven departments, thirteen employees" in the org-chart source's own header	8 departments, 13 roles. The comment is stale; the departments table below it is not.

The 202 protocol checks were the one number from the earlier page that survived scrutiny. It is also the one most worth distrusting on principle, which is why it was re-measured by running the suite rather than counted by reading it.

Three facts that follow

Node 24 is a real requirement, and pnpm comes from the manifest

The repository declares engines.node as >=24.0.0, and that is a genuine floor rather than a preference. Two things depend on it: SQLite, which persistence is built on, is reached through Node's own node:sqlite module with no native dependency and no driver package; and the suites run TypeScript directly under the built-in test runner rather than through a build step. On an older runtime the server may well start while the suites do not run at all, which is the worst of both worlds: you would be operating an untested build.

There is no engines.pnpm. The package-manager floor is the packageManager field, which pins pnpm 11.22.0 and is enforced by corepack. So the accurate statement of the requirement is Node 24 or newer, plus pnpm through the packageManager field — not "pnpm 11 is required", which is a claim the manifest does not make.

It is demoable before it is billable

With no provider keys configured, the server boots on scripted employees and the entire pipeline still runs: real stages, the real router, the real tool loop, real files on disk. The health endpoint reports the active mode and the console badges it, so nobody has to wonder whether a run is spending money.

The suites take the same position. The engine tests construct the registry in mock mode and run against the scripted provider; the provider-facing tests inject their own fetch. The whole of the evidence above therefore costs nothing to reproduce and requires no credentials. What it also means is that no suite in this repository tests a real model's behaviour, which is stated again under Known gaps.

The database is real and zero-dependency

Persistence is SQLite through Node's own module, at the configured database path, with no driver package. It holds the org chart, every run, turn, artifact and approval, plus an append-only event log. There is exactly one writer, which is what keeps the durable record and the broadcast from drifting apart.

The ordering is the interesting part. An event is persisted before it is broadcast, and the durable projections are updated in the same pass before subscribers are called. The consequence is a property worth stating precisely: a client that reconnects and replays from the persisted log sees a superset of what it had before, never a gap. A crash between the two steps cannot lose an event that a client already saw.

It also degrades honestly rather than silently. If the database path is unusable the server keeps running with a non-persistent store, logs why, and still accepts writes; the failure-path checks assert exactly that, including that the reason is logged rather than swallowed.

How to re-run all of it

Everything on this page is reproducible from a checkout. These are the scripts in the repository, with the underlying command beside them:

pnpm test              # every workspace suite
pnpm test:server       # node --test --test-isolation=none "src/**/*.test.ts" in apps/server
pnpm test:core         # the same, in packages/core
pnpm test:web          # node apps/web/.verify/smoke.ts
pnpm smoke             # node scripts/smoke-ws.mjs, against a running office
pnpm check:failures    # node scripts/check-failure-paths.mjs
pnpm check:css         # node scripts/check-css.mjs
pnpm check:blocks      # node blender/scripts/verify-blocks-glb.mjs

The smoke suite needs an office already listening, and takes a base URL as its argument. It also needs a writable workspace, because it submits real runs and builds real rooms; run it against a throwaway database rather than the one you keep work in.

How these suites are built

The mechanics matter, because they explain both why the evidence is cheap to reproduce and where its blind spots come from. Four choices run through all of it.

• The built-in test runner, with isolation switched off. Each workspace runs the runner over its own TypeScript test files directly: no build step, no test framework, no transform, and none of the usual multi-process isolation, which is why a run is fast and why a leaked global would be visible rather than hidden.
• A scripted provider by default. The engine tests construct the provider registry in mock mode, and everything provider-facing injects its own fetch instead of calling a vendor. No run anywhere in the repository needs a credential, a network route, or money.
• Real scratch state, cleaned up. The engine and tool tests create a temporary workspace and write real files into it, then remove it; the failure-path checks point a database at an unusable path on purpose. The assertions are about what landed on disk, not about what a mock recorded.
• The protocol suite has no dependencies at all. The smoke script uses only the runtime's own fetch and WebSocket, so it needs no install and no build, and it can be pointed at any running office by passing a base URL.

That set of choices buys reproducibility and costs realism. Everything is deterministic and free; nothing is live, and nothing is judged by eye. The limits column of the table above is the price of the column beside it, and it is worth reading the two together.

Where the server suite's tests actually are

Because the total is one number, here is how 305 tests are distributed, so that a future change in the total can be attributed rather than merely noticed:

Suite file	Tests	What it covers
Plugins	47	Manifest validation, discovery, the host, contribution wiring, install and removal, panels, namespacing.
Router scoring	25	The weighted score, the walk order, affinity, cost pressure, reliability, ties.
Benchmarks	24	The published-benchmark reader, against a real measured distribution rather than a fixed scale.
Pooled quality	23	The shared quality index, its cache, and its freshness window.
Router selection	21	Policy targets, postures, escalation, escalation bounds, pins, capability relaxation, rejection reasons.
Endpoint health	21	Uptime tracking and the penalty it feeds.
Quality	19	Curated quality, learned quality and their blend.
Discovery	18	Asking providers for their model lists, caching the answer, and keeping "I could not ask" distinct from "there are none".
Engine	17	Real runs on a scratch workspace: the budget halt, cancellation, debates, the review loop, cross-floor isolation.
Runtime	16	Organisation and floor editing, seating, style persistence, migration from the legacy single-organisation shape.
Model list	16	Parsing what providers return.
Config	14	Environment parsing, defaults, clamping and precedence.
Model adapters	14	The OpenAI-compatible and Anthropic request and response shapes.
Floor layout	13	The block jigsaw: mating edges, collision rejection, determinism.
Tools	9 (1 skipped)	The path resolver, the file tools, the shell approval gate, the registry.
Skills	8	Front-matter parsing, the loader's tolerance, and selection order.

Three claims worth guarding

Most of what the suite covers is ordinary. Three properties are load-bearing in a way that would be expensive to lose silently, and each has exactly one place that guards it:

• A run cannot spend money it does not have. Guarded by the engine tests, which drive a real run past its ceiling and assert that the run failed with the stage named. If this regressed, the first symptom would be an invoice.
• A reconnecting client never loses an event it already saw. Guarded by the protocol suite, which replays a persisted log and checks that finished turns and stages are in it. If this regressed, the symptom would be a console that disagrees with the server intermittently.
• A new protocol event cannot be dropped silently. Guarded twice: by the client store's exhaustive default branch, which fails the typecheck, and by the client harness, which asserts that every event variant was counted. If this regressed, the symptom would be a feature that works on the wire and never appears on screen.

Everything else in the evidence is either arithmetic, parsing, or a description of behaviour that a reader could check by hand. Those three are the ones where a quiet regression would be discovered by a customer rather than by a build.

When these numbers were measured

The counts on this page were measured while writing it, against release 1.0.0 of the orchestrator, on Node 24.19.0, with the office running on scripted employees and a throwaway database. The environment is stated because it is part of the method: a different runtime, a live catalog or an installed plugin changes the catalog size, and therefore some of the routing figures elsewhere on this site, though not the suite totals.

If you re-run any of it and get a different total, the useful next question is which of the four mechanics above differed — the runtime version, the plugin set, whether a workspace was writable, or whether a confined sandbox skipped a case. Those four explain almost every discrepancy worth chasing.

What a green suite does not prove

• Nothing about security. No suite here attempts an attack. The confinement of the tools, the absence of authentication and the reach of a plugin are described in Known gaps, and none of them are covered by an automated check.
• Nothing about a live provider. Every run in every suite is scripted. Real latency, real refusal behaviour, real pricing and real context limits are unexercised.
• Nothing about rendering. The client store is exercised headlessly; React and three.js are not.
• Nothing permanent. These are counts at release 1.0.0. They are evidence of the state of one commit, and the method is what transfers, not the number.

One skipped case deserves a note of its own, because it is skipped for an honest reason rather than disabled. It asserts that an approved shell command actually executes; the shell tool captures standard output by contract, which means spawning a child process with piped streams, and a confined sandbox blocks exactly that. The test detects the condition, reports itself as skipped with that reason printed next to it, and still runs — and still has to hold — anywhere child processes are permitted.