Troubleshooting

Every other page in this manual explains how a part of dev3d works. This section starts from the other end: the string on your screen, the run that stopped, the plugin that never appeared. Find your symptom, read the mechanism behind it, run the checks in order.

What this section is for

dev3d is unusually candid about its failure modes, and it leaves evidence almost everywhere it fails. Exits carry a message, refused paths name the root they were compared against, approval timeouts are logged with the fact that they counted as a refusal, and the router records the specific reason every candidate lost. Very little of it fails silently and nothing of it fails anonymously.

What is missing is a way in. The rest of the documentation is organised by subsystem, which is the right shape for someone learning the system and the wrong shape for someone standing in front of a broken one. You do not know yet whether a failing turn is a routing problem, a provider problem, a tool problem or a budget problem, and reading four concept pages to find out is a bad use of a bad afternoon. So this section is organised by symptom instead, and it names the file behind each mechanism so that a claim here can be checked rather than trusted.

How to use it

1. Find the exact string you are looking at. The quoted symptoms in the symptom callouts are copied from the source verbatim, so they are searchable. A message that only looks similar is usually a different mechanism.
2. Read "What it means" before running anything. Half of these symptoms are the system working as designed and telling you so — an optional stage that failed, a shell command a human declined, a run cancelled by the soft-spend gate. Knowing which half you are in changes what you do next.
3. Run "What to check" in order. The ordering is deliberate: it puts the check that eliminates the most possibilities first, which is usually the boot log or the health endpoint rather than the thing you were about to reconfigure.
4. Prefer the fix callout when there is one, and believe the warning callout when there is not. Where a symptom has an honest "this is a known gap" answer, the page says so and links to Known gaps instead of offering a workaround that does not exist.

The five pages, and what each one owns

Page	The failures it owns	The first thing it checks
Installation and startup failures	The process that will not start, the port already taken, the store that degraded to memory, environment values that were read once and then ignored.	The boot log and GET /api/health.
Providers, models and routing	A key that is missing, shadowed or rejected; a model id that is not a candidate; rate limits; a turn served by a fallback; a routing decision that looks wrong.	The provider list at boot and the turn's own routing record.
Workspace, shell and tools	A refused path, a denied tool, the approval round trip, capped or truncated tool output, web tools that cannot reach anything.	Which tool ran, and what its own result said.
Plugins and skills	A manifest the validator refused, a checksum that did not match, a contribution that never appeared, a skill file that was skipped, plugin tools and confinement.	The plugin's own status and the manifest problems.
Budget, cost and stuck runs	The hard ceiling that failed a run, the soft gate that cancelled one, a stage that failed without failing the run, and a run that looks frozen.	The run record's budget, status and error field.

The failures people hit most often

What you are looking at	Where it is covered
Port 8787 on 127.0.0.1 is already in use	Installation and startup failures
store: memory (...) in the boot log, and history that evaporates	Installation and startup failures
You edited .env and the running office ignored it	Installation and startup failures
providers configured: none, although a key is set	Providers, models and routing
all model routes failed: followed by HTTP statuses	Providers, models and routing
The router picked a model you did not expect	Providers, models and routing
Refusing path ...: it escapes the workspace root	Workspace, shell and tools
The human declined to run this shell command	Workspace, shell and tools
A run sitting in awaiting-approval and nobody was asked	Workspace, shell and tools
A plugin directory in place, and no plugin in the console	Plugins and skills
checksum mismatch: during an install	Plugins and skills
Skills loaded but never pulled into a turn	Plugins and skills
Run budget of $X was exhausted before stage "Y"	Budget, cost and stuck runs
The stage produced no turns	Budget, cost and stuck runs
A turn that reports done and an incomplete work product	Budget, cost and stuck runs

Two things worth checking before any of the rest

Both take seconds, and between them they answer most questions about a running office. Neither requires the console to be working, which is the point.

The health endpoint. GET /api/health answers with the resolved mode, why the mode is what it is, whether the process has drifted from its environment, which store backend is actually live, how many runs are active and how many approvals are pending. That one response distinguishes a configuration problem from a persistence problem from a run that is simply waiting for you.

curl -s localhost:8787/api/health

The boot log. The server narrates its own startup: the version, the bind address, the resolved mode and routing posture, how many floors it loaded, the skills, tools and model counts, which providers are configured, the workspace, the store backend, and how many values it took from .env. A surprising number of problems are visible in those ten lines and nowhere else. Set DEV3D_LOG_LEVEL=debug for more, remembering that from the second boot onwards the saved setting is what counts, so change it in the console rather than in the file. See Environment.

What this section does not cover

• Security. Nothing on these pages is a security review, and no automated suite in the project attempts an attack. The unauthenticated socket, the tools that are and are not confined, and the reach of a plugin are documented as limits in Known gaps.
• The composition of the office. Camera framing, lighting and how a large building feels are not diagnosed here; they are design limits, listed on the same page.
• This website. The manual describes the orchestrator. A problem with the marketplace listing, an account or a page on this site is a different system with a different failure mode, and contact is the route for it.
• Anything that is a bug rather than a limit. These pages describe how the shipped code behaves. If the code behaves differently from the description, that is worth reporting rather than working around.

When it is not in this list

Six of these pages cannot cover every way a system with a model in the middle can disappoint you. When nothing matches, the goal changes from "fix it" to "produce something whose cause is obvious to somebody else" — and that is mostly a matter of assembling what the office already recorded.

1. Capture the exact message. Copy the string rather than describing it. Include the level and scope of any log line, for example engine/stages or tool:run_shell, because the scope names the code path.
2. Capture the health response. curl -s localhost:8787/api/health puts the mode, the mode's reason, the drift state, the store backend and the active and pending counts in one block.
3. Capture the run record. The run id, the stage it stopped in, the run's status and error, and for the turn in question its status, error, route, servedBy and attemptedRoutes. A turn that failed for a provider reason and a turn that failed for a budget reason look identical from a screenshot and nothing alike in that record. See Wire protocol for how to read a run back.
4. Say what changed last. A new key, a new plugin, a new floor setting, an edited .env, a different runtime version. Most of these failures are a change that was read once at boot and never again.
5. Reproduce it on scripted employees if you can. DEV3D_LLM_MODE=mock runs the whole pipeline with deterministic employees and no billing. If the problem survives that, it is not the provider; if it disappears, the provider is the first suspect. See Quick start and Installation.
6. Send it through contact, with the above. The useful report is short and specific, and the one thing that makes it useless is a paraphrase of an error message.

If you are still orienting rather than diagnosing, How a run flows explains the record hierarchy these pages keep referring to, and Model routing explains the decision that a surprising number of these symptoms turn out to be about.