Errors, exit codes and diagnostics
What dev3d prints when something goes wrong, what it means, and where to look next.
What the system actually prints when something goes wrong, what each message means, and which page to read next. This is a partial list on purpose: every string below was read out of the real source, so a message that is not here is one nobody has catalogued yet rather than one that does not exist.
To find something missing, set DEV3D_LOG_LEVEL=debug, reproduce the failure and read the terminal; or search the checkout for the literal text, since messages are plain strings in the file that raises them. To report it, contact with the log line and the fields listed at the end.
Where the output goes, and which half carries what
| Where | What it carries |
|---|---|
| The server log | Boot lines, the store's state, plugin and skill loading, discovery, and every warn or error the engine raises. Formatted HH:MM:SS.mmm LEVEL [scope] message, the scope being a tag such as boot, store, skills, plugins or ws. |
| The activity feed | The subset an operator should see, mirrored as log events with the same level, scope and message, in time order with the run, turn, tool, approval and routing events. A failure is timestamped against its run here. |
| An HTTP response body | Anything a request asked for and did not get, as JSON pretty-printed with two-space indentation and a single error string. The status code classifies; the string is the reason. |
A socket error frame | Anything a command asked for and did not get. Cancellation, approvals, seating and the org chart are socket-only, so this frame is their whole error surface. Print every frame while debugging. |
Two record fields matter before the sections below. run.error and the per-stage error carry the message for a failure the engine decided to stop for; a turn has its own error, and can carry a real one while its status is still done with no error event emitted.
Startup and configuration
dev3d failed to start: <message>- Printed to standard error when boot throws; the process exits with code 1. Everything that can fail before the server listens arrives through this one line.
Port 8787 on 127.0.0.1 is already in use. Another dev3d orchestrator is probably still running - stop it, or set PORT to something else.- A bind failure, with a plain hyphen and the resolved address, so a changed
HOSTorPORTis reported as the process actually tried it. Install and startup. running without persistence (<reason>); history will be lost on exit- A
WARNfrom scopestore: SQLite could not be opened and the office fell back to memory. It keeps running and accepts writes, but the chart, runs and event log are gone at exit. skipped <file> — <reason>- A
WARNper malformed skill file, then one total line:<n> skill file(s) skipped — see the warnings above. Reasons name the key, such asSkill file <path> is missing the required frontmatter key "id".One bad file never stops the office opening. DEV3D_AUTO_APPROVE_SHELL is on: employees may run shell commands without asking.- A
WARNfrom scopeboot, printed only when the switch is on. It is what to look for when shell commands stopped asking, and that switch is the tool's only gate. model discovery: off — the curated catalog is the only source- An
INFOfrom scopeboot; the "on" variant reports the cache path and freshness window. If models are missing on a machine with no network, start here. Providers and models.
The mode line, and why it is a sentence
The boot banner reports the resolved mode and the reason for it, and GET /api/health returns the same sentence as llmModeReason. A bare "mock" is what makes a stale process look like a configuration bug.
| The reason string | What it means |
|---|---|
DEV3D_LLM_MODE=auto, and deepseek is configured | The mode resolved itself and a provider counts as configured, so the office is live and turns are billed. |
DEV3D_LLM_MODE=mock forces scripted employees even though deepseek is configured | The mode was forced while keys exist — once reported as "no provider keys found", which sent people hunting a configuration bug that did not exist. |
DEV3D_LLM_MODE=live, but no provider is configured — every turn will fail | A warning, not a validation error, and it means what it says: the office starts and every turn fails. |
Exit codes
| Code | When |
|---|---|
0 | A normal shutdown. SIGINT and SIGTERM are both handled, with a four-second watchdog forcing the exit if a wedged socket holds the process open. |
1 | main() rejected, printing dev3d failed to start: <message> first. The only non-zero exit the server produces. |
Exit codes from employees are different: a shell command's exit status is content in a tool result, not the orchestrator's exit code.
Configuration that has moved on since boot
GET /api/health answers with configStale and configStaleDetail, assembled from two signals and always ending in the same advice.
.env has been modified since this process started, and DEEPSEEK_API_KEY is now set for 'deepseek'. The environment is read once at startup, so restart the orchestrator to apply it.- The two signals joined: the file's modification time moving past boot, and a provider key having appeared in the environment. A key being removed is deliberately not detected, because a running process cannot observe that.
- A variable in
.envthat appears to have no effect - Three causes: it may be one of the eight values copied into installation settings on a first boot, a real environment variable may be beating the file, or the process may predate the edit. None prints an error. Environment.
Provider and model failures
The failover loop collects these rather than throwing, so the final error usually names several attempts. Providers and models.
HTTP <status> from <providerId>: <detail>- A provider rejected a completion request, with up to 300 characters of the response body. A
401is the credential, a404usually a wrong base URL or unserved model id, a429rate limiting or balance. HTTP <status> from <providerId>/models: <detail>- The same shape against the model-list endpoint, truncated at 200 characters. Discovery rather than a turn, so it never fails a run and leaves the curated catalogue in use.
<providerId>: streaming response had no body- The request was accepted and the stream was empty — in practice a proxy that does not forward a streamed body, or a provider configured to return everything at once.
all model routes failed: <provider>/<model>: <error>; <provider>/<model>: <error>- The failover loop ran out of candidates, each attempt in order and joined with
;. A candidate that could not be looked up appears as<provider>/<model> (unavailable), which separates "not catalogued" from "refused the call". no model in the catalog satisfied the request./model catalog is empty; no model can be routed.- Two emptinesses: a turn that had candidates and lost them all to filtering, versus a catalogue with nothing in it. Both return a decision with an empty fallback list.
this provider has no model-list endpoint; the curated catalog is in use- A discovery report's
errorfor a provider that cannot be asked what it serves, such as a local runtime without the endpoint. Recorded as an error and not one: the provider still routes. response was not a model list (no data/models array)- The model-list request succeeded and the payload was not a list — typically an HTML error page from a proxy. A discovery failure, never a turn failure.
no OpenRouter API key is configured/no Artificial Analysis API key is configured- Both are returned without making a request at all: a default install makes no outbound call it did not have to. Expected on an installation that never authorised the lookup.
One related state is not a message: GET /api/providers reports modelSource as discovered, degraded or seed. degraded means discovery was attempted and failed, with modelSourceDetail carrying the reason.
Routing decisions that surprise you
A routing decision is not an error, and this is where most "why did it use that model" questions are answered. Every decision carries a reason of clauses joined with — , and every candidate that lost carries its own reason. It is on the routing.decision event, in the transcript, and on the employee as lastRoute. Model routing.
The clauses a reason string is built from
task 'coding' maps to standard under this role's policy- The starting point: the stage kind mapped to a task class, and the class or the role's default produced a target tier. Everything after this clause adjusts it.
posture 'cheap' lowered to small/posture 'quality' raised to strong- The posture moved the target tier, down to the policy's floor or one tier up. A clause equal to the original is not emitted, so no posture clause means it changed nothing.
complexity 0.62 >= 0.6 escalated to strong- The turn's complexity crossed the role's
escalateAtComplexityandescalateTowas applied. Roles declaring no threshold never produce it. clamped to strong (policy bounds small..max)- The target left the policy's floor and ceiling and was pulled back. Clamping runs after escalation, so an escalation can be clamped, and this clause is how you find out.
budget nearly exhausted- Under five cents remain, so the posture was forced to
cheapregardless of the floor's setting. A run routing down a tier near its limit is doing what it was told to. pinned by this role's model policy- The role has
pin: true, so posture, escalation and the whole walk were skipped and nothing else influenced the choice. quality from curated+learned- Which sources fed the score.
curatedis the shipped table plus operator overrides,learnedis what this office observed, andpooledappears only when a benchmark lookup was authorised.
The rejection reasons, which are the useful half
| Reason on a candidate | What it means |
|---|---|
excluded | On the router's exclusion set for this choice. A plugin's avoidModelIds never produces it: an avoided model is penalised and still appears as a scored candidate. |
lacks tool calling / lacks vision | The turn has tools granted and the model cannot call them, or the request carries images it cannot see. |
context window too small | The assembled prompt does not fit. |
a pin is in force for this role | Another model was pinned, so nothing else was eligible. Expected wherever a policy has a pin. |
tier is not on this policy's walk | The model's tier was not among those the walk considered. The walk is short, so a model two bands away may never have been in it. |
scored 0.512 against the chosen 0.804 | Considered, scored and beaten. The numbers are real, which makes this the best answer to "how close was it". |
tied on score; more expensive, or later by id | A genuine tie broken by blended cost then by id, so the same pool always produces the same winner. |
When the router had nothing that met the request
Capability filtering is not a hard failure, so you get one of three clauses appended to the reason — a real degradation, and the only place it is visible.
| Relaxation note | What was given up |
|---|---|
no model met the capability requirements; dropped tool/vision requirements | The request wanted tools or vision and nothing had them. The turn proceeds on a model that cannot call tools, answering from the model alone. |
no model met capability or context requirements; dropped both | The context requirement went as well, so the prompt may not have fitted comfortably. |
no model satisfied the request; fell back to the full catalog | Every filter was dropped. Seeing this often means the policy's band is wrong for the work. |
One routing failure is a turn error rather than a clause list: No model could be routed for this turn., recorded when the router returned nothing usable at all. Budget and runs.
Tool and workspace failures
A tool failure is recorded on the call, with a status of ok, error, denied or running — an outcome the employee is expected to report and work around, not a reason to fail a run. Workspace and tools.
refused: <tool> is not granted to <role>- The role tried to call a tool it does not hold; status
denied. The fix is a grant, and grants are replaced rather than merged. The wire protocol. unknown tool <name>- The name is not in the registry: a tool from a plugin that was disabled or failed to load, or a hallucinated name. Plugin tools are namespaced, so a manifest's name is not the engine's.
The tool "<name>" crashed: <error>. Report this rather than working around it.- The tool threw. The advice is addressed to the employee and is deliberate: a crash is a bug to report rather than a condition to route around silently.
Refusing path <candidate>: it escapes the workspace root <root>./Refusing drive-relative path <candidate>: it is not confined to the workspace.- The two path refusals, with the candidate quoted as written and the root named being the run's. The second covers the Windows trick that makes a path look relative while resolving to a drive root, such as
C:foo. Path argument must be a non-empty string.- A tool was called without a path, usually from a malformed argument object rather than a workspace problem.
File does not exist: <path>/Directory does not exist: <path>/Not a directory: <path>/Path does not exist: <path>- The four "it is not there" shapes, one per tool family, kept distinct so a log line says which tool was asked for what. A directory passed to a file tool gets
"<path>" is a directory; use list_dir to see its contents., and a binary gets"<path>" looks like a binary file; refusing to read it. Only http:// and https:// URLs are allowed (got ftp://).- The web tools' only limit, and a protocol check rather than a path one — there is no workspace boundary involved because there is no path.
Tool error: <message>- The wrapper a filesystem or code tool puts around an unexpected exception, so the failure reaches the turn as a tool result rather than crashing the run.
Shell commands specifically
run_shell is the one built-in tool whose result carries its own exit status, and the one with a human in the loop.
| What the result says | What happened, and what to do |
|---|---|
exit code: 0 and the output | Succeeded. Long output is marked as truncated rather than silently cut. |
exit code: 1, or exit code: 1 (signal SIGKILL) | Ran and failed, with the signal named when it was killed. Read the captured standard error under [stderr]; this is the program's exit code, unrelated to the orchestrator's. |
Command timed out after 30000 ms and was killed. | Exceeded its timeout: 30 seconds by default, 120 hard maximum, anything else clamped. Ask for longer or split the command; output captured before the kill is included. |
The human declined to run this shell command. Do not retry the same command; ask for clarification or propose a non-shell alternative. | The approval was answered no, or timed out, which counts as no. Answer it or lengthen DEV3D_APPROVAL_TIMEOUT_MS; auto-approval also works and removes the tool's only gate. |
One shell behaviour is not a message: the command runs through the platform shell with the workspace root as its working directory and nothing else, so shell text writing to an absolute path outside the workspace succeeds and prints nothing about it. Known gaps.
Plugin loading and manifest validation
Failures here split in two, and the split is the most useful thing on this page. Problems reject the whole plugin: the host cannot tell what it would be running, so it does not run it. Warnings drop one malformed contribution and keep the rest, and every warning is reported, because a plugin that silently loses half its models is worse than one that refuses to load. Plugins and skills.
no plugin.json in the directory.- A directory under a plugins root held no manifest, so it was not a plugin. The record keeps the directory name and this reason.
plugin.json is not valid JSON: <error>- The manifest did not parse, with the parser's own message appended. A trailing comma or a comment is the usual cause; JSON has neither.
entry module "<path>" does not exist./entry module "<path>" does not export activate()./entry "<path>" escapes the plugin directory.- A code plugin that could not be imported, did not provide the one function the host calls, or declared an entry outside its own directory. All three leave it in status
errorwith that string as its reason. tool "<name>" is already registered.- An attempt to register an existing name. Namespacing makes this rare between plugins, so within one plugin it means two registrations of the same tool.
What a rejected manifest actually looks like
When several problems are found, all are collected before the manifest is refused, and the report is built as field: message pairs joined with ; . That string becomes the plugin's reason, which is what the console shows. A manifest with a malformed id and no apiVersion produces exactly this — note that the id message ends in a full stop of its own, so the joined string is genuinely punctuated that way:
id: must be lowercase reverse-dns style with at least two segments, e.g. "dev3d.cost-guard".; apiVersion: is required; this host implements "1".
Read it as a verdict list rather than one sentence. Each clause is independently actionable, and all have to be fixed before the plugin loads, because validation stops at the first stage that produced a problem. Plugin manifest.
The individual reject messages, met one at a time: manifest: must be a JSON object.; id: must be lowercase reverse-dns style with at least two segments, e.g. "dev3d.cost-guard".; version: must be semver, e.g. "1.0.0"., a shape check only; apiVersion: is required; this host implements "1". and apiVersion: targets API "2", but this host implements "1"., where only the leading integer is compared; and permissions: unknown permission(s): foo, bar., which rejects the manifest because the console has to render the list honestly. Two warnings fire about intent rather than correctness, neither fatal: declares an entry module but no permissions. and asks for the "tools" permission but ships no code, so it cannot register one.
Installing, updating and the install gate
Installing plugins is disabled. Set DEV3D_ALLOW_PLUGIN_INSTALL=true to allow running code downloaded from a marketplace.- The shipping default rather than a misconfiguration: the variable is false and only the literal
trueopens it. Installing unpacks an archive and, if the manifest declares an entry, runs someone else's code in the orchestrator's process. checksum mismatch: the marketplace published a1b2c3d4e5f6a7b8… but the bundle hashes to 9f8e7d6c5b4a3210…- The bytes downloaded are not the bytes hashed: the stored bundle was replaced outside the console, or something is rewriting it in transit. The first 16 characters of each digest are shown.
This plugin ships with the office in DEV3D_PLUGINS_DIR. Disable it, or remove the directory yourself.- A bundled plugin cannot be deleted through the API. The alternative would be an API call that deletes files arriving with the checkout.
"<id>" is already at 1.2.0; the marketplace offers 1.2.0./"<id>" ships with the office; update it by updating dev3d.- An upgrade refused because it is not newer, usually from forgetting to bump
version; and one offered for a bundled plugin, refused because replacing it leaves the installed files and the loaded set disagreeing. plugins[3].manifest is invalid — <clauses>- A catalogue document with one bad entry. The host refuses the whole document rather than skipping the entry, so a marketplace publishing one broken entry is unreadable to every host.
could not reach the marketplace: <error>/the marketplace returned HTTP <status>./could not reach the panel endpoint: <error>- The three network failures, one per reader. None fails its caller: an unreachable marketplace is recorded per source and the others are still checked. A cap comes with them —
this installation already reads 12 live panels; this one was not fetched.
Budget and cancellation
Two mechanisms stop a run, and they are less similar than they look. The ceiling is not a setting and it fails the run; the soft gate is a setting and it cancels the run. The value that governs is the floor's budget or the submitted amount, not DEV3D_RUN_BUDGET_USD, which is read and never used. Budget and runs.
Run budget of $<limit> was exhausted before stage "<stage name>".- The hard ceiling, checked before each stage on a greater-or-equal comparison, setting the run to
failedand emitting anerrorevent. The stage named is the one refused, so the overspend was in the stage before it. Run budget of $<limit> has been exhausted.- The same fact as the stage's abort reason. A stage may consult it between turns, and only
debateandreview-loopdo — which is why the other two can finish past the ceiling. The operator declined to continue spending on this run.- The soft gate was refused and the run became
cancelled, notpausedand notawaiting-approval. The approval carriesturnId: null, which tells it from a tool approval. A timeout refuses identically. The model hit its output limit mid-turn; the reported work product is incomplete./Stopped after 8 tool round trips without a final answer.- The two partial failures. Both now settle as failed turns that keep whatever they produced and emit an
errorevent, so run, stage and employee agree that something was cut short. This used to be a trap: the turn kept the statusdoneand nothing reached the wire, so a truncated work product looked clean everywhere. The caps are the role'smaxOutputTokensand a constant in the turn loop. Cancelled by the operator./Run "<id>" is not running.- The turn's error after a
cancelreached a run in flight, and theerrorframe replying to a cancel for a run that has finished or is not in the engine's in-memory set. Neither is a fault; for a finished run, read it back by id. The stage produced no turns.- A stage settled with nothing in it, recorded alongside the separate fact of its
turnIdsbeing empty. Together they separate "ran and produced nothing" from "failed"; the usual cause is a pipeline naming a role the floor lacks. Stage "<name>" has no resolvable participants.- A stage whose role list resolved to nobody, logged at
errorlevel with scopeengine/stages. A stage naming an individual unknown role is milder: a warning, and the role is skipped. The review ended with objections unresolved; the work was not approved.- A
review-loopthat exhausted its iterations with reviewers still objecting. Objections are matched by regular expressions over their text, so phrasing that dodges the patterns reads as approval.
Refusals at submission
| Message | What to change |
|---|---|
There is no workspace "<id>" in this office. | The named project does not exist — deliberately not a silent fallback to the default, which would mean a run writing into a directory nobody chose. |
This office has no workspaces to work in. | No organisation at all: the state before a first project exists, not a corruption. |
"<name>" has no pipelines configured. | The floor exists with no pipeline, usually because a plugin's pipeline was removed after the floor was created. |
Pipeline "<name>" needs roles this floor does not have: <ids>. Hire them first — a plugin offering this pipeline usually offers the role template to go with it. | A pipeline naming roles nobody holds. Without the check the run would start, every stage would produce no turns, and the operator would be told the stage produced no turns — true and useless. |
A non-empty "brief" string is required. | An empty or missing brief over HTTP. The socket path validates the same thing with the same intent. |
Wire protocol and client connection
This is where a misreading costs the most time, because a message that looks like a server fault is often a caller's mistake with a precise answer attached. The wire protocol, and the vocabulary is in the glossary.
Malformed command: not valid JSON.- An
errorframe replying to a socket frame that was not a JSON object. Frames are not newline-delimited here, and a CLI appending a prompt or trailing newline is the usual cause. Unknown command "<type>".- Either the name is misspelled, or it is one of the seven plugin commands —
setPluginEnabled,configurePlugin,refreshPlugins,installPlugin,removePlugin,addPluginSource,removePluginSource— declared in the shared union and never implemented on the socket. Every one has a working HTTP route. Approval "<id>" is no longer pending.- An
approvefor an approval already decided, or one that timed out and counted as refused. A timeout is a decision, so a client ignoringapproval.requestedgets this rather than a hang. Request body too large./The request body was not valid JSON.- A
413because the body cap is 256 KB and it refuses rather than truncates, and a400raised before any handler sees the request — distinct from a validation failure inside one, which names the field it wanted.
The route that does not exist
The catch-all answers any unknown path under /api/ with 404 and No such endpoint: <path> — the message that proves a route you remember is not there. Cancellation is the obvious one, and because every response is pretty-printed, a terminal shows this:
{
"error": "No such endpoint: /api/runs/run_ab12cd34/cancel"
}
A real response body, two lines because of the indentation, and it matters because cancelling a run looks exactly like an HTTP action. There is no POST /api/runs/:id/cancel: cancellation is the WebSocket cancel command, and answering an approval is socket-only too. Every other mutation has a route, so a console whose socket is down is still an operator's console — except for those two.
Internal error.- A
500with nothing else in it, when a handler threw and no response had been sent. The exception goes to the log aterrorlevel with scopehttp. Upgrade Required/ a socket that closes immediately / a first frame that is nothello- You are not talking to dev3d, or you are on a path that is not
/ws. Any other upgrade path has its socket destroyed, and the whole office state is sent immediately on connect, so a different first frame means something else is on the port. - A command that appears to do nothing at all
- It was accepted and refused, and the reason arrived as an
errorframe. Asubmitthat produced norun.createdhas almost certainly produced anerror. - A command that failed on the server side
- Two outputs: the socket gets an
errorframe with the exception's message, and the terminal a line aterrorlevel with scopews—command "<type>" failed: <message>. That scope tag finds which command a bare client-side message came from.
Producing a useful report
Most of the time spent answering a bug report goes into re-deriving what the reporter already knew. Four things make that go away.
- The health response.
curl -s http://127.0.0.1:8787/api/healthgivesllmMode,llmModeReason,configStale,configStaleDetail,version, the store's backend, uptime, active runs and pending approvals. - The exact log line, with its scope. The bracket tag says which subsystem spoke and the timestamp lines it up with the run. Widen with
DEV3D_LOG_LEVEL=debugand reproduce: the per-provider discovery and health lines exist only at that level. - The routing decision, not a summary of it. For an unexpected model, the whole
reasonclause list plus the considered candidates with their scores and rejection reasons, from the transcript orGET /api/runs/:id. - The turn or tool record itself. Which turn, which employee, its status, its
errorfield, and each tool call's name, status and result preview. For anything that looked like it worked and did not, thaterrorfield is the only place a partial failure is recorded.
What not to paste. Provider API keys and any value named by a variable ending in _API_KEY; marketplace API tokens; the contents of .env, which mixes secrets with machine-specific paths; and full HTTP request headers, where credentials usually hide unnoticed. Also trim the database and any transcript or artifact containing your source code. When in doubt send the error field and the scope tag, and offer the rest.
If the message you are looking at is not on this page, that is the honest boundary of what has been catalogued so far rather than a sign that the behaviour is undocumented. The two routes back are the ones at the top: search the checkout for the literal text, or contact with the log line.
Linked from
Did this page answer your question?