Providers, models and routing

This page covers the turn that will not run: a provider the office cannot see, a key that is found and refused, a model id that was never a candidate, and a routing decision whose winner you did not expect. If the turns run and the money is the problem, read Budget, cost and stuck runs instead.

What these failures have in common

Every turn involves two separate decisions, and the messages tell you which one failed. First the router chooses a model from a pool that has already been filtered down to providers this installation can actually reach; then the engine calls that model, and if the call fails it walks an ordered fallback list before giving up. A routing problem and a provider problem therefore produce completely different evidence, and reading the wrong one wastes the most time on this page.

The rule of thumb: if a turn ran — if it produced text, or cost money, or told you a model name — the problem is in the routing decision, and the turn's own record contains the whole explanation. If a turn failed with an HTTP status in the message, the problem is between the process and the vendor, and nothing the router chose would have helped with the same key.

Two things to check before the rest of this page. The boot log's provider line and llmModeReason tell you which providers are configured and why the mode resolved the way it did. And a turn's route, servedBy and attemptedRoutes together tell you what the router chose, what actually answered, and what was tried in between. Both decisions are recorded per turn rather than summarised, which is the whole point of the routing design; see Model routing for the mechanism itself.

providers configured: none, although the key is set

What it means

Keys are read from the environment once, at module load. The file is loaded first, but only into variables that are not already set: a real environment variable always wins over the file, and it wins even when its value is empty. An empty or whitespace-only value is then treated as absent. So the failure mode is not "the file was not read" but "a variable of that name already existed and was empty", which is exactly what an exported empty key or a shell profile that sets a placeholder produces.

Keys are also trimmed, which handles a stray space but not a placeholder value or a masked key pasted with an ellipsis in it.

What to check

1. Check the variable in the process environment, not the file. On Windows, Get-ChildItem Env:DEEPSEEK_API_KEY; on macOS and Linux, printenv DEEPSEEK_API_KEY. An empty result with the file set is the shadowing case.
2. Restart the process. The environment is read once, so a key added to a running installation changes nothing until the process is replaced.
3. Check the value has no whitespace-only or placeholder content. Keys are trimmed; a value of spaces counts as absent.
4. Check configStale at the health endpoint. It reports a key that has appeared since startup, which distinguishes "not set" from "set after the process started".
5. Check the mode is not forced to mock. The reason string says so explicitly when it is, and reports the providers it is overriding.
6. Check for a keyless local provider being the only configured one. providers configured: local is not none, and it changes the installation to live. See the local-runtime symptom below.

all model routes failed: HTTP 401 from <provider>

What it means

The engine called the routed model, the adapter received a non-success HTTP response, and the engine moved on to the next candidate. When every candidate has failed, the registry throws one error containing the whole chain, separated by semicolons, with each entry shaped providerId/modelId: message — and the adapter's message is the HTTP status, the provider id and the first part of the vendor's own response body.

That structure is the useful part. A 401 or 403 on every entry means the credential itself, because it fails identically on every model of that provider. A 401 on one provider and a success elsewhere means the credential for that one provider. And an entry reading <key> (unavailable) is not an HTTP failure at all: the provider or the model was not in the catalog, which means the route pointed somewhere the installation does not have.

What to check

1. Read every entry in the chain, not the first. The first failure is the routed model; the rest are the fallbacks, and their pattern is the diagnosis.
2. Confirm the key with one direct call. A curl to the provider's models endpoint with the same key settles whether the credential is the problem, without involving the office at all.
3. Check the base URL if the provider is not one of the four built-ins. The four shipped providers have real defaults, and a local or plugin-contributed provider is whatever base URL was configured for it.
4. Check the key is on the right provider. A DeepSeek key in the OpenAI variable produces a well-formed 401 from the wrong vendor.
5. Look at the turn record. attemptedRoutes keeps the same chain on the turn, and servedBy is absent when nothing answered.

HTTP 429 from <provider>

What it means

Rate limiting is a transport failure like any other, and it is now the one kind the adapter retries rather than merely surviving. A 429 — along with 408, 409, 425 and any 5xx — is retried up to twice with exponential backoff and jitter, capped at ten seconds, before the attempt is abandoned and the engine moves to the next candidate in the fallback list. What the retry does not do is wait out a long rate-limit window: two attempts over a few hundred milliseconds is a hiccup, not a quota. For parallel stages the pressure is amplified, because several employees call the provider at the same time up to the configured concurrency.

Nothing in the office treats a 429 as a special condition. There is no queue, no sleep and no circuit breaker; the only mitigation the design offers is the fallback walk, which helps only if the fallback models are on a different provider or a different tier with its own quota. The endpoint-health signal, when it is enabled, demotes a provider observed to be unreliable — but only for the provider ids configured for it, which default to the multi-vendor router rather than every vendor.

What to check

1. Confirm it is rate limiting and not quota. The vendor's own response body is included in the message after the status; a 429 with a quota message is a billing problem, not a burst problem.
2. Look at the stage that failed. A parallel build stage is the widest case, and its concurrency is DEV3D_MAX_CONCURRENCY, defaulting to 4 and clamped between 1 and 16.
3. Check whether a fallback answered. If servedBy names a different model from route, the office is already coping, and the cost model behind the decision is the thing to look at rather than the failure.
4. Lower the concurrency if bursts are the trigger. It is an installation setting: change it in the console rather than in the file after the first boot.
5. Consider a lower posture for the busy stage. cheap aims at the policy's minimum tier, which spreads the same work across cheaper models and different quotas.

A turn succeeded on a model the router did not choose

What it means

This is failover working, and it is recorded separately on purpose. The engine builds an ordered fallback list at routing time, tries the primary route first, and steps through the list until a call returns. Which model actually answered is a fact about that model's health, so it is stored as servedBy rather than being folded back into the routing decision — crediting the routed model for work a fallback did would make the routing record a lie.

The fallback order is deliberately not the cheapest-first order: same-tier models on other providers come first, ordered by their own score, and then at most one model from one tier up. So a fallback is usually a comparable model rather than a downgrade.

What to check

1. Read attemptedRoutes. It names the provider and model of every failed attempt with its error, which is why the routed model did not answer.
2. Treat a persistent pattern as a health problem, not a one-off. If one provider fails on most turns, the fallback is masking a broken key or a provider outage.
3. Check the route's fallback list. It is part of the decision record, so you can see what the engine would reach for before it happens.
4. Check whether the failure is a cancellation. A user-initiated abort is rethrown rather than routed around, so a cancelled turn will not show a fallback chain.

pinned model 'x' is not in the catalog

What it means

A role's model policy has two fields that are easy to confuse, and these messages are how the confusion surfaces. pin is a boolean that switches off the posture and escalation steps, honouring the task-class tier mapping exactly. preferredModelId names one concrete model and is a preference with a bounds check: it is honoured only inside the role's own minimum and maximum tier, and every way it can fail produces a note in the reason rather than a silent fallback.

"Not in the catalog" means the id does not exist in the merged catalog the router can see — which includes discovery results, not just the curated table. A typo, a model the provider no longer serves, or a model id that belongs to a provider this installation has no key for all land here, and all three are indistinguishable from the reason alone.

What to check

1. Query the catalog rather than the vendor. The model list route answers with what the office can actually see, which is the list the router reads.
2. Check the provider for that model is configured. Models on an unconfigured provider stay catalogued so the console can show them, but they never enter the routing pool.
3. Read the whole note. "Outside this policy's bounds" is not a catalog problem at all: the model exists and the policy forbids the tier, and the note spells out the tier and the bounds.
4. Decide which field you meant. If the goal is "this role must never use a bigger model", the policy's ceiling is the honest expression of that; a pin that contradicts the bounds is reported rather than obeyed, which is deliberate.

The router chose a model you did not expect

What it means

The decision is recorded in full on every turn, as a clause list joined with em dashes, and reading it is the entire diagnosis. It names the task class the stage produced, the tier the policy mapped it to, any posture or escalation note that changed the target, the clamp into the policy's bounds if one fired, any relaxation the capability filter had to take, the model that won, and the score explanation for why it won.

Four causes explain nearly all of these surprises. The posture may not be what you think, because a floor can override the installation default rather than inheriting it. The target tier may have moved: cheap aims at the policy's minimum, quality aims one tier up, and a nearly exhausted run budget forces the cheap target regardless of the posture. Escalation may have fired from the estimated complexity of the text. And a plugin routing rule that names a tier pulls that tier to the front of the walk without being clamped to the policy band.

The relaxation notes deserve their own warning. Capability filtering is not a hard failure: if nothing in the pool meets the requirements, the router drops the tool and vision requirements, then the context requirement, then falls back to the whole catalog, recording which relaxation it took. A turn that asked for tool calling can therefore be served by a model that cannot call tools, and the turn will answer from the model alone.

What to check

1. Read the route reason end to end. Every clause names the step that produced it, including which step moved the target tier and why.
2. Check the floor's posture, not the environment variable. The routing variable sets the installation default; a floor that has been given a posture of its own overrides it.
3. Check the turn's complexity against the role's escalation threshold. The reason says when escalation fired and what it escalated to.
4. Check the run's remaining budget. Under five cents remaining, the target is forced to the cheap tier with a note saying so.
5. Check for installed plugin routing rules. A rule that names a tier moves the walk, and a rule with no task class applies to every task class.
6. Check whether a relaxation fired, and treat it as the explanation rather than as noise: it usually means the catalog lacks something the turn needed.

The same decision is available three ways, which is useful when you are looking at a finished run rather than a live one: attached to the turn as its route, emitted on the wire as a routing event, and summarised onto the employee as their last route. The full mechanism, including the weighted score, is on Model routing.

no model in the catalog satisfied the request

What it means

That reason is the router's hard guard for an empty pool, and the odd-looking failure after it is the same fact one step later: with no candidate chosen, the route carries an empty provider and model, and the provider registry reports the combination it was handed — a bare slash — as unavailable.

An empty pool has few causes. Nothing is configured, which is the live-mode-with-no-providers case. Or every model that could have been a candidate was removed by an exclusion, which is how a provider observed to be down is taken out of the pool. Or the installation has providers configured but discovery has withdrawn every model they serve, which is the case worth checking when the provider list is not empty.

What to check

1. Read llmModeReason. It is the fastest way to distinguish "nothing configured" from "configured and empty".
2. Compare the boot log's model count with zero. The boot banner prints skills: ... | tools: ... | models: ...; a model count that collapsed to nothing points at the catalog rather than the keys.
3. Ask the model list route what is routable. Models on an unconfigured provider are catalogued for display but never routed, so a healthy-looking list is not proof of a non-empty pool.
4. Check whether discovery is on and failing. Discovery can be disabled or its cache pointed away; when it cannot reach a provider, the curated overlay is what remains.

Every model is unrated and routing looks simplistic

What it means

Cost pressure drops to zero when no model in the pool carries a quality rating, and that rule is deliberate: letting price pull a turn off the tier the policy asked for is only meaningful when something is known about what the money buys. With a completely unrated catalog every candidate gets the same population-mean priors, the quality terms cancel out, and the outcome is the pre-scoring behaviour exactly.

Ratings come from sources that are all optional. A curated table ships with the office, a shared quality index is used when the variable it names holds a key, and a published-benchmark reader runs unless it is switched off. Each has a cache with a freshness window and each can be disabled, and none of them is required for the office to run. Their absence is not an error and is not logged as one.

What to check

1. Check the signal coverage the registry reports. The console exposes how much of the catalog is rated, which tells you whether the sources are contributing anything.
2. Check the variable named by the pooled-quality key setting. The default is ARTIFICIAL_ANALYSIS_API_KEY, and the setting holds the name of the variable rather than the key, so an obvious-looking mistake here is to put the key in the wrong place.
3. Check the benchmark reader is not switched off, and that its cache is writable if it has been pointed somewhere.
4. Check no model in the pool is rated. The rule is about the whole pool, so one rated model is enough to bring cost pressure back.

My provider serves a model that is not in the list

What it means

The curated table is an overlay rather than a roster. At boot, unless discovery is switched off, the office asks each configured provider what models it actually serves and treats that answer as membership: a model the curated table has never heard of is still routable, flagged as unrated, and a model the provider no longer serves is withdrawn rather than left in place to fail a turn. Mock mode never discovers, so a keyless office keeps its full demonstrable catalog.

Two honest consequences follow. A model can be reachable and still be unrated, which changes how the router weighs it rather than whether it can be chosen. And if the provider's own list endpoint is unreachable or answers in a shape the office does not understand, the curated overlay is what you are looking at — which is a smaller catalog that will look like a missing model.

What to check

1. Ask the provider's own list endpoint with the configured key. If the vendor does not list the model, the office cannot know about it.
2. Check discovery is enabled for this installation. It is on by default and only a literal false disables it.
3. Check the discovery cache. A stale cache may still hold the previous answer; the freshness window is a setting, and clearing the file costs one network round trip on the next boot.
4. Check the model list route in the console against the catalog route. They answer different questions, and only one of them is the routing pool.
5. Add or correct the entry in the settings screen if the model is real but unrated. The catalog prices and quality figures exist so that routing and reporting can be ordered plausibly; they are estimates, not billing truth.

Turns are going to a local runtime

What it means

The local provider is the one keyless provider that ships. It counts as configured on the strength of its base URL alone, which is what makes it usable without a credential and also what makes it easy to enable by accident: setting the variable at all, with any value, is the whole switch. Its presence is also what moves an auto mode installation into live, so the side effect can be billing on a paid provider as well as local inference.

The curated table describes exactly one local entry, at the small tier, so a stronger local model is still priced and tiered as if it were that one until the entry is corrected or discovery replaces it.

What to check

1. Check whether the base URL variable is set at all. An empty value counts as absent and does not enable the provider, but any non-empty value does.
2. Check the boot provider line. providers configured: local is the confirmation, and it is listed alongside any keys.
3. Check the mode reason. With auto, a keyless local provider alone is enough for live; the reason says which providers were found.
4. Check the local runtime is actually running. A local provider that is configured but unreachable produces ordinary HTTP failures at turn time, which is the flaky-instead-of-scripted case.
5. Decide between local inference and a free demonstration. The two cannot both be true: local counts as configured, so it is live. If you want a deterministic free office, unset the variable or force mock mode.

A plugin's provider never serves anything

What it means

Provider-key filtering happens before the router is called, and it is the same rule for a contributed provider as for a built-in one: a provider with no key and no keyless declaration never contributes candidates. The provider's models stay catalogued so the console can show them and so a policy can name them, which is exactly why this looks like a routing bug rather than a missing key.

Routing to an unconfigured provider is not a fallback, it is a guaranteed failed turn. With plugins able to declare providers, forgetting the key is easy, and the pin case is how it becomes loud.

What to check

1. Check the provider's configured flag in the console, which distinguishes catalogued from routable.
2. Check the environment variable the plugin declared. A plugin names the variable; the value is read from the process environment, so a plugin can never ship a credential itself.
3. Check whether the provider is keyless by declaration. A keyless provider counts as configured from its base URL, and its base URL is subject to the same https-or-loopback rule as any other.
4. Check the plugin is loaded and enabled. Contributions are withdrawn when a plugin is disabled. See Plugins and skills.

When it is not in this list

Collect the two records rather than the screenshot, because between them they contain the whole story:

• The provider picture at boot: the provider line, the mode and its reason, and the model count.
• The turn's own record: its route with the full reason string, its servedBy, its attemptedRoutes, its error and its usage.
• The exact failure text, including the vendor's response body, which the adapter includes after the HTTP status.
• Which provider answered last successfully, and whether the problem follows the provider, the model or the role.
• Whether it reproduces in mock mode, which separates the routing logic from the vendor entirely.

Send it through contact. If the turn runs and only the bill is surprising, go to Budget, cost and stuck runs; if turns fail while touching files, go to Workspace, shell and tools.