Models and routing

This page is for organization admins and install operators. It covers how the Auto, Smart, Balanced and Cheap presets are routed, how to change and test that routing, and where the install decides which models exist and what they cost. For what people see in the model menu, see Choose a model.

Who sets what

Model settings sit in the same three layers as other controls (see Enterprise controls).

What	Who	Where
Which providers and models the install’s gateway serves	The install admin	Admin Console, LLM Configuration group, applied after the app restarts with the new settings
Gateway models added live, the default model and the small model	The setup operator (setup password)	Setup console, Models and Gateway pages
The install’s default routing	The setup operator (setup password)	The setup API
The organization’s own routing	Organization admins and owners	Evals > Routing

Open Routing

Routing lives in Evals > Routing. Only organization admins and owners can manage it. Anyone else sees “You need the Admin or Owner role to manage model routing.”

Routing used to be under Settings > Models. That page was removed, and its old address now opens Routing.

The page has three parts:

Tiers: the tier rows, the fallback, and a JSON button that opens the same settings as JSON.
Presets: one line on what each preset does. “All four read the tiers above; none has settings of its own.”
Try it: a tester that routes sample messages.

The tiers document

All four presets are built from one tiers document. It holds four tiers, cheapest first, and a fallback.

Tier	Built-in label	Meant for	Built-in model
SIMPLE	Fast	Quick, simple asks: a fact, a short rewrite, a greeting	The install’s small model
MEDIUM	Balanced	Everyday work: drafting, summarizing, explaining, routine code	The install’s small model
COMPLEX	Deep	Hard work: long multi-step tasks, debugging, design	The install’s default model
REASONING	Deepest	The hardest: careful reasoning, proofs, decisions under tradeoffs	The install’s default model

Label: what people read in the menu and in a reply’s footer. Up to 40 characters.
Model: any model the install’s gateway serves. Save refuses a model the gateway doesn’t serve.
Fallback: “The tier a task runs on when the router can’t decide.” The built-in fallback is Balanced (MEDIUM). A task also goes to the fallback when the small model fails to answer.

The built-in models are a starting point, taken from the default model and small model set in the setup console. You can change every tier.

Each preset limits the tiers it can use. A tier outside the range runs as the nearest tier inside it, and the fallback moves into range too.

Preset	Tiers	How it decides
Auto	All four	The small model reads the first message, or a wording score if there is no small model
Smart	Deep, Deepest	Same as Auto
Balanced	Fast, Balanced	Same as Auto
Cheap	Fast	A wording score, with no model call

The decision is made on a task’s first message, and the task keeps that model. A task idle for more than 30 days is decided again on its next message.

Two facts follow from the tier models:

Images. The presets accept images only if every tier model can read images. If one can’t, the organization’s Routing page says “A tier model can’t read images, so these presets don’t accept images.”
Context window. A preset offers the smallest context window among the models it can land on, so a task never outgrows the tier it runs on.

Edit the tiers

Go to Evals > Routing.
In Tiers, change a tier’s name, pick its model, or pick a Fallback.
Check the line under the rows. It says “Unsaved changes. Try them below first.” while you have changes. Discard throws them away.
Test the draft in Try it (see the next section).
Click Save as this org’s routing the first time, or Save routing after that. New tasks use the saved routing. Tasks already running keep their model.

To edit as JSON, click JSON to open Tiers as JSON:

Copy or Download the current draft. The download is named routing-tiers.json.
Edit as JSON puts the draft in the text box, or Upload file loads a file into it.
When the box shows “Valid tiers document.”, click Use this JSON to replace the draft. Then test and save as usual.

A tiers document looks like this. The model names are placeholders; use the names your gateway serves.

{
  "version": 2,
  "tiers": {
    "SIMPLE": { "label": "Fast", "modelId": "litellm/small-model" },
    "MEDIUM": { "label": "Balanced", "modelId": "litellm/small-model" },
    "COMPLEX": { "label": "Deep", "modelId": "litellm/default-model" },
    "REASONING": { "label": "Deepest", "modelId": "litellm/default-model" }
  },
  "fallback": "MEDIUM"
}

Problems are listed by field, and nothing is applied until the whole document is valid.

Test routing

Try it sends sample first messages through the router and shows where each lands. No task is created and nothing is saved.

Paste messages, one per line, and press the arrow. Or click Try 12 examples.
With unsaved changes, it routes “Against your unsaved tiers”. Otherwise it routes “Against the saved routing”.
Results show each message, the tier it Went to, the model, How it was decided, and the time. With a draft, the summary also counts how many messages moved tier.
Clear empties the results.

Limits: up to 50 messages per run, each up to 20,000 characters, and 20 runs an hour per organization.

Organization routing and the install default

An organization starts on the install default. Routing then shows “Using the install default”, with the install’s tiers.

Your first save creates the organization’s own routing. Routing then shows “This organization’s own routing” (“Other organizations are not affected.”).
Reset to the install default removes the organization’s routing. New tasks use the install default, and tasks already running keep their model.
Deleting the organization removes its routing too.
Every save and reset is written to the server log with who made it and the tiers before and after.

If the install’s Auto isn’t set up yet, Routing shows suggested tiers, the presets are off for every organization, and Save stays off until an operator sets up the install’s Auto. You can still test tiers.

The install default is set by the setup operator through the setup API, behind the setup password. It writes all four install presets from one tiers document. If the operator removes the install’s routing, the presets are off for every organization, and tasks on them run on the default model. An organization’s own routing is kept and works again once the install’s routing is back.

Which models are served

Every model in the menu, and every model a tier can use, is one the install’s model gateway serves. The presets only route among those models.

Install admin (Admin Console). On a self-hosted install, the LLM Configuration group sets which providers and models exist. Changes apply after the app restarts with the new settings. For the security effect of these settings, see Model gateway and third parties.

Setting	What it sets
Enabled LLM Providers	Which providers the install uses. The default is the LiteLLM gateway.
Run LiteLLM in-cluster	Runs the model gateway inside the cluster. On by default.
LiteLLM Base URL and LiteLLM API Key	An external gateway to use instead.
Upstream: AWS Bedrock (via pod IAM role) and Bedrock model IDs	Bedrock models the in-cluster gateway serves.
Upstream: Anthropic API key and Anthropic model IDs	Anthropic models the in-cluster gateway serves.
Upstream: OpenAI API key and Upstream: OpenAI-compatible base URL (optional)	OpenAI models, or an OpenAI-compatible endpoint, for the in-cluster gateway.
OpenAI-compatible upstream models (with pricing)	Models served through an OpenAI-compatible upstream. Each entry declares its own per-token price, or its calls are recorded at $0.
Live gateway model management (/setup)	Lets the setup operator add and remove gateway models live, with no redeploy. On by default with the in-cluster gateway.
Retired Model Ids	Maps a retired model to the model that serves it now. A task on a retired model moves to the first listed successor the install serves, and the task says so. Without an entry, it moves to the default model.
Forward User Email to LiteLLM	Tags each model request with the sender’s email, so the gateway’s spend reports can name the person. On by default.

Setup operator (setup console). The Models page sets the Default model (“Used for agent turns”) and the Small model (“Used for titles and compaction, and to pick an Auto tier”), then Save models. Running sessions pick up the change on their next launch. The same page adds custom models and renames models in the picker. The Gateway page imports models into the gateway and shows gateway usage.

Applets. Applets that call AI use the install’s models too. The Admin Console’s applet inference settings turn it on or off (on by default), set an optional default model for applets (blank uses the deployment default), and set an optional model allowlist. A blank allowlist offers every served model, and a malformed one turns applet inference off.

Credits and spend

On a deployment that bills in credits, each model call’s cost is recorded from the gateway’s spend log into the billing ledger and converted to credits.

Conversion. Credits are the gateway’s cost, times the inference cost multiplier, divided by the credit value. The deployment’s operator sets the credit value and the cost multiplier, and new usage is rated with the saved values.
Model choice drives spend. The tier models you pick decide what most tasks cost. Cheap keeps every task on the Fast tier, and Balanced keeps tasks off the two top tiers.
Tester calls. Routing tester runs use the install’s gateway key. They are recorded as install spend, not billed to the organization.
Applets. Applet model calls are billed to the organization that owns the applet, with a per-owner daily credit ceiling (20,000 credits by default).
Where to see it. Settings > Usage shows spend from the billing ledger, including Top models. The Billing section shows the plan and credit balance where purchases are available. When a workspace runs out of credits or reaches its spending limit, running tasks pause and show a notice, with an Upgrade action where purchases are available.

Next steps

Choose a model: what people see in the model menu and how to pick a preset.
Enterprise controls: the three layers of control and when each change applies.
Security configuration reference: gateway settings and their security effect.
Roles: who holds the Admin and Owner roles that manage routing.
Evals: compare models on your own work before you change a tier.