# Open Models

> Run Claude Code, Codex, and other coding agents on open models hosted on your hardware or by a managed service. Set them up once, orchestrate them alongside frontier agents, and cut token costs.

Open models can run on your own hardware or through a managed host such as [z.ai](https://docs.z.ai/devpack/overview). crystl gives both the same setup: save the connection once, check that it is reachable, and point a shard at it by name.

That makes open models useful for more than a local experiment. Keep architecture and difficult debugging on a frontier model, then route test runs, lint fixes, migrations, documentation passes, and other token-heavy work to lower-cost open-model workers. They stay visible and steerable as normal crystl shards.

The setup lives with your hosted provider keys: **Settings → agents → providers & keys** (this page used to be called api keys). Hosted keys are at the top, open-model connections below them. See [API key storage](/docs/api-keys/) for the hosted-key settings.

## Tutorials

- [Use z.ai open models with crystl](/docs/open-models-zai/): create the Coding Plan account and key, connect the hosted GLM models, and start your first Claude Code shard.
- [Use Ollama open models with crystl](/docs/open-models-ollama/): install Ollama, download a coding model, and start a Codex shard on your Mac or another machine.
- [Use LM Studio open models with crystl](/docs/open-models-lm-studio/): download and serve a model through the desktop app, then connect Claude Code directly.
- [Use vLLM open models with crystl](/docs/open-models-vllm/): serve a tool-capable model on a Linux GPU machine and connect Codex over its OpenAI-compatible API.
- [Use llama.cpp open models with crystl](/docs/open-models-llama-cpp/): install the native server, run a GGUF coding model, and start a Codex shard.

## What a connection stores

A connection is a saved description of one model server. It has:

| Field | What it does |
|-------|--------------|
| Label | A name for the slot, so you can tell your laptop from the closet box |
| Kind | **Ollama**, **OpenAI-compatible** (LM Studio, vLLM, llama.cpp), or **anthropic-compatible** |
| Base URL | Where the endpoint lives, as seen from your Mac |
| As seen from remote | Optional. The URL an agent running on another machine should use instead (see [remote vantage](#remote-vantage-when-the-agent-is-somewhere-else)) |
| Key | Optional. A per-slot token, stored in your Keychain and shown masked |
| Env var aliases | Which environment variables crystl exports for this slot. Editable, so an unusual tool can get the variable it actually reads |
| Enable toggle | Off means crystl exports nothing for this slot |

The defaults cover the common cases, so most people never touch the aliases:

| Kind | Typical base URL | Exports |
|------|------------------|---------|
| Ollama | `http://localhost:11434` | `OLLAMA_API_BASE`, `OLLAMA_HOST` |
| OpenAI-compatible | `http://localhost:1234/v1` | `OPENAI_API_BASE`, `OPENAI_BASE_URL` |
| anthropic-compatible | `http://localhost:4000` | `ANTHROPIC_BASE_URL`, plus `ANTHROPIC_AUTH_TOKEN` / `ANTHROPIC_API_KEY` |

vLLM and llama.cpp both speak the OpenAI API, so they use the OpenAI-compatible kind with their own port.

## Claude Code on open models

Claude Code reads `ANTHROPIC_BASE_URL`, so an **anthropic-compatible** slot is all it takes. When a shard is served by one of these slots, crystl owns the auth variables for that shard too: it sends the slot's own key, or a documented dummy value when the endpoint doesn't need one.

That last part is the safety rule, and it is worth stating plainly. **A real hosted Anthropic key is never sent to a configured endpoint.** If you point Claude Code at somebody else's server, your frontier key stays home.

There are two ways people use this, and both are first-class.

### A local Anthropic-compatible server

Current LM Studio, vLLM, and llama.cpp servers expose the Anthropic Messages API directly. Save their base URL as an **anthropic-compatible** connection and Claude Code can use it without a proxy. LM Studio documents the complete path in its [Claude Code guide](https://lmstudio.ai/docs/integrations/claude-code).

Ollama does not expose the same Messages endpoint through crystl's Ollama connection. Put a translation proxy such as [LiteLLM](https://docs.litellm.ai/) in front of it when you want Claude Code rather than Codex or another OpenAI/Ollama-native agent.

```bash
# on the machine with the GPU
litellm --model ollama/qwen3-coder --port 4000
```

Then add a connection:

| Field | Value |
|-------|-------|
| Label | `proxy-box` |
| Kind | anthropic-compatible |
| Base URL | `http://localhost:4000` |
| Key | leave empty if the proxy is open, or paste the proxy's master key |

Open a new shard and Claude Code runs on your own hardware. Nothing about the prompt leaves the machine.

### A hosted open model

You do not need your own GPU. Managed services such as z.ai host open models and provide an Anthropic-compatible URL for Claude Code. Get the service's API key, save its base URL and key in a slot, and Claude Code runs against its models instead of the frontier API. Follow the [z.ai tutorial](/docs/open-models-zai/) for the full account-to-shard setup.

| Field | Value |
|-------|-------|
| Label | `z-ai` |
| Kind | anthropic-compatible |
| Base URL | `https://api.z.ai/api/anthropic` |
| Key | the key from that provider |

Two honest things about this. These are **third-party services with their own terms**, their own retention policy, and their own view of your code, so read the terms the same way you would for any vendor. And **quality varies by task**: one of these can be great at mechanical work and weak on the reasoning-heavy change, and that ordering is not stable between providers or over time. Try it on your own work rather than trusting a benchmark. We don't rank them here and we don't quote prices, because both change faster than this page does.

## Codex on open models

Codex has its own flags for open models, and crystl passes them for you:

```bash
crystl shard create --gem myapp --agent codex --local closet-box \
  --prompt "fix the lint errors in src/"
```

That resolves to `codex --oss --local-provider ollama` for an Ollama slot, or `--local-provider lmstudio` for an OpenAI-compatible one. Verified against codex-cli 0.147.0.

If your Ollama isn't on the default address, crystl also exports `CODEX_OSS_BASE_URL`. Treat that one as **best effort**: it comes from the provider table inside the Codex binary rather than from documented behavior, so it may change under you. On a default-port setup you need nothing at all.

## The lightweight agents

aider, opencode, and goose still work the way they always did. They read the endpoint URL out of the environment, and `--local` validates that the slot exists before the shard opens.

```bash
crystl shard create --gem myapp --agent aider    --local closet-box --prompt "fix the lint errors in src/"
crystl shard create --gem myapp --agent opencode --local closet-box --prompt "run the test suite and report failures"
crystl shard create --gem myapp --agent goose    --local closet-box --prompt "rename the helper across the repo"
```

goose is launchable now too, steered by `GOOSE_MODEL`. It's prompt-run form only, so give it a `--prompt`.

### Model id shapes

The CLIs name models differently, and getting this wrong is the most common first-run stumble.

- **aider** takes LiteLLM-style ids: `provider/model`. For Ollama, use `ollama_chat/qwen3-coder` rather than `ollama/qwen3-coder`. Both prefixes resolve, but `ollama_chat` uses the chat endpoint and gives noticeably better results.
- **opencode** takes `provider/model`, for example `ollama/qwen3-coder`.
- **goose** takes the bare model name in `GOOSE_MODEL`.

## Spawning with --local

`--local <label>` names the endpoint you want, and crystl resolves it per agent:

| Agent | What `--local` does |
|-------|---------------------|
| codex | Adds `--oss` and the matching `--local-provider`, plus `CODEX_OSS_BASE_URL` when the address is non-default |
| claude | Checks that the shard's environment really points at that endpoint. The env vars do the steering |
| aider, opencode, goose | Validates the slot. The environment carries the URL, same as before |

The check happens **before any shard opens**. Name a slot that doesn't exist, or one that's disabled, or one whose kind is wrong for the agent you asked for, and you get an error listing your configured slots instead of a shard that quietly ran against the wrong model. A worker that silently used the frontier API when you meant to use your GPU is an expensive surprise, so crystl would rather fail early.

## Per-slot keys

The key field solves three problems:

- **Hosted open models** need their provider's key, and it should not be your Anthropic key.
- **vLLM started with `--api-key`** rejects anything without a matching token.
- **OpenAI-SDK tools refuse to start with no `OPENAI_API_KEY` at all**, even against a keyless local server. A keyless slot exports the dummy value `local` so those tools start.

Keys live in your macOS Keychain, not in a config file, and the settings field shows them masked.

## Verify it with crystl status

You don't have to guess whether the endpoint is up. `crystl status` prints a reachability line next to the memory telemetry:

```bash
$ crystl status
memory: app 412 MB · pressure normal
local model: closet-box http://100.x.y.z:11434 reachable (loaded: qwen3-coder · pulled: +5)
```

Three states:

- `reachable`: something answered.
- `unreachable`: nothing answered at that URL from your Mac's vantage.
- `checking…`: the first probe is still running. Read it again in a moment.

### Loaded versus pulled

For Ollama slots the reachable line splits the models in two, and the difference matters for how long your first request takes.

- **loaded** means the model is in VRAM right now. Ask it something and it answers.
- **pulled** means the model is on disk but cold. The first request loads it, and on a big model that can take minutes with no output while it happens.

So `loaded: qwen3-coder · pulled: +5` reads as: one model ready to go, five more you could use if you're willing to wait for the load. If a worker looks hung on its first turn, this line is usually the answer.

### Auth challenges count as reachable

anthropic-compatible and OpenAI-compatible slots probe `/v1/models`. A `401` or `403` back is treated as **reachable**, on purpose. Something is listening and it told you your credentials are the problem, which is a different fix from a dead address. Check the key, not the URL.

The probe is cached for about 20 seconds and times out after 1.5, so `status` never blocks or hangs on a sleeping server. The same information rides on `GET /gems`, so the [API](/docs/cli/) and the [phone app](/docs/mobile-app/) see it too.

### Remote vantage: when the agent is somewhere else

Here's the case that trips people up. You have a home server with a GPU, and you reach it from your Mac over Tailscale at `http://100.x.y.z:11434`. That address is correct from your desk. It is the wrong address for an agent running *on that server over [SSH](/docs/remote-ssh/)*, because from there the endpoint is simply localhost.

That's what **as seen from remote** is for. Fill in both:

| Field | Value |
|-------|-------|
| Base URL | `http://100.x.y.z:11434` |
| As seen from remote | `http://localhost:11434` |

Local shards get the base URL. SSH shards get the remote-vantage URL. If you leave the remote field empty, SSH shards fall back to the base URL, which is the right answer whenever the address works from both sides.

## How the URLs reach your agents

Every **new** shard gets the enabled slots' URLs as environment variables at launch. Three rules worth knowing:

- **You always win.** If your shell profile already exports `OLLAMA_HOST`, crystl leaves it alone. Anything you export yourself beats anything crystl would have injected, so a per-project override in `.zshrc` or a `direnv` file keeps working exactly as before.
- **Existing shards don't change.** Environment variables are set when a process starts, so a shard that was already running when you edited the slot keeps the old values. Open a new shard, or restart the one you're in.
- **Disabled means absent.** Turning a slot off doesn't set an empty value, it just stops exporting.

## Let crystl fill in the model names

You don't have to go read the endpoint's model list yourself. Ask for a suggestion:

```bash
crystl agent profile suggest --agent codex --slot closet-box
```

crystl reads the models the endpoint serves and proposes a small, standard, and large mapping for that agent. It shows you the mapping and waits for a yes. **It never writes a profile unprompted.** The [agents & models](/docs/model-sizes/) settings tab does the same thing by click: probed model names fill the tier fields when you pick them.

Set it by hand instead if you'd rather:

```bash
crystl agent profile set --agent aider \
  --small ollama_chat/<your-small-model> \
  --standard ollama_chat/qwen3-coder \
  --large ollama_chat/<your-largest-model> \
  --model-via '--model {model}'
```

After that, anything that asks for a size gets the right model.

## Fanning out onto your own hardware

A local endpoint is one machine with one pool of memory, and that changes the arithmetic an orchestrator is used to. Hosted agents scale out; your GPU does not. Guidance for an [orchestrating agent](/docs/fanout/):

- **Check the status line before fanning out.** The reachability line is a precondition, not a nice-to-have. Read it first, then decide.
- **Keep it to one or two local workers.** Past that they queue behind each other on the same hardware and everything gets slower, including the one you were waiting on.
- **Unreachable means route around it, not debug it.** If the endpoint isn't answering, spawn a hosted agent for that task and move on. Chasing a sleeping GPU is not the orchestrator's job.
- **Idle is not finished.** Workers on hookless agents (aider, opencode, goose) report done or blocked automatically when their process exits, so trust that signal. Mid-run `idle` is a guess based on CPU, and a worker waiting on a cold model looks exactly like a worker that has stopped. Wait for the exit report.

A hosted open model doesn't have the one-machine problem, so fan out on it the way you would on any hosted agent.

## Honest caveats

**Know what leaves the network.** "Local model" and "private" are not the same claim, and the topology decides which one you get.

| Topology | What leaves your network |
|----------|--------------------------|
| Agent on your Mac, model on your Mac or your LAN box | Nothing. This is the private one |
| Local model worker, hosted agent orchestrating it | The orchestrator's whole conversation still goes to a hosted API. The worker being local doesn't change that |
| Claude Code against a hosted open model | Everything Claude Code sends goes to that provider instead of Anthropic. Different vendor, not fewer vendors |

So a local worker under a hosted manager is not an air gap. Only the first row is.

**Open models are not frontier models.** A good open coder model is genuinely useful on mechanical work. Give it an architectural decision and you may be disappointed. Match the task to the model, and keep the hard calls with the strongest model you have.

## Troubleshooting

**The agent's answers are garbled, or tool calls print as raw JSON:**

The server's context window is almost always the cause. Ollama defaults to 4096 tokens, and an agent harness like Claude Code or Codex carries a standing prompt bigger than that on its own, so the server silently truncates it and the model loses the plot. crystl warns about this two ways: `crystl status` shows a `ctx` warning next to `reachable`, and a `--local` spawn tells you in its create response. The fix lives on the server, not in crystl:

```bash
# systemd (Linux)
sudo systemctl edit ollama
# add under [Service]:
#   Environment="OLLAMA_CONTEXT_LENGTH=16384"
sudo systemctl restart ollama
```

On a Mac running Ollama directly, `launchctl setenv OLLAMA_CONTEXT_LENGTH 16384` and restart the app. 16384 is a sane floor for agent work; more costs RAM.

**Every turn takes minutes:**

Check `crystl status` for a `cpu-only` tag. That means the loaded model reports no GPU memory in use, so inference runs at CPU speed, and every turn also re-processes the prompt at that speed. Your options, in order of effect: use a smaller model (a 7B is several times faster than a 14B on CPU), accept it for short tasks only, or move the endpoint to a machine with a GPU the server can use. A `cpu-only` endpoint is fine for one worker on small tasks. Do not fan out onto it.

**`unreachable` in `crystl status`:**

1. Is the endpoint actually running? `curl http://localhost:11434/api/tags` for Ollama, or `curl http://localhost:1234/v1/models` for an OpenAI-compatible or anthropic-compatible one.
2. Is the address right *from your Mac's vantage*? crystl probes from your Mac, so `localhost` in the base URL means the Mac's localhost, never the remote box's.
3. For an endpoint on another machine, check the firewall and the tunnel. Ollama binds to loopback by default, so a LAN or Tailscale peer can't reach it until you set `OLLAMA_HOST=0.0.0.0` on the *server*. On Tailscale, confirm both machines are up in `tailscale status`.

**`reachable` but every request is rejected:**

The probe counts a `401` or `403` as reachable on purpose, so this is the auth case. Put the provider's key in the slot's key field, and check you copied the whole thing.

**Claude Code still hits the frontier API:**

The shard was probably open before you added the slot. Environment variables are fixed when a process starts, so open a new shard. If it's a fresh shard, run `echo $ANTHROPIC_BASE_URL` in it: empty means the slot is disabled or the wrong kind, and your own export in `.zshrc` wins over crystl's if you have one.

**`--local` errored and no shard opened:**

That's the design. The message lists your configured slots, so compare the label you typed against that list, and check the slot is enabled and the right kind for the agent.

**The agent starts but can't find a model:**

- Check the model id shape. `ollama_chat/qwen3-coder` for aider, `ollama/qwen3-coder` for opencode.
- Confirm the model is actually pulled. The reachable line lists what the endpoint is serving; if your model isn't in that list, pull it first.
- Tags are part of the name. A bare model name and the same name with a `:tag` suffix are two different models to the server, so copy the id from the reachable line exactly.

**The first request takes forever, then works:**

Cold model. Check whether the status line has it under `pulled` rather than `loaded`. Loading a large model into VRAM can take minutes, and there's no progress output while it happens.

**It worked in a new shard but not the one I was in:**

That's the expected behavior. Environment variables are fixed when a process starts. Open a new shard or restart the current one after any change to a slot.

**A variable has the wrong value:**

Something in your shell profile is exporting it, and your export wins on purpose. Run `echo $OLLAMA_HOST` in the shard to see what it resolved to, then either remove your export or set the slot's base URL to match it.

## Availability

Configuring open-model connections and reading the reachability line in `crystl status` are free on every plan. Spawning shards from the CLI with `crystl shard create` is a [Guild](/pricing/) control command like the rest of the CLI's mutations. Starting a shard on an endpoint from the desktop app is not gated.

---
Source: https://crystl.dev/docs/open-models/