Open Models

Open models can run on your own hardware or through a managed host such as z.ai. crystl gives both the same setup: save the connection once, check that it is reachable, and point a shard at it by name.

That makes open models useful for more than a local experiment. Keep architecture and difficult debugging on a frontier model, then route test runs, lint fixes, migrations, documentation passes, and other token-heavy work to lower-cost open-model workers. They stay visible and steerable as normal crystl shards.

The setup lives with your hosted provider keys: Settings → agents → providers & keys (this page used to be called api keys). Hosted keys are at the top, open-model connections below them. See API key storage for the hosted-key settings.

Tutorials

• Use z.ai open models with crystl: create the Coding Plan account and key, connect the hosted GLM models, and start your first Claude Code shard.
• Use Ollama open models with crystl: install Ollama, download a coding model, and start a Codex shard on your Mac or another machine.
• Use LM Studio open models with crystl: download and serve a model through the desktop app, then connect Claude Code directly.
• Use vLLM open models with crystl: serve a tool-capable model on a Linux GPU machine and connect Codex over its OpenAI-compatible API.
• Use llama.cpp open models with crystl: install the native server, run a GGUF coding model, and start a Codex shard.

What a connection stores

A connection is a saved description of one model server. It has:

Field	What it does
Label	A name for the slot, so you can tell your laptop from the closet box
Kind	Ollama, OpenAI-compatible (LM Studio, vLLM, llama.cpp), or anthropic-compatible
Base URL	Where the endpoint lives, as seen from your Mac
As seen from remote	Optional. The URL an agent running on another machine should use instead (see remote vantage)
Key	Optional. A per-slot token, stored in your Keychain and shown masked
Env var aliases	Which environment variables crystl exports for this slot. Editable, so an unusual tool can get the variable it actually reads
Enable toggle	Off means crystl exports nothing for this slot

The defaults cover the common cases, so most people never touch the aliases:

Kind	Typical base URL	Exports
Ollama	http://localhost:11434	OLLAMA_API_BASE, OLLAMA_HOST
OpenAI-compatible	http://localhost:1234/v1	OPENAI_API_BASE, OPENAI_BASE_URL
anthropic-compatible	http://localhost:4000	ANTHROPIC_BASE_URL, plus ANTHROPIC_AUTH_TOKEN / ANTHROPIC_API_KEY

vLLM and llama.cpp both speak the OpenAI API, so they use the OpenAI-compatible kind with their own port.

Claude Code on open models

Claude Code reads ANTHROPIC_BASE_URL, so an anthropic-compatible slot is all it takes. When a shard is served by one of these slots, crystl owns the auth variables for that shard too: it sends the slot’s own key, or a documented dummy value when the endpoint doesn’t need one.

That last part is the safety rule, and it is worth stating plainly. A real hosted Anthropic key is never sent to a configured endpoint. If you point Claude Code at somebody else’s server, your frontier key stays home.

There are two ways people use this, and both are first-class.

A local Anthropic-compatible server

Current LM Studio, vLLM, and llama.cpp servers expose the Anthropic Messages API directly. Save their base URL as an anthropic-compatible connection and Claude Code can use it without a proxy. LM Studio documents the complete path in its Claude Code guide.

Ollama does not expose the same Messages endpoint through crystl’s Ollama connection. Put a translation proxy such as LiteLLM in front of it when you want Claude Code rather than Codex or another OpenAI/Ollama-native agent.

# on the machine with the GPU
litellm --model ollama/qwen3-coder --port 4000

Then add a connection:

Field	Value
Label	proxy-box
Kind	anthropic-compatible
Base URL	http://localhost:4000
Key	leave empty if the proxy is open, or paste the proxy’s master key

Open a new shard and Claude Code runs on your own hardware. Nothing about the prompt leaves the machine.

A hosted open model

You do not need your own GPU. Managed services such as z.ai host open models and provide an Anthropic-compatible URL for Claude Code. Get the service’s API key, save its base URL and key in a slot, and Claude Code runs against its models instead of the frontier API. Follow the z.ai tutorial for the full account-to-shard setup.

Field	Value
Label	z-ai
Kind	anthropic-compatible
Base URL	https://api.z.ai/api/anthropic
Key	the key from that provider

Two honest things about this. These are third-party services with their own terms, their own retention policy, and their own view of your code, so read the terms the same way you would for any vendor. And quality varies by task: one of these can be great at mechanical work and weak on the reasoning-heavy change, and that ordering is not stable between providers or over time. Try it on your own work rather than trusting a benchmark. We don’t rank them here and we don’t quote prices, because both change faster than this page does.

Codex on open models

Codex has its own flags for open models, and crystl passes them for you:

crystl shard create --gem myapp --agent codex --local closet-box \
  --prompt "fix the lint errors in src/"

That resolves to codex --oss --local-provider ollama for an Ollama slot, or --local-provider lmstudio for an OpenAI-compatible one. Verified against codex-cli 0.147.0.

If your Ollama isn’t on the default address, crystl also exports CODEX_OSS_BASE_URL. Treat that one as best effort: it comes from the provider table inside the Codex binary rather than from documented behavior, so it may change under you. On a default-port setup you need nothing at all.

The lightweight agents

aider, opencode, and goose still work the way they always did. They read the endpoint URL out of the environment, and --local validates that the slot exists before the shard opens.

crystl shard create --gem myapp --agent aider    --local closet-box --prompt "fix the lint errors in src/"
crystl shard create --gem myapp --agent opencode --local closet-box --prompt "run the test suite and report failures"
crystl shard create --gem myapp --agent goose    --local closet-box --prompt "rename the helper across the repo"

goose is launchable now too, steered by GOOSE_MODEL. It’s prompt-run form only, so give it a --prompt.

Model id shapes

The CLIs name models differently, and getting this wrong is the most common first-run stumble.

• aider takes LiteLLM-style ids: provider/model. For Ollama, use ollama_chat/qwen3-coder rather than ollama/qwen3-coder. Both prefixes resolve, but ollama_chat uses the chat endpoint and gives noticeably better results.
• opencode takes provider/model, for example ollama/qwen3-coder.
• goose takes the bare model name in GOOSE_MODEL.

Spawning with —local

--local <label> names the endpoint you want, and crystl resolves it per agent:

Agent	What --local does
codex	Adds --oss and the matching --local-provider, plus CODEX_OSS_BASE_URL when the address is non-default
claude	Checks that the shard’s environment really points at that endpoint. The env vars do the steering
aider, opencode, goose	Validates the slot. The environment carries the URL, same as before

The check happens before any shard opens. Name a slot that doesn’t exist, or one that’s disabled, or one whose kind is wrong for the agent you asked for, and you get an error listing your configured slots instead of a shard that quietly ran against the wrong model. A worker that silently used the frontier API when you meant to use your GPU is an expensive surprise, so crystl would rather fail early.

Per-slot keys

The key field solves three problems:

• Hosted open models need their provider’s key, and it should not be your Anthropic key.
• vLLM started with --api-key rejects anything without a matching token.
• OpenAI-SDK tools refuse to start with no OPENAI_API_KEY at all, even against a keyless local server. A keyless slot exports the dummy value local so those tools start.

Keys live in your macOS Keychain, not in a config file, and the settings field shows them masked.

Verify it with crystl status

You don’t have to guess whether the endpoint is up. crystl status prints a reachability line next to the memory telemetry:

$ crystl status
memory: app 412 MB · pressure normal
local model: closet-box http://100.x.y.z:11434 reachable (loaded: qwen3-coder · pulled: +5)

Three states:

• reachable: something answered.
• unreachable: nothing answered at that URL from your Mac’s vantage.
• checking…: the first probe is still running. Read it again in a moment.

Loaded versus pulled

For Ollama slots the reachable line splits the models in two, and the difference matters for how long your first request takes.

• loaded means the model is in VRAM right now. Ask it something and it answers.
• pulled means the model is on disk but cold. The first request loads it, and on a big model that can take minutes with no output while it happens.

So loaded: qwen3-coder · pulled: +5 reads as: one model ready to go, five more you could use if you’re willing to wait for the load. If a worker looks hung on its first turn, this line is usually the answer.

Auth challenges count as reachable

anthropic-compatible and OpenAI-compatible slots probe /v1/models. A 401 or 403 back is treated as reachable, on purpose. Something is listening and it told you your credentials are the problem, which is a different fix from a dead address. Check the key, not the URL.

The probe is cached for about 20 seconds and times out after 1.5, so status never blocks or hangs on a sleeping server. The same information rides on GET /gems, so the API and the phone app see it too.

Remote vantage: when the agent is somewhere else

Here’s the case that trips people up. You have a home server with a GPU, and you reach it from your Mac over Tailscale at http://100.x.y.z:11434. That address is correct from your desk. It is the wrong address for an agent running on that server over SSH, because from there the endpoint is simply localhost.

That’s what as seen from remote is for. Fill in both:

Field	Value
Base URL	http://100.x.y.z:11434
As seen from remote	http://localhost:11434

Local shards get the base URL. SSH shards get the remote-vantage URL. If you leave the remote field empty, SSH shards fall back to the base URL, which is the right answer whenever the address works from both sides.

How the URLs reach your agents

Every new shard gets the enabled slots’ URLs as environment variables at launch. Three rules worth knowing:

• You always win. If your shell profile already exports OLLAMA_HOST, crystl leaves it alone. Anything you export yourself beats anything crystl would have injected, so a per-project override in .zshrc or a direnv file keeps working exactly as before.
• Existing shards don’t change. Environment variables are set when a process starts, so a shard that was already running when you edited the slot keeps the old values. Open a new shard, or restart the one you’re in.
• Disabled means absent. Turning a slot off doesn’t set an empty value, it just stops exporting.

Let crystl fill in the model names

You don’t have to go read the endpoint’s model list yourself. Ask for a suggestion:

crystl agent profile suggest --agent codex --slot closet-box

crystl reads the models the endpoint serves and proposes a small, standard, and large mapping for that agent. It shows you the mapping and waits for a yes. It never writes a profile unprompted. The agents & models settings tab does the same thing by click: probed model names fill the tier fields when you pick them.

Set it by hand instead if you’d rather:

crystl agent profile set --agent aider \
  --small ollama_chat/<your-small-model> \
  --standard ollama_chat/qwen3-coder \
  --large ollama_chat/<your-largest-model> \
  --model-via '--model {model}'

After that, anything that asks for a size gets the right model.

Fanning out onto your own hardware

A local endpoint is one machine with one pool of memory, and that changes the arithmetic an orchestrator is used to. Hosted agents scale out; your GPU does not. Guidance for an orchestrating agent:

• Check the status line before fanning out. The reachability line is a precondition, not a nice-to-have. Read it first, then decide.
• Keep it to one or two local workers. Past that they queue behind each other on the same hardware and everything gets slower, including the one you were waiting on.
• Unreachable means route around it, not debug it. If the endpoint isn’t answering, spawn a hosted agent for that task and move on. Chasing a sleeping GPU is not the orchestrator’s job.
• Idle is not finished. Workers on hookless agents (aider, opencode, goose) report done or blocked automatically when their process exits, so trust that signal. Mid-run idle is a guess based on CPU, and a worker waiting on a cold model looks exactly like a worker that has stopped. Wait for the exit report.

A hosted open model doesn’t have the one-machine problem, so fan out on it the way you would on any hosted agent.

Honest caveats

Know what leaves the network. “Local model” and “private” are not the same claim, and the topology decides which one you get.

Topology	What leaves your network
Agent on your Mac, model on your Mac or your LAN box	Nothing. This is the private one
Local model worker, hosted agent orchestrating it	The orchestrator’s whole conversation still goes to a hosted API. The worker being local doesn’t change that
Claude Code against a hosted open model	Everything Claude Code sends goes to that provider instead of Anthropic. Different vendor, not fewer vendors

So a local worker under a hosted manager is not an air gap. Only the first row is.

Open models are not frontier models. A good open coder model is genuinely useful on mechanical work. Give it an architectural decision and you may be disappointed. Match the task to the model, and keep the hard calls with the strongest model you have.

Troubleshooting

The agent’s answers are garbled, or tool calls print as raw JSON:

The server’s context window is almost always the cause. Ollama defaults to 4096 tokens, and an agent harness like Claude Code or Codex carries a standing prompt bigger than that on its own, so the server silently truncates it and the model loses the plot. crystl warns about this two ways: crystl status shows a ctx warning next to reachable, and a --local spawn tells you in its create response. The fix lives on the server, not in crystl:

# systemd (Linux)
sudo systemctl edit ollama
# add under [Service]:
#   Environment="OLLAMA_CONTEXT_LENGTH=16384"
sudo systemctl restart ollama

On a Mac running Ollama directly, launchctl setenv OLLAMA_CONTEXT_LENGTH 16384 and restart the app. 16384 is a sane floor for agent work; more costs RAM.

Every turn takes minutes:

Check crystl status for a cpu-only tag. That means the loaded model reports no GPU memory in use, so inference runs at CPU speed, and every turn also re-processes the prompt at that speed. Your options, in order of effect: use a smaller model (a 7B is several times faster than a 14B on CPU), accept it for short tasks only, or move the endpoint to a machine with a GPU the server can use. A cpu-only endpoint is fine for one worker on small tasks. Do not fan out onto it.

unreachable in crystl status:

1. Is the endpoint actually running? curl http://localhost:11434/api/tags for Ollama, or curl http://localhost:1234/v1/models for an OpenAI-compatible or anthropic-compatible one.
2. Is the address right from your Mac’s vantage? crystl probes from your Mac, so localhost in the base URL means the Mac’s localhost, never the remote box’s.
3. For an endpoint on another machine, check the firewall and the tunnel. Ollama binds to loopback by default, so a LAN or Tailscale peer can’t reach it until you set OLLAMA_HOST=0.0.0.0 on the server. On Tailscale, confirm both machines are up in tailscale status.

reachable but every request is rejected:

The probe counts a 401 or 403 as reachable on purpose, so this is the auth case. Put the provider’s key in the slot’s key field, and check you copied the whole thing.

Claude Code still hits the frontier API:

The shard was probably open before you added the slot. Environment variables are fixed when a process starts, so open a new shard. If it’s a fresh shard, run echo $ANTHROPIC_BASE_URL in it: empty means the slot is disabled or the wrong kind, and your own export in .zshrc wins over crystl’s if you have one.

--local errored and no shard opened:

That’s the design. The message lists your configured slots, so compare the label you typed against that list, and check the slot is enabled and the right kind for the agent.

The agent starts but can’t find a model:

• Check the model id shape. ollama_chat/qwen3-coder for aider, ollama/qwen3-coder for opencode.
• Confirm the model is actually pulled. The reachable line lists what the endpoint is serving; if your model isn’t in that list, pull it first.
• Tags are part of the name. A bare model name and the same name with a :tag suffix are two different models to the server, so copy the id from the reachable line exactly.

The first request takes forever, then works:

Cold model. Check whether the status line has it under pulled rather than loaded. Loading a large model into VRAM can take minutes, and there’s no progress output while it happens.

It worked in a new shard but not the one I was in:

That’s the expected behavior. Environment variables are fixed when a process starts. Open a new shard or restart the current one after any change to a slot.

A variable has the wrong value:

Something in your shell profile is exporting it, and your export wins on purpose. Run echo $OLLAMA_HOST in the shard to see what it resolved to, then either remove your export or set the slot’s base URL to match it.

Availability

Configuring open-model connections and reading the reachability line in crystl status are free on every plan. Spawning shards from the CLI with crystl shard create is a Guild control command like the rest of the CLI’s mutations. Starting a shard on an endpoint from the desktop app is not gated.