Run your coding agent on your own hardware

Updated August 10, 2026

Goal: run an agent in crystl against a model you host yourself, instead of the frontier API.

What you need

  • A model server you can reach: Ollama, LM Studio, vLLM, or llama.cpp, on your Mac or on a box on your network.
  • For Claude Code, an Anthropic-compatible server. Current LM Studio, vLLM, and llama.cpp releases provide a Messages endpoint directly; put a translation proxy such as LiteLLM in front of Ollama. Codex, aider, opencode, and goose can use the Ollama or OpenAI-compatible connection directly.
  • Configuring open-model connections and reading the reachability line are free on every plan. Starting an open-model shard from the desktop app is not gated either. Only crystl shard create from the CLI is a Guild control command.

Steps

  1. Start the server. For example, ollama serve on the machine with the GPU. For Claude Code, put a proxy in front of it:

    litellm --model ollama/qwen3-coder --port 4000

  2. Add an open-model connection. Open Settings → agents → providers & keys, scroll past the hosted keys to open models, and add one:

    FieldValue
    Labelcloset-box
    Kindanthropic-compatible for the Claude Code proxy, Ollama or OpenAI-compatible for everything else
    Base URLthe address as seen from your Mac, for example http://localhost:4000
    Keythe proxy or server key if it has one, empty if it doesn’t

  3. Check that it answers before you spend a shard on it:

    crystl status

  4. Open a new shard. Environment variables are set when a process starts, so a shard that was already open keeps the old values. From the CLI you can name the slot explicitly:

    crystl shard create --gem myapp --agent codex --local closet-box \
  --prompt "fix the lint errors in src/"

    --local is checked before any shard opens. A label that doesn’t exist, is disabled, or is the wrong kind for that agent gets you an error listing your slots instead of a worker that quietly ran somewhere else.

What you’ll see when it works

crystl status prints a reachability line next to the memory telemetry:

local model: closet-box http://100.x.y.z:11434 reachable (loaded: qwen3-coder · pulled: +5)

loaded means the model is in VRAM and will answer now. pulled means it’s on disk and cold, and the first request can take minutes with no output while it loads. In a Claude Code shard, echo $ANTHROPIC_BASE_URL returns your slot’s URL rather than nothing.

Two honest limits. A good local coder model is genuinely useful on mechanical work and will disappoint you on an architectural decision, so keep the hard calls with the strongest model you have. And a local worker under a hosted manager is not an air gap: the manager’s whole conversation still goes to a hosted API.

Go deeper