Run Grok Build with a Local Model (Ollama)

Who this is for

Developers who want a full-screen terminal coding agent whose harness they can read and build from source, and who would rather run inference on their own GPU than pay per token. Also useful if you are deciding whether Grok Build's local route is mature enough to rely on yet.

Recommended stack

• Grok Build, installed with the official installer or built from source with cargo
• Ollama serving its OpenAI-compatible endpoint at http://localhost:11434/v1
• A coding model sized to your GPU: qwen2.5-coder:7b for 8 GB, qwen2.5-coder:14b for 12 to 16 GB, muse-glimmer:30b for 24 GB
• The Local LLM Compatibility Checker to confirm the fit before you edit config.toml

What is open here, and what is not

The harness is genuinely open: xAI's repository states that first-party code is licensed under the Apache License, Version 2.0, and the README shows how to build the binary yourself. The model is a separate question. Out of the box the harness talks to Grok 4.6, which is hosted and metered by xAI; at the time of writing the API price is $2 per million input tokens, $0.50 per million cached input tokens and $6 per million output tokens for prompts under 200K tokens, rising to $4, $1 and $12 above that. xAI also states that external contributions are not accepted, and the repository has issues disabled, so this is open source you can read, fork and audit rather than a community project you can steer. Nothing in this guide changes those facts. It only changes which model the harness sends its requests to.

Install the harness

xAI documents two one-line installers: curl -fsSL https://x.ai/cli/install.sh | bash on macOS and Linux, and irm https://x.ai/cli/install.ps1 | iex in Windows PowerShell. Both pipe a remote script straight into your shell, so download and read the script first if that matters to you. The auditable alternative is to build from source. The README asks for Rust (pinned by rust-toolchain.toml, which rustup installs automatically), DotSlash (cargo install dotslash, on your PATH before building) and protoc, and then cargo build -p xai-grok-pager-bin --release produces the release binary; cargo run -p xai-grok-pager-bin builds and launches the TUI. Building yourself is the step that actually cashes in the Apache 2.0 licence, because you know exactly which code is running. Once installed, the config file lives at ~/.grok/config.toml on macOS and Linux and %USERPROFILE%\.grok\config.toml on Windows.

Point config.toml at Ollama

xAI's overview page documents the mechanism in one block: a [model.<id>] table with model, base_url, name and env_key, plus a [models] table whose default names the id to use for new sessions. The settings reference adds the keys that matter for a local server: api_backend, which must be one of chat_completions, responses or messages; context_window, the context size in tokens; api_key as an inline alternative to env_key (the docs prefer env_key); and description for the model picker. Applied to Ollama, whose docs give the base URL http://localhost:11434/v1 and say the API key is required but ignored (the placeholder value is ollama), a working recipe is: under [models] set default = "local-coder"; then under [model.local-coder] set model = "qwen2.5-coder:14b" (or any tag you have pulled), base_url = "http://localhost:11434/v1", name = "Qwen2.5-Coder 14B (Ollama)", api_key = "ollama", api_backend = "chat_completions" and context_window = 32768. Save the file, run grok inspect to confirm the active configuration, and launch with grok -m local-coder or switch inside a session with /model local-coder (alias /m). The environment variables GROK_DEFAULT_MODEL and GROK_MODELS_BASE_URL are documented as equivalents for the default model and inference base URL. Two honesty notes: xAI's docs show the generic custom-model block, and the Ollama-specific values above come from Ollama's own docs, not from an xAI example we could find on 16 August 2026; and OpenSourcesAI has not yet run this configuration end to end.

# ~/.grok/config.toml  (Windows: %USERPROFILE%\.grok\config.toml)
# Keys and allowed values per docs.x.ai/build/overview and /build/settings/reference.
# Ollama values per docs.ollama.com/api/openai-compatibility.
# Documented by xAI and Ollama; not yet tested end to end by OpenSourcesAI.

[models]
default = "local-coder"                    # model id used for new sessions

[model.local-coder]
model = "qwen2.5-coder:14b"                 # any tag you have pulled: ollama pull qwen2.5-coder:14b
base_url = "http://localhost:11434/v1"     # Ollama's OpenAI-compatible endpoint
name = "Qwen2.5-Coder 14B (Ollama)"         # label in the model picker
api_key = "ollama"                          # Ollama ignores the value; xAI's docs prefer env_key over inline api_key
api_backend = "chat_completions"           # chat_completions | responses | messages
context_window = 32768                      # keep in step with OLLAMA_CONTEXT_LENGTH on the Ollama side

# Then: grok inspect        (confirm the active model and base_url)
#       grok -m local-coder (or /model local-coder inside a session)

Choose a model your GPU can actually hold

Run the Local LLM Compatibility Checker first; the tags below are the ones from our catalog we would try, by VRAM tier. On an 8 GB card, qwen2.5-coder:7b (about 6 GB at Q4_K_M) is a purpose-built coder with Ollama's tools capability. On 12 to 16 GB, qwen2.5-coder:14b (about 9 GB at Q4_K_M) is the middle step, with 12 GB the practical minimum. On 24 GB, muse-glimmer:30b (18 GB at Q4_K_M including its vision projector) is Meta's dense agentic model tuned for tool use and long tasks, and 24 GB is the floor rather than the target; qwen3-coder:30b at about 19 GB is the sparse alternative if you want faster tokens per second. Two Ollama details matter for an agent loop. Ollama's default context window is 4,096 tokens, which is far too small for a coding agent that keeps files and tool results in context, and Ollama's docs say the OpenAI-compatible API has no way of setting the context size per request. So start the server with OLLAMA_CONTEXT_LENGTH set (for example OLLAMA_CONTEXT_LENGTH=32768 ollama serve) and set context_window in config.toml to the same figure. Larger contexts cost VRAM, so re-check the fit after you raise it.

First run and sign-in: what is documented, what is not

xAI's overview says that on first launch Grok opens a browser for authentication, and that non-browser environments should set the XAI_API_KEY environment variable instead. The enterprise page adds a device-code login (grok login --device-auth) for SSH sessions, containers and headless hosts, and confirms that third-party endpoints keep working because their base_url is not on x.ai. What the docs do not say is whether setting a custom local model as default lets you skip the xAI sign-in altogether. OpenSourcesAI has not tested it, so plan on signing in once and then confirming with grok inspect that inference is going to localhost. Also read the [models] table carefully: the settings reference lists separate model slots for web_search, session_summary and image_description, and the example config points web_search at grok-4.6. If your goal is that nothing leaves your machine, decide what those slots should do, and check the /privacy command, which the docs describe as showing the privacy and data-retention toggle.

What to expect from local tool calling

A coding agent is a loop of small tool calls: read this file, run that command, apply this edit. Ollama's docs list tools among the supported features of /v1/chat/completions, and Ollama's library shows the tools capability on both qwen2.5-coder and muse-glimmer, so the plumbing exists. Reliability is the model's job, and it degrades as models get smaller: expect a 7B model to occasionally emit a malformed call or stop early, a 14B model to be noticeably steadier, and a 30B agentic model to be the first size that feels comfortable on multi-step tasks. Whether Grok Build's harness handles those failure modes gracefully with a non-Grok model is documented nowhere and not yet tested by OpenSourcesAI. Keep tasks bounded, keep the diff small, and review everything before you accept it, exactly as you would with the hosted model.

The July 2026 upload incident, and why an auditable build matters

Before the source was published, the closed Grok CLI (version 0.2.93) uploaded entire tracked git repositories, including history and any committed secrets, to an xAI-controlled storage bucket regardless of the 'Improve the model' privacy toggle; a security researcher published the analysis on 12 July 2026, xAI stopped the uploads server-side on 13 July without a formal advisory, The Hacker News covered it on 14 July, and the open-source release followed on 15 July. That sequence is the practical case for this guide: run a build you can read, check what your config.toml and the /privacy command actually say, and keep secrets out of tracked files whichever model you point the harness at.

Practical recommendations

• Size the model with the Compatibility Checker before you edit config.toml; treat 24 GB as the floor for muse-glimmer:30b at Q4
• Set api_backend = "chat_completions" explicitly for Ollama and keep context_window in step with OLLAMA_CONTEXT_LENGTH
• Run grok inspect after every config change to confirm which model and base_url are active
• Start with one-file, one-bug tasks and review every diff; widen the scope only once tool calls are reliable on your model
• Prefer building from source over the pipe-to-shell installer if auditability is the reason you chose Grok Build
• Treat everything marked 'documented by xAI, not yet tested by OpenSourcesAI' as something to verify on your own machine first

Tradeoffs

The custom-endpoint route is documented by xAI, not yet tested by OpenSourcesAI, so treat this guide as a recipe to verify on your own machine rather than a guarantee. A 7B to 30B model on one GPU will not match Grok 4.6 on long, multi-file agent runs, tool-call formatting is less reliable on smaller models, and xAI's docs do not say whether a custom default model lets you skip the first-run xAI sign-in entirely. What you gain is inference that stays on your machine, no per-token bill, and a harness whose source you can audit.

Related links

FAQ

Sources