Stack
Ollama + Open WebUI Starter Stack
A practical local AI stack for beginners: Ollama runs the models, and Open WebUI gives you a private browser-based chat workspace on top.
Bill of materials
Model runner
Runs local models via a REST API on localhost:11434 — the standard local inference backend for this stack
Chat interface (optional)
Self-hosted browser chat UI; connects to Ollama at localhost:11434 with no cloud dependency — runs via Docker
Starter models
Lightweight 8B model — strong general chat, fits on 8 GB VRAM at Q4_K_M quantization
Very small footprint — good choice for laptops and low-VRAM tests where speed matters more than depth
Google open-weight model — efficient, well-documented, reliable default for beginners
Install Ollama and run your first model
1. Install Ollama
One-line install for macOS and Linux. On Windows, download the installer from ollama.com/download.
# macOS / Linux curl -fsSL https://ollama.ai/install.sh | sh # Windows (winget) winget install Ollama.Ollama
2. Pull a small model first
Start with an 8B model. Larger models are available once you know the stack is working.
ollama pull qwen3:8b
3. Run the model in your terminal
ollama run qwen3:8b
4. Add Open WebUI (browser interface via Docker)
Run this after Ollama is working. Opens at http://localhost:3000 — first-run sign-up creates a local-only account with no external connections.
docker run -d \ -p 3000:8080 \ --add-host=host.docker.internal:host-gateway \ -v open-webui:/app/backend/data \ --name open-webui \ ghcr.io/open-webui/open-webui:main
Ran the commands? Measure what your setup actually does → — speed, load time, and whether anything is silently throttling it.
Stack components
- Ollama: local model runtime and API.
- Open WebUI: self-hosted chat interface for local and hosted model backends.
- Models: start with smaller Qwen, Gemma, Phi, Mistral, or DeepSeek distilled models.
- Optional RAG layer: add Qdrant, Chroma, or another vector database later.
Hardware requirements
Start smaller than you think. A reliable 7B or 8B quantized model is a better first milestone than a large model that barely fits and makes every test slow.
Recommended first models
- Qwen3 Coder for coding experiments.
- Gemma 4 or Gemma-family models for general chat and local testing.
- Phi-4 Mini for small-model laptop tests.
- Mistral Small 3.1 for efficient local chat workflows.
Setup path
- Install Ollama on your local machine.
- Pull one small model first instead of starting with a huge checkpoint.
- Run a few prompts directly through Ollama.
- Install or run Open WebUI and connect it to Ollama.
- Create a test workspace and compare two or three models on the same prompts.
- Add RAG, vector search, or MCP tools only after the basic chat stack is stable.
Best for
- Beginners building their first private local AI setup
- Developers testing local models before wiring them into apps
- Teams that want a private browser chat interface over local models
- Builders who want a foundation for RAG, coding assistants, and MCP workflows