Ollama + Open WebUI Starter Stack

Stack components

• Ollama: local model runtime and API.
• Open WebUI: self-hosted chat interface for local and hosted model backends.
• Models: start with smaller Qwen, Gemma, Phi, Mistral, or DeepSeek distilled models.
• Optional RAG layer: add Qdrant, Chroma, or another vector database later.

Hardware requirements

Start smaller than you think. A reliable 7B or 8B quantized model is a better first milestone than a large model that barely fits and makes every test slow.

Recommended first models

• Qwen3 Coder for coding experiments.
• Gemma 4 or Gemma-family models for general chat and local testing.
• Phi-4 Mini for small-model laptop tests.
• Mistral Small 3.1 for efficient local chat workflows.

Setup path

1. Install Ollama on your local machine.
2. Pull one small model first instead of starting with a huge checkpoint.
3. Run a few prompts directly through Ollama.
4. Install or run Open WebUI and connect it to Ollama.
5. Create a test workspace and compare two or three models on the same prompts.
6. Add RAG, vector search, or MCP tools only after the basic chat stack is stable.

Best for

• Beginners building their first private local AI setup
• Developers testing local models before wiring them into apps
• Teams that want a private browser chat interface over local models
• Builders who want a foundation for RAG, coding assistants, and MCP workflows

Related pages