Knowledge base · OpenSourcesAI

Everything you need to build with local AI.

Seven topic areas — from basic concepts to advanced workflows. Jump to the right section using the links below, or start with the orientation if you're new to local AI.

Browse by topicStart the 15-minute orientation
7Topic categories
34Reference entries

Structured, not random

Seven topic areas cover foundations through agents and coding workflows in order.

Start here if new

The beginner orientation walks the same material in a guided, step-by-step path.

Tools, not just docs

Quick-access tools get you to an answer faster than reading a full guide.

Jump to topic

Not sure where to start?

Use the Start Here page for a guided orientation.

The Start Here page walks you through the key concepts, hardware check, and first tools in order — no searching required.

Go to Start HereCheck hardware fitBrowse guides

Quick-access tools

Start with a tool, not a doc

These interactive tools get you to an answer faster than reading a guide.

LLM Compatibility CheckerMatch GPU VRAM to model sizesOpen →Model Builder WizardAnswer 5 questions, get a model shortlistOpen →AI PlaygroundTest prompts in the browser before downloading modelsOpen →

Reference index

Browse by topic

Start here

Local AI Foundations

Core concepts every local AI builder needs before choosing tools, models, or hardware. The first three decide what your machine can run; the next three decide what you build with it.

What is a local LLM?A large language model that runs entirely on your own hardware. Your prompts never leave your machine.GuideWhat is VRAM?GPU memory — the single most important hardware constraint for running local models. Model size must fit here for fast inference.GuideWhat is quantization?Compressing model weight precision (e.g. 16-bit → 4-bit) to cut memory use. Q4 models run on half the VRAM with a small quality tradeoff.GuideWhat is an AI stack?The set of separate programs you run together — model, runtime, interface. Only the first two are required.GuideWhat is RAG?Answering from your own documents by searching them at question time. It does not train the model on your files.GuideWhat is MCP?A standard way to give a model tools — files, APIs, databases — so it can act rather than only answer.GuideOpen-weight vs open-source AIWhy the two terms are not the same claim, and what each licence actually permits you to do.GuideStart here: beginner orientationThe same material as a guided path, in order, with nothing to search for.Overview

Hardware

Hardware & VRAM Planning

How to match model size to GPU memory, quantization tiers, KV cache overhead, CPU offload, and Apple Silicon unified memory considerations.

What is VRAM for local AI?How GPU memory caps model size, plus KV cache overhead and the sizing formulas.GuideLocal LLM compatibility checkerEnter your GPU and RAM, get the model sizes and quantization levels that realistically fit.ToolModel builder wizardAnswer a few questions about your hardware and task, get a shortlist of models to try.ToolSet up local LLMs on WindowsA full Windows setup: Ollama, LM Studio, Open WebUI, coding assistants, and model testing.Guide

Models

Models & Quantization

Open-weight model families, quantization formats (GGUF, Q4, Q8, FP16), parameter counts, context windows, and how to choose the right model for your hardware and task.

Browse all model familiesEvery open-weight family profiled — sizes, licences, strengths, and hardware fit.HubHow to test open-weight coding modelsA repeatable way to compare Qwen, DeepSeek, GLM and others against your own code.GuideChoose a model for coding, RAG, agentsMatch the job to the model family instead of picking by benchmark score.GuideFrontier model comparisonHow the closed frontier models compare with what you can run yourself.Overview

Tools & runtimes

Tools & Runtimes

Local LLM runners (Ollama, LM Studio, Jan), web interfaces (Open WebUI), inference servers (vLLM, GPUStack), and how to connect them into a working workflow.

What is an AI stack?What the layers are and how these tools fit together. Read this before browsing the directory.GuideBrowse all AI toolsThe full reviewed directory — runtimes, interfaces, inference servers, and vector databases.HubOllama tool profileThe most common starting runtime. Terminal-first, with an API behind everything else.ToolLM Studio tool profileA desktop app for browsing, downloading, and testing models without a terminal.ToolOpen WebUI tool profileA browser chat interface that sits in front of Ollama and behaves like the hosted tools.Tool

RAG & data

RAG & Private Data

Retrieval-augmented generation architecture, embedding models, vector databases, chunking strategies, and how to build a private document search workflow locally.

What is RAG?The concept first: what happens at question time, and why it is not fine-tuning.GuideBuild a local RAG stack (Ollama + Qdrant)The full build: document prep, embeddings, Qdrant, and testing retrieval quality.GuideChoose a vector database for RAGChroma, Qdrant, pgvector, Weaviate, Milvus and LanceDB weighed for a small project.GuideWhat is a reranker in RAG?The next layer down — reordering results when search finds the right passages but ranks them badly.GuideQdrant tool profileThe vector database used in most of the local RAG recipes here.Tool

Agents & MCP

Agents & MCP

Model Context Protocol architecture, MCP servers and clients, agent tool use, security boundaries, and how to extend AI agents with reusable external connections.

What is MCP for AI agents?How a model gains tools — files, APIs, databases — through reusable connections.GuideMCP security best practicesWhat an MCP server can reach once connected, and how to keep that boundary tight.GuideMCP servers directoryReviewed MCP servers grouped by what they connect to, with install entries and evidence grades.DirectoryBrowse MCP integrationsThe full integration directory across clients and servers.Hub

Coding & IDE

Coding Assistants & IDE Workflows

AI coding assistants, IDE integrations, local vs hosted models for code, code review agents, and how to set up Continue, Cline, Cursor, or Windsurf with a local backend.

Best open-source AI coding assistantsThe assistants that work against a local backend, ranked and licence-labelled.Best-ofCursor vs Windsurf vs ClineThree editors compared on local model support, pricing, and day-to-day workflow.CompareContinue tool profileAn open-source IDE plugin you can point at your own runtime instead of a vendor API.ToolCline tool profileAn autonomous coding agent for VS Code that can run against local models.Tool