Getting started · OpenSourcesAI
Start Here with OpenSourcesAI
OpenSourcesAI helps you choose and deploy practical AI tools, local LLMs, models, hardware, and workflows — with compatibility checks, PC build guidance, setup notes, comparisons, and beginner-friendly deployment guides. Tell us what you want to do below and we'll point you to the right place.
Step 0
What would you like to do today?
Pick your goal and we'll show you the best place to start, why, and what to do next.
What would you like to do today?
What is it?
What is OpenSourcesAI?
OpenSourcesAI is a practical guide and toolset for choosing, comparing, and deploying AI tools, local LLMs, models, hardware, and workflows. It is not just a list of AI tools — it is built to help you go from “I don't know where to start” to a working setup on your own machine.
The site is built to help you answer:
- What AI tool should I use?
- What model can my computer actually run?
- What hardware do I need, and what should I buy?
- Should I use local AI or cloud AI for this?
- What setup problems should I avoid?
- What guide should I follow next?
Who is it for?
Built for anyone deciding how to run AI
Local AI, defined
What is a local LLM?
A local LLM (large language model) is an AI model that runs entirely on your own computer — not in the cloud. Instead of sending your prompts to a server run by OpenAI or Google, the model weights live on your disk and inference happens on your CPU or GPU. Your data never leaves your machine.
The practical result: no third party sees your prompts, it works offline, no provider can change the model, price or terms under you, latency stays low enough for always-on agents, and there is no per-token fee. Hosted inference is now inexpensive for light use, so the case for local is control and privacy first, cost second. The tradeoff is hardware — the model has to fit in your GPU memory, which limits how large it can be.
Full explanation: What is a local LLM? →Foundation · Can you run it
Three concepts that decide what your machine can run
You don't need to memorize these. You just need enough of each to avoid the most common mistakes.
What is a local LLM?
A large language model that runs entirely on your own hardware. Your prompts never leave your machine.
Read the full guide →What is VRAM?
GPU memory — the single most important hardware constraint for running local models. Model size must fit here for fast inference.
Read the full guide →What is quantization?
Compressing model weight precision (e.g. 16-bit → 4-bit) to cut memory use. Q4 models run on half the VRAM with a small quality tradeoff.
Read the full guide →Foundation · What you're building
Three concepts that decide what you build
The three above answer whether a model will run. These three answer what you assemble around it — and they are the terms used everywhere else on this site.
What is an AI stack?
The set of separate programs you run together — model, runtime, interface. Only the first two are required.
Read the full guide →What is RAG?
Answering from your own documents by searching them at question time. It does not train the model on your files.
Read the full guide →What is MCP?
A standard way to give a model tools — files, APIs, databases — so it can act rather than only answer.
Read the full guide →Hardware tiers
What your hardware can run
Find your GPU or memory tier below to know what model sizes are realistic before you download anything.
CPU only or 4–6 GB VRAM
1B–3B models
Good for learning the workflow. Slow for daily use.
8 GB VRAM
7B–8B models at Q4
The most common starting tier. Fast enough for daily chat and coding.
12–16 GB VRAM
7B (Q8), 13B–14B (Q4)
More model variety and context headroom. Comfortable for most builders.
24 GB VRAM or 64 GB Apple Silicon
32B (Q4), 70B (Apple Silicon)
Consumer high-end. Covers nearly all everyday AI workflows.
Before you download
Check your hardware first
The most common beginner mistake is downloading a 14B model on 8 GB VRAM and watching it crawl or crash. The compatibility checker takes 60 seconds and tells you which model sizes realistically fit your GPU memory.
Prefer a plain-language warm-up first? Read the 60-second local LLM quickstart →
First build path
Five steps to your first working local AI setup
Follow these in order. Each step builds on the last. Do not skip ahead to RAG or coding assistants until plain chat is working reliably.
Check your hardware
Before downloading any model, check your GPU memory. The compatibility checker gives you a shortlist of model sizes that fit your hardware.
Open compatibility checker →Install Ollama
Ollama is the fastest path from zero to a running local model. It handles download, runtime, and a local API in a single install.
Ollama setup guide →Pull one small model and test it
Run: ollama pull qwen3:8b — then test a few prompts. Log response speed and answer quality before adding any other tools. This baseline matters.
Add Open WebUI for a better chat workspace
Once Ollama is running, Open WebUI adds a browser-based chat interface. Add it only after the model itself is working.
Ollama + Open WebUI stack →Add RAG, coding tools, or agents when ready
Only after plain chat quality is acceptable on your hardware. Each layer adds complexity — add one at a time and test between steps.
Build a local RAG stack →Choose your starting path
Or find your goal in this list
The same idea as the picker above, laid out as a quick reference.
| If your goal is… | Start here | Then go to |
|---|---|---|
| I want to know what my computer can run | Compatibility Checker → | Model Builder Wizard → |
| I want to build or buy a PC for AI | AI PC Builder → | Linux Local AI Setup Guide → |
| I want to choose a model | Models directory → | Comparisons → |
| I want to find AI tools | Tools directory → | Best-of pages → |
| I want to build a workflow | Model Builder Wizard → | Stacks → |
| I am new and overwhelmed | Beginner workflow → | Compatibility Checker → |
Essential tools
The tools most beginners need first
Start with Ollama. Add LM Studio or Open WebUI only after you have one model running. Everything else comes later.
Ollama
The simplest CLI runtime for local models. Exposes a local API that other tools connect to.
CLI · Free · Open sourceView profile →LM Studio
Desktop GUI with a model browser, chat interface, and local server. Best for visual workflows.
Desktop · FreeView profile →Open WebUI
Browser-based chat workspace for Ollama and OpenAI-compatible backends. Self-hostable.
Web · Open sourceView profile →Continue
Open-source AI coding assistant for VS Code and JetBrains. Connects to local or cloud models.
IDE · Open sourceView profile →Qdrant
Local vector database for RAG workflows. Add it once your base chat setup is working.
Vector DB · Open sourceView profile →Model Builder Wizard
Answer a few questions and get a shortlist of model families that fit your use case and hardware.
Tool · Free · On this siteView profile →Main page directory
Where everything lives
A one-line map of the major sections of the site.
What makes OpenSourcesAI different?
More than a list of tools
Most sites either teach AI, list AI tools, or offer an isolated local LLM calculator. OpenSourcesAI connects discovery, compatibility, hardware planning, deployment warnings, and setup guides into one practical path — so you're not stitching together five different sites to go from “what should I use?” to a working setup.
- Not just “best tools” lists — compatibility checks against your actual hardware
- Hardware-aware recommendations, not generic ones
- Setup notes and evidence-backed deployment risks, not just install commands
- A practical next step after every page, not a dead end
- Beginner-readable guides, written to be understood, not skimmed and abandoned
Common questions
FAQ
Do I need a powerful computer to run a local LLM?
Not necessarily. A laptop or desktop with 8 GB of GPU VRAM can run practical 7B-class models at a comfortable speed. CPU-only setups work for small models too, just more slowly. Use the compatibility checker to see what your specific hardware supports.
Is this safe and private?
Yes — that is one of the main reasons people choose local AI. Your prompts and documents never leave your machine. There is no cloud service receiving your data. The model itself is open-weight, meaning its weights are publicly available and you can audit what you are running.
How is this different from ChatGPT?
ChatGPT runs on OpenAI's servers. When you type a message, it travels over the internet to their infrastructure and back. A local LLM runs inference entirely on your hardware — nothing leaves your machine. Local models are generally smaller and less capable than the latest frontier models, but for many everyday tasks the difference is smaller than you might expect.
What is the easiest way to try this right now?
Install Ollama from ollama.com, then run: ollama run llama3:8b. That is it. The first run downloads the model (~4 GB), then you can chat in the terminal immediately. Add Open WebUI afterward if you want a browser interface.
Your actual next step
Pick whichever of these matches what you're here to do. There's no wrong choice — they all connect back to each other.