Hardware tier · 8GB VRAM
What Can 8GB VRAM Run? Local AI at the 8GB Tier
8GB of GPU VRAM is the entry point for local AI. It runs the smaller model classes at full precision and 7B/8B models at Q4 — enough for private chat, lightweight coding help, and small document Q&A. It is best understood as an entry-level, existing-owner tier: if you already have an 8GB card such as an RTX 4060, RTX 3070, or RX 6600, it is a real way into local LLMs. If you are buying new in 2026, this guide explains why it is usually worth starting at 12GB or more.
Editorial review
This page covers the 8GB VRAM tier as it applies across multiple GPUs (RTX 4060, RTX 3070, RTX 3060 Ti, RX 6600 / 6650 XT, Arc A750 / A580). These cards share an 8GB model ceiling but differ significantly in runtime maturity — CUDA is the most turnkey path. VRAM estimates are derived from GGUF quantization formulas for roughly 4-bit weights and require additional memory for quantization metadata, runtime overhead, KV cache, and context length. Verify current pricing and availability before purchasing.
Verdict: a real entry point, but not a 2026 buying target
8GB covers a lot of everyday local AI at the smaller end — but it sits a full tier below the practical 12GB floor, and headroom is what it gives up.
- 1B–4BFull precision — fast and comfortable
- 7B–8BQ4 with room for context; higher quant means a short window
- 13BNot practical — weights nearly fill 8GB
- 14B+CPU offload only — slow, not interactive
Good fit for
- Existing 8GB-card owners starting local AI with no upgrade
- Everyday chat and tools at 1B–8B
Wrong fit for
- A fresh purchase aimed at local AI — 12GB is the sensible floor
- 13B–14B as a comfortable full-GPU workload
8GB VRAM sits a full tier below the 12GB most guides treat as the practical starting point for local AI. What it does well is the smaller end of the spectrum: 1B–4B models at full precision, and 7B/8B models at Q4 with room for a reasonable context window. That covers a lot of everyday local AI. What it gives up is headroom — higher quantization on 7B models is only possible with a short context, and 13B–14B models are not a comfortable full-GPU workload at 8GB.
If you already own an 8GB card, it is a legitimate way to start and there is no need to upgrade before exploring what local LLMs can do. But for a fresh purchase aimed at local AI, 12GB is the sensible floor, with 10GB a step up for existing owners. Buying new specifically to land on 8GB is rarely the right call in 2026.
Model fit at 8GB VRAM
The table below applies to any GPU with 8GB of VRAM. Cards differ in token generation speed (memory bandwidth) and in how easy they are to set up, but the model ceiling is the same. The VRAM figures are for roughly 4-bit weights; add memory for the KV cache and context on top.
| Model size | Best quantization | VRAM used (weights) | Fits in 8GB? | Notes |
|---|---|---|---|---|
| 1B–4B | FP16, Q8, or Q4 | 0.7–8 GB | Yes | Full precision on the smaller sizes. Fast and comfortable for tools and chat. |
| 7B | Q4_K_M (Q5/Q6 short context) | ~4.1 GB | Yes | Q4 is the normal operating point with room for context. Q8 (~7.7 GB) fits only with a very short window. |
| 8B | Q4_K_M | ~4.9 GB | Yes | Llama 3 8B, Gemma 3 8B. Q4 fits with headroom; higher quant tightens context sharply. |
| 13B | Q4_K_M | ~7.9 GB | No (not practical) | Weights nearly fill 8GB — no room for KV cache. Needs 10GB+ for real use. |
| 14B | Q4_K_M | ~9 GB | CPU offload only | Exceeds 8GB. RAM offload is slow and not interactive. |
| 30B+ | Q4 | ~18–20 GB+ | No | Far exceeds 8GB. Needs a much larger card or cloud. |
Use the Local LLM Compatibility Checker to match specific models against your exact hardware configuration and workflow.
What 8GB VRAM does well
- 1B–4B at FP16: Full precision on small models, very fast. Ideal for autocomplete, tool-call loops, embeddings pipelines, and low-latency assistants.
- 7B at Q4_K_M: The workhorse of the 8GB tier. Uses about 4.1 GB and leaves room for a usable context window. Good for chat, summarisation, and general coding help.
- 8B at Q4_K_M: Llama 3 8B and Gemma 3 8B at Q4 fit with headroom. A small step up in capability from 7B at the same quantization.
Where 8GB runs out: higher quant, 13B–14B, and context
The 8GB budget is spent quickly once you move past 7B/8B at Q4. Pushing a 7B model to Q8 (~7.7 GB) leaves almost nothing for the KV cache, so it only works with a very short context. A 13B model at Q4 (~7.9 GB) nearly fills the card before any context is added, which makes it impractical rather than merely tight. A 14B model at Q4 (~9 GB) does not fit at all and must offload to system RAM.
- Keep 7B/8B at Q4: that is where 8GB has room to breathe and stay fully on-GPU.
- Treat higher quant as short-context only: Q5/Q6/Q8 on 7B trades context room for precision.
- Step up for 13B–14B: genuine 13B–14B use wants 10GB, and 12GB for comfort.
CPU and RAM offload: possible, but slow
Runtimes like Ollama and llama.cpp can offload the layers that do not fit in VRAM to system RAM, where the CPU processes them. This lets an 8GB card technically load a larger model, but at a heavy speed cost — offloaded inference is several times slower than a fully GPU-resident model and is not comfortable for interactive chat. Fast DDR5 softens the penalty but does not remove it. Treat offload as a way to occasionally experiment with a bigger model, not as a daily driver.
Representative 8GB GPUs and their runtime paths
8GB cards exist across all three GPU vendors. They share the 8GB model ceiling but not the runtime experience — the most important practical difference between them for local AI.
| GPU | Platform | Memory bandwidth | Runtime path | Notes |
|---|---|---|---|---|
| RTX 4060 8GB | NVIDIA · Ada Lovelace | ~272 GB/s (GDDR6) | CUDA — mature, broadly supported | Current-generation 8GB NVIDIA card. CUDA works out of the box in Ollama, LM Studio, and llama.cpp. Modest bandwidth, but the least setup friction at 8GB. |
| RTX 3070 / RTX 3060 Ti 8GB | NVIDIA · Ampere | ~448 GB/s (GDDR6) | CUDA — mature, broadly supported | Higher bandwidth than the 4060 at the same 8GB ceiling. Strong used-market local-AI value if already owned. |
| Radeon RX 6600 / RX 6650 XT 8GB | AMD · RDNA 2 | ~224–280 GB/s (GDDR6) | ROCm or Vulkan — workable, more setup | Official ROCm coverage for these consumer RDNA 2 parts is narrow; the llama.cpp Vulkan backend is often the more reliable path. Linux is the better-tested platform. |
| Intel Arc A750 / A580 8GB | Intel · Alchemist | ~512 GB/s (GDDR6) | oneAPI / SYCL, IPEX-LLM, or Vulkan — newer stack | High memory bandwidth for the tier. Local AI runs via Intel oneAPI / SYCL, IPEX-LLM, or Vulkan — capable but a faster-moving, less mature runtime path than CUDA. |
- NVIDIA CUDA (RTX 4060, RTX 3070, RTX 3060 Ti): The most mature local AI path. Ollama, LM Studio, and llama.cpp detect CUDA automatically. See the dedicated RTX 4060 8GB guide.
- AMD ROCm / Vulkan (RX 6600, RX 6650 XT): Works, but expect more setup. Official ROCm support for these consumer RDNA 2 cards is narrow, and the llama.cpp Vulkan backend is frequently the more dependable route. Linux is the better-tested platform.
- Intel oneAPI / SYCL / IPEX-LLM / Vulkan (Arc A750, A580): A newer, faster-moving stack. Intel's IPEX-LLM and the SYCL and Vulkan backends can run 7B–8B models on Arc, but driver and runtime maturity trails the NVIDIA path.
Recommended runtimes for the 8GB tier
- Ollama — The easiest start on NVIDIA. Pull a 7B or 8B model at Q4 and it detects your GPU automatically.
- LM Studio — Desktop GUI for downloading and testing quantizations without a terminal. Good for finding the Q4 balance that fits your 8GB budget.
- llama.cpp: The underlying engine. Its Vulkan backend is especially useful on AMD and Intel cards where CUDA is not available, with fine control over GPU layers and context.
- Open WebUI — A private browser chat interface over Ollama, via Docker.
Recommended models for 8GB VRAM
- Qwen 3 8B at Q4_K_M — daily driver: A strong general model that fits with context headroom at 8GB. Pull with
ollama pull qwen3:8b. - Gemma 3 4B FP16 — fast and precise: Full precision on a small model. Ideal for autocomplete, tools, and lightweight agents where latency matters.
- Phi-4 Mini / Qwen 3 4B — efficient small models: Comfortable at Q8 or FP16 on 8GB with plenty of room for context. Good reasoning per parameter.
Getting started on an 8GB GPU
Start with a 7B or 8B model at Q4 to confirm GPU acceleration, then keep context modest:
# Install Ollama (Linux/macOS)
curl -fsSL https://ollama.com/install.sh | sh
# Pull a 7B/8B model at Q4 — the normal operating point at 8GB
ollama pull qwen3:8b
ollama run qwen3:8b
# Fast small-model option at full precision
ollama pull gemma3:4b-fp16
ollama run gemma3:4b-fp16Upgrade paths from 8GB
- 10GB (tier guide): More headroom for 7B/8B at higher quant, plus tight 13B–14B Q4. A good step up for existing owners (e.g. an RTX 3080 10GB).
- 12GB (tier guide): The recommended floor for a new purchase — 7B/8B at Q8 and 13B–14B at Q4 (both tight but workable). RTX 3060 12GB, RTX 4070, and RTX 5070 land here.
- 16GB (tier guide): The headroom tier — comfortable 7B–8B Q8 and 13B–14B Q4, plus a tight 24B Q4.
- 24GB (tier guide): The consumer sweet spot — 30B at Q4 and 7B at full precision.
Cloud GPU fallback
An 8GB card is well suited to small models and 7B/8B at Q4. When a workflow needs larger models, longer context, or higher concurrency than the 8GB budget allows, renting a larger cloud GPU is the practical fallback. See the tools directory for managed GPU options, and the AI PC Builder if you are planning a new machine.
Related hardware
FAQ
What is the best model size for an 8GB VRAM GPU?
For 8GB, 7B and 8B models at Q4_K_M are the practical operating point. A 7B model at Q4 uses roughly 4.1 GB and an 8B model about 4.9 GB, leaving several gigabytes for the KV cache and context. Smaller 1B–4B models run at full FP16 precision very fast. Higher quantization on a 7B model — Q5, Q6, or Q8 — is sometimes possible but only with a short context window, because the weights leave much less room for the KV cache at 8GB.
Can 8GB VRAM run 13B or 14B models?
Not comfortably as a full-GPU workload. A 13B model at Q4_K_M needs about 7.9 GB — that nearly fills 8GB before any context is added, leaving essentially no room for the KV cache, so it is not practical for real use. A 14B model at Q4 needs about 9 GB (measured) and exceeds 8GB entirely, requiring CPU/RAM offload that runs slowly. At 8GB, stay with 7B/8B models; step up to 10GB or 12GB for genuine 13B–14B use.
Is 8GB VRAM enough to start local AI in 2026?
If you already own an 8GB card, yes — it is a genuine entry point. You can run compact models and 7B/8B models at Q4 for private chat, lightweight coding help, and small document Q&A. But if you are buying new hardware specifically for local AI in 2026, start at 12GB or more. 8GB is a tight budget once context and KV cache are accounted for, and the jump to 12GB meaningfully expands what runs comfortably.
Why does context length reduce what 8GB can run?
The model weights are only part of the VRAM budget. The KV cache — memory that stores attention keys and values for every token in the context window — grows with context length and sits in VRAM alongside the weights. At 8GB, a 7B model at Q4 leaves a few gigabytes for KV cache, which is fine for short and medium context but tightens quickly at long context (16K+). Keeping context modest is the main way to stay fully on-GPU at this tier.
Do the RTX 4060, RX 6600, and Arc A750 run local AI equally well at 8GB?
No — they share the 8GB ceiling but not the runtime experience. The NVIDIA RTX 4060 (and RTX 3070 / 3060 Ti) run on CUDA, the most mature and widely supported local-AI path. AMD cards like the RX 6600 / 6650 XT depend on ROCm or a Vulkan backend, which work but need more setup and have narrower official support for these consumer RDNA 2 parts. Intel Arc A750 / A580 rely on Intel oneAPI / SYCL, IPEX-LLM, or Vulkan — a newer, faster-moving stack. All can run 7B–8B models at Q4; the setup effort differs.
Disclosure
OpenSourcesAI produces hardware assessments independently. Hardware specifications are sourced from manufacturer documentation. Model VRAM estimates are derived from GGUF quantization formulas for approximately 4-bit weights and may vary across runtime versions, context lengths, and model architectures. Verify before making purchasing decisions.
Check your specific GPU against models
Enter your VRAM, RAM, and workflow into the compatibility checker to see which models and quantization levels are recommended for your 8GB setup.
For builders
Selling AI hardware, cloud GPUs, or local AI tools?
Sponsor a contextual placement on this page or submit your product for editorial review on OpenSourcesAI. For sponsorship options, advertise with us. For submissions or corrections, use the submit page.