RTX 3080 10GB for Local LLMs: Existing-Owner Guide

Measured on this card

Measured by CJ Zwart on the 2570server machine (RTX 3080 10 GB). Runs dated 2026-08-01 to 2026-08-07 (UTC).

Every row below is one run of @opensourcesai/bench on this machine, read from the result file linked in the last column. Nothing in the table is estimated.

• GPU: NVIDIA GeForce RTX 3080 with 10,737,418,240 bytes (10.74 GB) of VRAM as reported by nvidia-smi; driver 595.71.05, 595.84.
• CPU and RAM: 12th Gen Intel Core i9-12900K, 24 CPUs; 32 GB nominal / ~30 GiB usable RAM.
• OS: Ubuntu 26.04 LTS.
• Runtime and tool: Ollama 0.32.5; @opensourcesai/bench 0.10.0, protocol osai-bench/1.3, scoring osai-bench-derive/1.3.
• Context: 4,096 tokens for every run.

What the runs show

• Weights that fit are not a model that fits. Five 14B-class models at Q4_K_M (14.0B to 14.8B parameters, 8.57 GB to 9.28 GB of weights, all under the card's 10.74 GB) passed the bench's weights-exceed-VRAM check, and every one still spilled: at a 4,096-token context between 8.79% and 12.68% of each one's resident bytes sat in system RAM. Where generation was measured it ran 22.93 tok/s (phi4:latest), 26.97 tok/s (qwen3:14b), 30.51 tok/s (deepseek-r1:14b). Qwen3 14B, run twice more on 2026-08-03 under local tags, spilled the same 11.45% and generated 26.72 tok/s and 26.75 tok/s. Qwen3 8B at Q8_0 (8.85 GB of weights) did fit at 4,096 tokens, all of it in VRAM, at 73.76 tok/s. A separate check of the same model on 2026-08-09 found Ollama reporting “100% GPU” at 4,096 tokens but “23%/77% CPU/GPU” with a footprint of 11 GB at 16,384 tokens. On a 10 GB card the line is set by the footprint at the context you use, not by the size of the weight file. Runs: 2026-08-07T06-23Z-rtx3080-phi4.json, 2026-08-07T07-19Z-rtx3080-phi3-medium-128k-q4km.json, 2026-08-07T07-46Z-rtx3080-qwen3-14b.json, 2026-08-07T07-48Z-rtx3080-deepseek-r1-14b.json, 2026-08-07T07-55Z-rtx3080-qwen2-5-14b.json, 2026-08-03T06-21Z-rtx3080-qwen3-14b-1k.json, 2026-08-03T06-24Z-rtx3080-qwen3-14b-2k.json, 2026-08-03T06-09Z-rtx3080-qwen3-8b-q8_0.json. The 16,384-token check is an ollama ps reading recorded in a dated lab note, not a bench run.
• Past 10 GB, the model architecture decides the speed. Seven models whose weights are larger than the card ran with between 53.60% and 65.23% of their resident bytes in system RAM. The four that Ollama labels mixture-of-experts (qwen3moe, cohere2moe, qwen35moe) generated 30.41 to 34.25 tok/s; the three dense ones generated 2.34 to 3.08 tok/s. A mixture-of-experts model reads only a few experts per token, so the part left in system RAM costs it little; a dense model reads every weight for every token. gemma4:26b (29.82 tok/s with 58.51% in system RAM) is in neither group: Ollama does not label it mixture-of-experts, and the lab note leaves it as an open question. Runs: 2026-08-03T04-40Z-rtx3080-qwen3-30b.json, 2026-08-03T06-15Z-rtx3080-qwen3-coder-30b.json, 2026-08-03T04-22Z-rtx3080-north-mini-code-1.0.json, 2026-08-03T04-45Z-rtx3080-qwen3.6-35b-a3b.json, 2026-08-03T05-09Z-rtx3080-qwen3.6-27b.json, 2026-08-03T08-30Z-rtx3080-qwen3-32b.json, 2026-08-03T01-32Z-rtx3080-gemma4-31b.json, 2026-08-03T06-30Z-rtx3080-gemma4-26b.json.
• Q8_0 costs speed and context room, not fit, at short context. Measured in the same sweep, Qwen3 8B at Q4_K_M generated 109.56 tok/s with 5.58 GB in VRAM; at Q8_0 it generated 73.76 tok/s with 8.89 GB. Both stayed fully in VRAM at 4,096 tokens. The higher precision leaves less room for a longer context, which is where the first finding's spill begins. Runs: 2026-08-03T06-16Z-rtx3080-qwen3-8b.json, 2026-08-03T06-09Z-rtx3080-qwen3-8b-q8_0.json.

Limits of this data

• One machine, one runtime. Everything here ran on Ollama 0.32.5 with no non-default server settings declared. The bench reads that declaration from its own environment, not the server's, and marks it non-authoritative in every file.
• One context length. Every run used 4,096 tokens. The bench did not measure usable context beyond that; the Qwen3 8B Q8_0 check in the first finding is the only longer-context reading for this card.
• Single runs. Each figure is the median of the measured passes inside one run. A model run more than once is listed once per run, and the runs are not averaged.
• Sweep conditions. Most runs dated 2026-08-03 come from one back-to-back sweep of the whole model catalog. The lab note for that sweep found it read 3.6% low on generation for llama3.1:8b against an isolated run, so treat those rows as slightly conservative.
• A shared card. This rig's GPU also serves another always-on service, so not all of its 10.74 GB is free. Free VRAM when each run began is recorded in its file; for the runs above it was between 1.19 GB and 10.33 GB. The low figure is a single run, phi4:latest on 2026-08-07; every other run began with at least 10.09 GB free. The file does not record what held the memory.
• Gaps. Five runs (phi3:3.8b-mini-128k-instruct-fp16, qwen2.5:7b-instruct-q8_0, phi3:14b-medium-128k-instruct-q4_K_M, qwen2.5:14b, laguna-xs-2.1:latest) have no generation figure: the bench rejected every one of their generation passes, so the table says “Not measured”. Their memory placement is shown. Two further runs forced model layers onto the CPU on purpose to test the bench's diagnostics and are left out; the benchmark page covers them.
• Not measured at all. Other runtimes (llama.cpp directly, LM Studio, vLLM), quantizations other than those listed, long contexts, output quality, and power or heat.

How each figure is measured is on the methodology page. The raw files are published as they came off the machine; read the note on what they contain. They are licensed CC BY 4.0: reuse them with credit to CJ Zwart, OpenSourcesAI and a link to this page.

RTX 3080 10GB vs 3080 12GB vs 3080 Ti

"RTX 3080" covers three different cards for local AI purposes. The distinction matters because VRAM capacity — not core count — sets the model ceiling.

If your card reports 12GB, follow the 12GB VRAM tier guide instead — the extra 2GB changes what runs comfortably. This page is specifically about the 10GB variant.

What this card can run

The model fit table below applies to the RTX 3080 10GB. Its strong bandwidth means it generates tokens quickly for the models that fit; the 10GB capacity is what sets the ceiling.

For a full explanation of how the 10GB ceiling interacts with context length across all three representative 10GB cards, see the 10GB VRAM tier guide.

Best model sizes for this card

• 7B at Q8: The recommended daily tier. Q8 quality is very close to FP16 and fits with a workable margin. The 3080 10GB's bandwidth makes this fast and responsive.
• 8B at Q5_K_M or Q6_K: Slightly lower precision than Q8 but leaves more room for context — the better default on an 8B model at 10GB.
• 14B at Q4_K_M (spills to system RAM): More depth at reduced precision, but only about 1GB of VRAM is free once the ~9GB of weights load. Measured on this card, part of the model offloads to host even at a 4K context, and generation runs at a fraction of full-GPU speed (the runs are under Measured on this card).
• 4B at FP16: Full precision, very fast. Great for autocomplete and low-latency agent loops.

Context and KV-cache limits at 10GB

The 3080 10GB's weakness is not speed — it is capacity. After a 7B–8B model at Q8 loads, only 1–2GB of VRAM remains for the KV cache, which grows with context length. That is fine for short and medium conversations, but long-context workloads (16K+ tokens) can exhaust the free space and force the runtime to cap context or offload to system RAM, which slows generation sharply.

• Keep context modest on the larger models: 4K–8K suits a 13B at Q4; a 14B at Q4 spilled to system RAM even at 4K in our runs.
• Choose a smaller model for long documents: a 7B at Q8 with room for context often beats a cramped 14B.
• Lower the quant a notch for more headroom: Q5/Q6 on an 8B model frees VRAM for a longer window.

Recommended runtimes

• Ollama — The easiest start. Detects CUDA automatically and manages model downloads. Run ollama pull qwen3:8b to begin.
• LM Studio — Desktop GUI for browsing and testing quantizations without a terminal. Useful for finding the Q5/Q6/Q8 balance that fits 10GB.
• Open WebUI — Private browser chat over Ollama, via Docker. A clean local workspace with no external dependencies.
• llama.cpp: The underlying engine, for fine control over GPU layers, context size, and quantization selection.

Getting started on the RTX 3080 10GB

# Install Ollama (Linux/macOS)
curl -fsSL https://ollama.com/install.sh | sh

# Confirm CUDA is detected
ollama serve &
ollama list

# Start with an 8B model at Q4 — fast download, confirms CUDA is working
ollama pull qwen3:8b
ollama run qwen3:8b

# Balance quality vs context headroom on 10GB (Q5_K_M frees KV-cache room)
ollama pull qwen3:8b-q5_K_M
ollama run qwen3:8b-q5_K_M

# Depth option: 14B at Q4 — keep the context window short at this tier
ollama pull qwen2.5:14b
ollama run qwen2.5:14b

What this card cannot do well

• Dense 30B+ models at interactive speed: 30B at Q4 needs ~18–20GB. CPU offload is possible but slow and not comfortable for chat. Mixture-of-experts models such as Qwen3 30B are the exception: measured on this card they stayed usable with more than half of the model in system RAM.
• 7B or 8B at FP16: FP16 on a 7B needs ~14GB. Q8 is the maximum practical quality tier for 7B–8B on this card.
• Long context with 13B–14B models: the 10GB budget leaves little KV-cache room once a larger model is loaded. This is where 12GB and 16GB cards pull ahead.

Upgrade guidance

• RTX 3060 12GB / RTX 4070 12GB: The most direct step up in capacity. 12GB adds real context headroom on 13B–14B models — the single biggest practical gain over the 3080 10GB. See the 12GB tier guide.
• 16GB cards (RTX 4070 Ti Super, RTX 4080, RTX 5080): The quality-upgrade tier — 7B at FP16 and 13B–14B at Q8. See the 16GB tier guide.
• RTX 3090 24GB (used market): The value jump to 24GB — 30B at Q4, 14B at Q8, and 7B at full precision, with generous context headroom. See the RTX 3090 guide.

Related hardware

FAQ

Disclosure

OpenSourcesAI produces hardware assessments independently. Hardware specifications are sourced from manufacturer documentation. Model VRAM estimates are derived from GGUF quantization formulas and may vary across runtime versions, context lengths, and model architectures. The figures under Measured on this card are measurements, not estimates, and each links to its result file. Verify before making purchasing decisions.

Last verified: October 2026