How the Compatibility Checker works.

The fit formula

The Checker grades a model against your estimated total VRAM need: the model's weights at a given quantization, plus roughly 1.5 GB of runtime overhead for the inference server itself. KV cache, the memory that grows with your context length, is not included, because it depends on how you use the model rather than on the model. Long contexts need headroom beyond any verdict here.

• Comfortable: the model fits with at least 2 GB of VRAM to spare above its estimated requirement. Context can grow without immediate risk of eviction.
• Usable, but tight: the model fits, but with less than 2 GB to spare. Close other GPU apps and avoid very long prompts.
• Runs via CPU RAM offload: GPU VRAM alone is not enough, but your system RAM can hold the overflow. This needs roughly 1.6× the model's weights in system RAM, counting weights alone, not the weights-plus-overhead figure above. The multiplier was calibrated end-to-end on real offloaded runs and already absorbs the runtime overhead, so applying it to the larger figure would count that overhead twice. For dense models expect 1–5 tokens/s. A sparse mixture-of-experts model activates only a fraction of its parameters per token, so offload stays usable; each result says which case applies.
• Does not fit: neither VRAM nor RAM offload can hold the model at any catalogued quantization.

Quantization selection

For each model the Checker picks the best quantization your memory can hold, preferring higher precision when it fits: FP16 or Q8_0 (near full precision) over Q4_K_M (roughly half the size with moderate quality loss). This pick depends on your memory alone; your “Preference” choice does not change which quantization a given model is offered. What Preference changes is the ranking between models: Speed first ranks models that run at the smaller quantization higher, Quality first boosts models running at FP16 or Q8_0 and penalizes offload fits. The quantization named in a verdict is the one that was actually graded.

Fit rules as implemented

The grading lives in lib/checker-engine.js. Its three constants are RUNTIME_OVERHEAD_GB = 1.5, COMFORTABLE_HEADROOM_GB = 2 and RAM_OFFLOAD_MULTIPLIER = 1.6. For a model whose catalog weights at a quantization are W GB, on a GPU with V GB of VRAM and R GB of system RAM:

• Comfortable when V is at least W + 1.5 + 2.
• Tight when V is at least W + 1.5, but less than that plus 2.
• Partial (CPU RAM offload) when the GPU check fails but R is at least W × 1.6. The multiplier applies to bare weights, without the overhead.
• Too large otherwise.

W is the weights-only figure in src/data/checker-models.json (quantizations[q].vramGb). The quantization offered is the first of FP16, Q8_0 and Q4_K_M whose W + 1.5 fits in V (pickBestQuantization in the same file); when none fits, Q4_K_M is offered through offload if R covers it. On Apple Silicon, V is a share of unified memory, rounded to one decimal: APPLE_USABLE_MEMORY_FRACTION_SMALL = 0.6667 (two-thirds) up to APPLE_SMALL_POOL_MAX_GB = 36 GB and APPLE_USABLE_MEMORY_FRACTION_LARGE = 0.75 above (src/lib/appleMemory.js). The estimated speed class on hardware presets is BANDWIDTH_EFFICIENCY = 0.7 × memory bandwidth ÷ W, banded as Very fast (60+ tokens/s), Fast (25–60 tokens/s), Moderate (8–25 tokens/s), Slow (under 8 tokens/s), one band lower for a tight fit, and absent for offload fits, sparse mixture-of-experts models, Apple Silicon, multi-GPU and manual entries (src/lib/throughputBand.js). The code, as it ships:

export function fitRequirementGb(weightsGb) {
  const weights = Number(weightsGb);
  if (!Number.isFinite(weights) || weights <= 0) return weightsGb;
  return weights + RUNTIME_OVERHEAD_GB;
}
export function gradeVramFit(vramGb, requiredVramGb, systemRamGb = 0, offloadBasisGb = requiredVramGb) {
  if (vramGb > 0) {
    if (vramGb >= requiredVramGb + COMFORTABLE_HEADROOM_GB) return 'comfortable';
    if (vramGb >= requiredVramGb)                           return 'tight';
  }
  if (systemRamGb >= offloadBasisGb * RAM_OFFLOAD_MULTIPLIER) return 'partial';
  return 'too_large';
}
export function gradeModelFit(vramGb, weightsGb, systemRamGb = 0) {
  return gradeVramFit(vramGb, fitRequirementGb(weightsGb), systemRamGb, weightsGb);
}

One known gap, from measurement: the overhead is a flat 1.5 GB whatever the context length, and the KV cache grows with context. On our RTX 3080 10GB a model can fit at a 4K context and spill at 16K; the measured results for that card show it.

Apple unified memory

Apple Silicon shares one memory pool between the CPU, the GPU, and everything else you have open, so not all of it is available to a model. The Checker sizes every Apple fit against the share macOS lets a model use by default: about 67% of the reported total on Macs up to 36 GB and 75% above that. An 8 GB Mac is graded on 5.3 GB of effective model memory and a 64 GB Mac on 48 GB.

The two-thirds figure is measured: on this project's own 8 GB MacBook Air M1 (macOS 26.7.1, with the default iogpu.wired_limit_mb of 0) Metal reported a recommendedMaxWorkingSetSize of 5,726,633,984 bytes on 1 November 2026, two-thirds of the pool. The three-quarters figure is what owners of 64 GB and 128 GB Macs consistently report, and the size at which the share changes is not measured; the 36 GB boundary is the one those reports repeat. Where it is wrong for a Mac between those sizes, it understates that Mac rather than promising a fit macOS refuses. You can read the limit your own Mac applies in Terminal; it prints the usable figure in GiB: swift -e 'import Metal; print(Double(MTLCreateSystemDefaultDevice()!.recommendedMaxWorkingSetSize) / 1073741824)'

A GPU shared with other workloads

The VRAM figure you select means the whole card is free for a model. If the GPU also runs services that must keep running, enter the memory that stays free while they run in the optional GPU memory for models field, and every fit is graded against that budget instead of the card. Left blank, nothing changes. An entry the Checker cannot use pauses the results rather than falling back to the full card. A budget of 0 GB grades every result as CPU-only, the same as selecting 0 GB of VRAM. A budget can never be larger than the VRAM you selected.

The budget changes how results are graded and nothing else: it does not limit how much GPU memory Ollama, LM Studio or any other runtime will use, and a run command shown beside an estimated fit starts that runtime with its default settings. Estimated full-GPU fits are listed separately from CPU-offload options, and those options (CPU-only ones at 0 GB) are shown for comparison without run commands, because a default command would not keep a model on the CPU or inside the budget. A copied result carries the budget as a range, never the figure you typed. A budget is a memory estimate only: it does not test whether a model can run alongside your other workloads, which also use the GPU's compute and can slow generation. The context-fit sweeps on our RTX 3080 page stopped the service that normally shares that card, so they show what fits on a free card, not what fits beside that service. The Model Builder Wizard cannot carry a budget yet, so budget results do not hand off to it.

Runtime suggestions

The runtimes a result suggests follow your effective memory tier. They are starting points, not endorsements of one tool over another:

• Workstation (48 GB+ VRAM): Ollama, LM Studio, vLLM, SGLang, GPUStack
• Strong GPU (24–47 GB VRAM): Ollama, LM Studio, GPUStack, vLLM
• Mid-to-high GPU (16–23 GB VRAM): Ollama, LM Studio
• Mainstream GPU (8–15 GB VRAM): Ollama, LM Studio
• Entry GPU (4–7 GB VRAM): Ollama, LM Studio
• CPU-only (32 GB+ system RAM, no discrete GPU): Ollama

What a copied result contains

The Checker's “Copy result summary” button produces a plain-text snapshot built for pasting into a GitHub issue, a forum thread, or a chat. Three rules govern it:

• Hardware appears as ranges, not exact figures. Your VRAM and RAM are reduced to bands like “12–15 GB”, and your GPU to a vendor class (NVIDIA, AMD, Apple, Intel). The free text you typed into the form is not echoed, so a snapshot is safe to paste in public.
• Assumptions travel with the claim. Each snapshot carries the same threshold sentences this page documents, so a verdict can't circulate without the rules that produced it.
• Nothing is transmitted. Copying happens entirely in your browser via the clipboard. No result, hardware detail, or identifier is sent to us or anyone else when you copy, and results are not encoded into shareable URLs.

Each snapshot ends with Contract v6, the version of the result format. The version changes only when the shape or meaning of a result changes (v2 added honest wording for sparse mixture-of-experts offload; v3 corrected the offload RAM rule to state its weights-only basis), so an old citation can always be read against the rules that were in force when it was made.

Limits

These are sizing rules, verified against vendor documentation and hands-on runs, not a benchmark of your machine. Real generation speed depends on memory bandwidth, thermals, drivers, and context length. To measure what your running setup does, use the benchmark tool; to understand the memory arithmetic in depth, read What is VRAM? and What is quantization?.

When you pick a hardware preset, the Checker also shows an estimated speed class per model, a coarse tokens-per-second band derived from the GPU’s memory bandwidth divided by the model’s size, derated to match our own measured bench runs. It is an estimate calibrated against real measurements, not a measurement of your machine, and it does not affect which models are recommended or how they are ranked. Where no sound estimate exists (manually entered hardware, Apple Silicon, multi-GPU rigs, sparse mixture-of-experts models, CPU-offload fits) the Checker shows no class rather than a guess.

Found a verdict that disagrees with your real-world experience? Tell us; hands-on corrections outrank our estimates.

Measurement protocol

The measured sections on our hardware pages come from @opensourcesai/bench (protocol osai-bench/1.3; releases 0.10.0 (scoring osai-bench-derive/1.3) and 0.12.0 (scoring osai-bench-derive/1.4)), run against Ollama on machines we own. Every release listed ran the same workloads, so one table describes them all. Each run sends four workloads to the model with temperature 0, seed 42 and streaming on. This table is read from the published result files:

• Tokens per second. Generation is the median, over W4's measured passes, of Ollama's own eval_count divided by eval_duration. Prompt processing is the median over W3 of prompt_eval_count divided by prompt_eval_duration. Time to first token is the median over W2 of the time from request to the first streamed token. Cold load is W1's load_duration. CV is the coefficient of variation across the measured passes. Our test suite recomputes each of these from the raw passes in every result file.
• VRAM. After loading, the bench reads Ollama's /api/ps: size, the loaded model's resident footprint including its KV cache and compute buffers, and size_vram, the part of it in VRAM, both in bytes. It stores them as residentBytes and vramResidentBytes, with hostResidentBytes = size − size_vram and vramResidentFraction = size_vram ÷ size, and computes nothing else. Our pages show size_vram as “VRAM used” and the spill as hostResidentBytes ÷ residentBytes. These are Ollama's own figures, not a separate GPU reading: deepseek-r1:14b and qwen2.5:14b, whose files differ in size, report byte-identical size and size_vram on our RTX 3080, so their shares are identical too. Because this is the model's own footprint, other programs using the card do not inflate it.
• Preconditions. The bench refuses to start when the GPU is already busy (pre-existing utilization above 10%) or another model is resident, and records the free VRAM it sees when it checks the GPU and when the model loads.
• Validity. A pass whose prompt token count falls outside its workload's range, or whose generated token count differs from the target, is invalid and retried. A run whose generation passes all fail reports no generation figure, and our tables say so instead of showing one.
• Ceiling. The bench also divides the card's memory bandwidth from its built-in table (NVIDIA GeForce RTX 3080: 760 GB/s; NVIDIA GeForce RTX 4070 Ti: 504 GB/s) by the weight size, as a theoretical upper bound for generation.
• How we run it. One model at a time. In a catalog sweep the previous model is stopped first, with an 8-second pause before the next run's precondition check. A model measured more than once is listed once per run; runs are not averaged. Tables round to two decimals (milliseconds to one) and give sizes in decimal GB; each row links to its file, which holds the unrounded value.
• Scope. Each published GPU run used one discrete NVIDIA GPU and Ollama, at the context lengths in the table. The CPU-only runs used the same bench on the same Ollama, started with the GPU hidden from it (CUDA_VISIBLE_DEVICES=-1 and GGML_VK_VISIBLE_DEVICES=-1), so every byte of the model sat in system RAM; each file records 0 bytes in VRAM. That is how a machine without a usable GPU runs a model. The bench still sees the card and computes its bandwidth ceiling from it, which does not apply to a CPU run, so no page shows that figure for them. A Modelfile setting of num_gpu 0 is different: it keeps the layers on the CPU but leaves the GPU visible, and our sweep used it only as a diagnostics fixture, which no page shows. One further run, on our M1 MacBook Air with bench 0.11.0 (scoring osai-bench-derive/1.4), is not published: the bench cannot see Apple GPUs, so that rig is used for detection checks, not benchmarks.

Context-fit sweep

The bench measures speed at one context length. How long a context still fits on a card is a separate question, answered by scripts/lab/context-fit-sweep.mjs (protocol osai-ctxfit/1), which lives in this site's repository and runs on the rig against its own Ollama. For each context length (4,096, 8,192, 16,384, 32,768 tokens in the published sweeps, skipping any longer than the model was trained for) it:

1. reads the GPU's used and free memory with nvidia-smi, with no model loaded;
2. loads the model fresh with that num_ctx and runs 4 short generation passes of 64 tokens each, the first one a warm-up;
3. reads Ollama's /api/ps: size (the footprint), size_vram (the part on the card) and context_length, which must equal the length requested;
4. unloads the model and waits before the next length.

Each length is then labelled from those readings alone. In VRAM, full speed: Ollama put the whole footprint on the card, the footprint was no larger than the free VRAM measured before the load, and generation stayed within 25% of the sweep's speed at its shortest length. Slowed: Ollama still reported all of it on the card, but the footprint was larger than the free VRAM or generation fell further than that. On Windows the NVIDIA driver can move GPU memory into system RAM when the card is nearly full, which Ollama does not see; we name that as the likely cause and have not verified it. Split: Ollama itself put part of the model in system RAM. Generation in a sweep uses a short prompt, so the cache is allocated but nearly empty; it shows what a poor fit costs, not speed with a full context window. Our test suite recomputes every step's generation figure and label from the raw passes in the published file.

Published result files

Each file behind a measured page is published unedited under /lab/results/, with a note on what the files contain. Each was checked for paths, addresses, host names and credentials before publication; none needed redacting. The files are licensed Creative Commons Attribution 4.0 International (CC BY 4.0): reuse them, including commercially, with credit to CJ Zwart, OpenSourcesAI and a link to this page. The license covers the result files only.

Changelog

Changes to the fit rules and the measurement protocol. The date is the commit date, or the run date for a result file; the reference is the commit or file that records the change.

Who maintains this

The fit engine, this page and the measurements are maintained by CJ Zwart, Editor and founder, OpenSourcesAI, who also builds @opensourcesai/bench. Corrections go through the contact page.

Last updated: 2026-11-01