Measure what your machine actually does with local AI.

Install & run

One command. It walks you through everything interactively — detecting Ollama, listing your installed models, and asking you to pick one:

npx @opensourcesai/bench

You need

What it measures

Real output from our own machines

The first five runs of the published tool, hours after release — fetched from npm with the exact command above, on the two machines we develop on. Not mockups, and not cherry-picked: the slow run and the refusals are the point. Our own records have since grown to a wall-to-wall sweep of the 22 models installed on the Linux machine — including runs that ended with no valid figure at all, kept on file as exactly that.

Generation throughput          143.46 tok/s  (CV 0.06%)
Prefill throughput             9353.02 tok/s  (CV 0.18%)
Time to first token            199.11 ms  (CV 1.13%)
Cold load time                 2.06 s
Pass failure rate              0.00% (0/16 scheduled measured passes)

Measured 2026-08-01 with bench 0.10.0 on Ollama 0.32.5, protocol osai-bench/1.3, scoring osai-bench-derive/1.3 — every result file records all three, so you always know which rules produced a number. These numbers describe each machine on its own terms: the two setups run different models under different runtime settings, and they deliberately cannot be lined up against each other.

We broke the same model twice, and it caught both

The diagnostics card above is a claim, so here is its test. During a full-catalog sweep on our Linux rig (RTX 3080 10 GB, bench 0.10.0, 2026-08-03) we ran the identical llama3.1:8b Q4_K_M three ways: untouched, deliberately pinned to 8 GPU layers, and deliberately forced onto the CPU. Same weights, same session, same machine — the only variable is the misconfiguration.

Each diagnostic fired on exactly its own failure mode and stayed silent on the other two runs — no false alarms, no misses. And this is the quiet argument for running the benchmark at all: all three configurations work. Ollama answers prompts in every one of them, and nothing on screen looks broken. The 12.9× is invisible until something measures it. All three numbers come from the same machine in the same session; they say nothing about how this card compares to yours, and they are not meant to.

What you won’t see here — on purpose

No leaderboards, no percentiles, no “your PC ranks better than X%”. Honest machine-to-machine comparison needs many independent submissions measured under identical rules — the protocol itself demands that before any comparison may be shown, and that corpus does not exist yet. Until it does, your results are what they honestly are: a precise diagnostic of your machine, for you.

Scope limits, stated plainly: protocol v1 covers one discrete GPU or CPU-only execution on Windows and Linux. A second GPU is caught by the preconditions and refused. Apple Silicon is the sharper edge, so here is exactly what happens: the benchmark will run on a Mac and complete — we’ve done it on our own M1 — but v1’s detection cannot see an Apple GPU, so the run is silently mislabelled CPU-only, every placement diagnostic returns not-applicable, no bandwidth ceiling gets computed, and nothing in the output warns you. Treat any number produced on a Mac as outside the protocol: measuring unified memory honestly needs protocol work that hasn’t been done yet, and until it is, a clean-looking Mac result is the one output of this tool you shouldn’t take at face value.

Questions you’ll actually have

Why did the benchmark refuse to run?

That’s the run-quality guard doing its job. It refuses when anything could contaminate the measurement: the GPU already above 10% utilization, non-Ollama processes holding more than a token amount of GPU memory, a different model still resident, or more than one GPU (v1 measures exactly one). The usual culprits are a leftover model — ollama stop <name> clears it, and note that a long OLLAMA_KEEP_ALIVE keeps models resident for hours after you last touched them — and jumping back in too fast: give the GPU a few seconds to settle after unloading. If you must run anyway, --quality-override will — and permanently marks the result JSON as compromised, so the shortcut can’t be laundered out later.

How long does a run take?

Under a minute is typical when the model fits in VRAM. Misconfiguration stretches it far more than model size does — our worst real case, a 31B model split across GPU and CPU on a 10 GB card, took about half an hour. The tool prints an estimate up front and revises it once, live, when the first real measurements show what your hardware actually delivers.

Where do my results go?

A single JSON file — osai-bench-result-<timestamp>.json — in the directory you ran it from, and nowhere else. Nothing is uploaded. Useful flags: --model selects a model non-interactively, --output picks the path (it refuses to overwrite an existing file), and --memory-bandwidth supplies the figure by hand if your GPU is missing from the built-in bandwidth table.

Why is time to first token “unavailable” for my thinking model?

Reasoning models can spend the entire measured budget thinking, and Ollama discards that reasoning server-side — there is genuinely no first token to time. The tool reports the metric as unavailable instead of inventing one; since 0.11.0 it also records how many passes withheld output and reports a separate time-to-first-visible-token figure from the long-form workload, which answers the question you were probably asking.

Does it run on my Mac?

It will run and complete — and that is exactly the problem. See the scope section above: v1 cannot see Apple GPUs, so a Mac result carries no diagnostics, no bandwidth ceiling, and no warning that anything is missing. Treat Mac numbers as outside the protocol.

Ran it? Bring the result somewhere useful.

A result file is worth more with someone to read it with — especially a strange one. Our Discord has a #bench-results channel for exactly that: post what your machine measured, attach the JSON (it carries no file paths and nothing personal, by design), and ask what a diagnostic is actually telling you. Nothing posted there feeds this site’s data — it’s somewhere to think out loud, not a collection point.

There’s a monthly Cleanest Run challenge running too, and it turns on the lowest run-to-run variance — the most carefully measured run, not the fastest hardware. A modest GPU on a quiet machine can win it outright against a much faster one with a browser left open.

Next steps