Granite 4.2 8B
8.8B parameter open-weight model. Dense 8.8B reasoning model from IBM's Granite 4.2 generation (GraniteForCausalLM, 40 layers, grouped-query attention with 8 KV heads, config.json read 2026-10-16), with a built-in think block that can be switched to non-thinking or low-effort per request, tool calling, and a 128K native context that IBM states extends to 512K. Apache 2.0. The granite4.2:8b tag was verified against the Ollama registry manifest (checked 2026-10-16). Measured on this project's own hardware: 5.6 GB resident at 100% GPU on an RTX 3080 (10 GB) with nothing else on the card, 100.3 tok/s generation (2026-09-05, docs/lab/radar-2026-09-05-model-validation.md). Re-run 2026-10-16 while another service held 3.9 GB of the same card: Ollama split it 18% CPU / 82% GPU and generation fell to 23.7 tok/s, so plan around 6 GB of free VRAM, not a 10 GB card (docs/lab/radar-2026-10-09-model-validation.md). On this project's repo-help test it reached the same findings as Qwen3 8B in about three times the wall clock, because it reasons before it answers.
IBM · Granite 4.2
Model overview
Granite 4.2 8B is IBM's mid-size reasoning model: a dense 8.8B transformer that thinks step by step before it answers, calls tools, and was tested across a dozen languages, released under Apache 2.0 with no access gate. At Q4_K_M it needs about 6 GB of free VRAM, so it fits an 8 GB card with room to spare and a 6 GB card only when nothing else is using it. Expect slower, longer answers than a non-thinking 8B model on simple tasks; the extra reasoning is for math, multi-step logic and tool-calling work.
Editorial review
VRAM figures are empirical estimates. Actual usage varies by runtime, context length, and system configuration. Verify on your specific hardware before production use.
Will Granite 4.2 8B run on your machine?
Granite 4.2 8B is 8.8B parameters and needs 7.1 GB of VRAM at Q4_K_M: 5.6 GB of weights plus 1.5 GB of runtime overhead for the inference server itself.
VRAM by quantization
| Quantization | Weights | Needs (with overhead) | Quality |
|---|---|---|---|
| Q4_K_M | 5.6 GB | 7.1 GB | good |
Fit on common hardware at Q4_K_M
| Hardware | Memory the model can use | System RAM | Verdict |
|---|---|---|---|
| CPU Only | None (CPU only) | 16 GB | CPU offload |
| RTX 4060 Laptop | 8 GB | 16 GB | Tight |
| RTX 3060 (12GB) | 12 GB | 32 GB | Comfortable |
| RTX 4060 Ti (16GB) | 16 GB | 32 GB | Comfortable |
| RTX 3090 | 24 GB | 64 GB | Comfortable |
Comfortable means VRAM clears the requirement by 2 GB or more. Tight means it covers the requirement with no margin. CPU offload means the model does not fit in VRAM but system RAM is at least 1.6× the weights, so it will run at reduced speed: expect roughly 1–5 tokens per second. Figures are weights plus a fixed runtime overhead and exclude KV-cache growth, which scales with context length.
Want a broader hardware plan around Granite 4.2 8B? Open the PC Builder for the whole 7B / 8B class → The Builder plans across that class; it does not replace the exact-model fit verdict above.
Ready to run this model locally?
Find a compatible interface in our Local AI Tools directory →