Gemma 4 12B (QAT)
12B parameter open-weight model. Quantization-aware trained variant of Gemma 4 12B. QAT recovers most of the quality lost to Q4 quantization versus post-training quantization of the base checkpoint, at the cost of a slightly larger Q4_K_M footprint (9.5 GB vs 7.6 GB). QAT is published as a Q4-class artifact only — for Q8 or BF16 inference use the base Gemma 4 12B entry instead.
Google · Gemma 4
Model overview
Gemma 4 12B (QAT) is a quantization-aware trained release of the base Gemma 4 12B model, meaning the model was trained with Q4 quantization already factored in rather than quantized after the fact. That process recovers most of the quality that post-training quantization typically loses, at the cost of a slightly larger Q4_K_M footprint (9.5 GB vs. 7.6 GB for the base model's Q4). It's the better pick if you're committed to running Q4 and want the best possible quality at that precision; if you have the VRAM for Q8 or BF16 instead, use the base Gemma 4 12B entry, since this QAT build is published as a Q4-only artifact.
Editorial review
VRAM figures are empirical estimates. Actual usage varies by runtime, context length, and system configuration. Verify on your specific hardware before production use.
Will Gemma 4 12B (QAT) run on your machine?
Gemma 4 12B (QAT) is 12B parameters and needs 11 GB of VRAM at Q4_K_M — 9.5 GB of weights plus 1.5 GB of runtime overhead for the inference server itself.
VRAM by quantization
| Quantization | Weights | Needs (with overhead) | Quality |
|---|---|---|---|
| Q4_K_M | 9.5 GB | 11 GB | good |
Fit on common hardware at Q4_K_M
| Hardware | Memory the model can use | System RAM | Verdict |
|---|---|---|---|
| CPU Only | None (CPU only) | 16 GB | CPU offload |
| RTX 4060 Laptop | 8 GB | 16 GB | CPU offload |
| RTX 3060 (12GB) | 12 GB | 32 GB | Tight |
| RTX 4060 Ti (16GB) | 16 GB | 32 GB | Comfortable |
| RTX 3090 | 24 GB | 64 GB | Comfortable |
| Apple Silicon (Unified Memory) 36 GB | 27 GB of 36 GB | 36 GB | Comfortable |
Comfortable means VRAM clears the requirement by 2 GB or more. Tight means it covers the requirement with no margin. CPU offload means the model does not fit in VRAM but system RAM is at least 1.6× the weights, so it will run at reduced speed — expect roughly 1–5 tokens per second. Figures are weights plus a fixed runtime overhead and exclude KV-cache growth, which scales with context length.
Apple Silicon shares one pool of memory between the system and the GPU, so a model cannot use all of it. These rows apply the same 75% usable fraction the Compatibility Checker uses, which is why a 36 GB Mac is graded on less than 36 GB.
Need more hardware for Gemma 4 12B (QAT)? Open the PC Builder for the 13B / 14B tier →
Ready to run this model locally?
Find a compatible interface in our Local AI Tools directory →