Chat
Gemma 4 26B (A4B)
Sparse mixture-of-experts Gemma 4 - 26.5B total parameters, 128 experts, roughly 4B active per token - pairing large-model quality with per-token compute close to a 4B model.
Google · Gemma
Editorial review
Model checkpoints, context windows, provider support, local runtime compatibility, and license terms can change quickly. Verify the exact model card before production or commercial use.
Best for
High-quality local chat and reasoning on 24 GB cards, and usable split-mode performance below that thanks to sparse activation.
Who should use it
- High-quality local chat and reasoning on 24 GB cards, and usable split-mode performance below that thanks to sparse activation.
- Teams with access to hosted inference or server-class deployment paths.
Common workflows
- Large open-weight chat, reasoning, and long-context workflows
Deployment and hardware notes
Ollama Q4_K_M is ~18 GB (26B A4B) / ~20 GB (31B): full-GPU on 24 GB cards, split placement with reduced throughput on smaller GPUs.
License and usage notes
Apache 2.0. Open weights. Verify the exact model card and license terms for the checkpoint or hosted provider you use.
Strengths
- Open weights model option for Gemma workflows.
- High-quality local chat and reasoning on 24 GB cards, and usable split-mode performance below that thanks to sparse activation.
Limitations
- Q4 weights need a 24 GB card for full-GPU placement; below that Ollama splits across VRAM and system RAM. As a sparse MoE its split-mode penalty is much smaller than a dense model of the same size. Verified against the HF config and Ollama library 2026-08-01.
- Ollama Q4_K_M is ~18 GB (26B A4B) / ~20 GB (31B): full-GPU on 24 GB cards, split placement with reduced throughput on smaller GPUs.
- Context window and limits: 256K tokens (max_position_embeddings 262144).
- Verify the exact model card, provider docs, license, and serving support before production use.
Will Gemma 4 26B (A4B) run on your machine?
Gemma 4 26B (A4B) is 26B parameters and needs 19.5 GB of VRAM at Q4_K_M — 18 GB of weights plus 1.5 GB of runtime overhead for the inference server itself.
VRAM by quantization
| Quantization | Weights | Needs (with overhead) | Quality |
|---|---|---|---|
| Q4_K_M | 18 GB | 19.5 GB | good |
Fit on common hardware at Q4_K_M
| Hardware | Memory the model can use | System RAM | Verdict |
|---|---|---|---|
| CPU Only | None (CPU only) | 16 GB | Too large |
| RTX 4060 Laptop | 8 GB | 16 GB | Too large |
| RTX 3060 (12GB) | 12 GB | 32 GB | CPU offload |
| RTX 4060 Ti (16GB) | 16 GB | 32 GB | CPU offload |
| RTX 3090 | 24 GB | 64 GB | Comfortable |
| Apple Silicon (Unified Memory) 36 GB | 27 GB of 36 GB | 36 GB | Comfortable |
| RTX 5090 | 32 GB | 64 GB | Comfortable |
Comfortable means VRAM clears the requirement by 2 GB or more. Tight means it covers the requirement with no margin. CPU offload means the model does not fit in VRAM but system RAM is at least 1.6× the weights, so it will run at reduced speed — expect roughly 1–5 tokens per second. Figures are weights plus a fixed runtime overhead and exclude KV-cache growth, which scales with context length.
Apple Silicon shares one pool of memory between the system and the GPU, so a model cannot use all of it. These rows apply the same 75% usable fraction the Compatibility Checker uses, which is why a 36 GB Mac is graded on less than 36 GB.
VRAM fit by quantization level
Enter your GPU VRAM below to see which quantization of Gemma 4 26B (A4B) fits and get the Ollama run command.
Sources to verify
Related resources
Continue with model source notes, local tools, and implementation guides related to this model.
Model ecosystem connections
Use these next-step links to move from this profile into related tools, comparisons, guides, stacks, and curated shortlists.
Ready to run this model locally?
Find a compatible interface in our Local AI Tools directory →