compactGemma Terms of Use4B paramsOpen weights

Gemma 3 4B

4B parameter open-weight model. 128K context at 4B scale is a standout feature. Q4 fits in 4 GB VRAM. Useful for RAG and summarisation on constrained hardware.

Google · Gemma 3

Model overview

Gemma 3 4B is Google's compact open-weight model, notable for pairing a small 4B parameter count with an unusually large 128K token context window. That combination makes it useful for tasks like summarizing long documents or maintaining longer conversation history on hardware that couldn't otherwise support a large-context model. It's released under Google's Gemma Terms of Use rather than a fully permissive license, so it's worth reading the terms before commercial use. On constrained hardware — a 4-6 GB consumer GPU at Q4 quantization — it's a reasonable option for RAG retrieval and summarization workflows where long context matters more than raw reasoning power.

Check your GPU fitWhat is VRAM?What is quantization?Local LLM quickstart

Editorial review

Reviewed byOpenSourcesAI EditorialLast updatedJune 2026SourcesHuggingFace model card (google/gemma-3-4b-it), official docs, OpenSourcesAI editorial review.

VRAM figures are empirical estimates. Actual usage varies by runtime, context length, and system configuration. Verify on your specific hardware before production use.

Will Gemma 3 4B run on your machine?

Gemma 3 4B is 4B parameters and needs 4 GB of VRAM at Q4_K_M — 2.5 GB of weights plus 1.5 GB of runtime overhead for the inference server itself.

VRAM by quantization

QuantizationWeightsNeeds (with overhead)Quality
Q4_K_M2.5 GB4 GBgood
Q8_04.3 GB5.8 GBhigh
FP168 GB9.5 GBreference

Fit on common hardware at Q4_K_M

HardwareMemory the model can useSystem RAMVerdict
CPU OnlyNone (CPU only)16 GBCPU offload
RTX 4060 Laptop8 GB16 GBComfortable
RTX 3060 (12GB)12 GB32 GBComfortable
RTX 4060 Ti (16GB)16 GB32 GBComfortable

Comfortable means VRAM clears the requirement by 2 GB or more. Tight means it covers the requirement with no margin. CPU offload means the model does not fit in VRAM but system RAM is at least 1.6× the weights, so it will run at reduced speed — expect roughly 1–5 tokens per second. Figures are weights plus a fixed runtime overhead and exclude KV-cache growth, which scales with context length.

Check Gemma 3 4B against your exact hardwareHow these verdicts are calculated

Need more hardware for Gemma 3 4B? Open the PC Builder for the 7B / 8B tier →

Related models

Qwen3 4B4B parameter model — shared focus: chat, coding, summarisation.DeepSeek-R1 8B (0528 Qwen3)8B parameter model — shared focus: reasoning, coding, chat.Llama 3 8B8B parameter model — shared focus: chat, coding, rag.CodeGemma 7BCoding assistant experiments and developer workflow prototypes.

Ready to run this model locally?

Find a compatible interface in our Local AI Tools directory →