Phi-3 Medium 128K Instruct
14B parameter open-weight model. 14B with 128K context. Strong reasoning for size; context-heavy RAG tasks benefit from the long window. Q4_K_M weights measured at 8.57 GB, so 12 GB is the practical minimum — on an RTX 3080 (10 GB) 9% of resident bytes spilled to host (measured 2026-08-07, Ollama 0.32.5). Note the tag: the friendly `phi3:medium` serves Q4_0 (7.9 GB), not the Q4_K_M these figures describe, so this entry points at the explicit Q4_K_M tag instead. Throughput is unmeasured here because this model stops generating early against the benchmark's fixed prompt (163 tokens when 512 were requested) — the same behaviour Phi-3 Mini shows at 107 tokens, so it looks architectural rather than model-specific.
Microsoft · Phi
Model overview
Phi-3 Medium 128K Instruct is Microsoft's larger Phi-3 checkpoint, pairing 14B parameters with the same 128K context window as the smaller Phi-3 Mini. That combination makes it a good fit for long-document RAG and summarization workloads that need more reasoning headroom than a 3-4B model can offer, while still fitting a single 12 GB consumer GPU at Q4 — though with less headroom than the older 8.4 GB figure implied. It carries the same permissive MIT license as the rest of the Phi-3 family, which simplifies commercial evaluation. It's a reasonable step up from Phi-3 Mini when you need better reasoning quality and can spare the extra VRAM.
Editorial review
VRAM figures are empirical estimates. Actual usage varies by runtime, context length, and system configuration. Verify on your specific hardware before production use.
Will Phi-3 Medium 128K Instruct run on your machine?
Phi-3 Medium 128K Instruct is 14B parameters and needs 10.1 GB of VRAM at Q4_K_M — 8.6 GB of weights plus 1.5 GB of runtime overhead for the inference server itself.
VRAM by quantization
| Quantization | Weights | Needs (with overhead) | Quality |
|---|---|---|---|
| Q4_K_M | 8.6 GB | 10.1 GB | good |
| Q8_0 | 15 GB | 16.5 GB | high |
| FP16 | 28 GB | 29.5 GB | reference |
Fit on common hardware at Q4_K_M
| Hardware | Memory the model can use | System RAM | Verdict |
|---|---|---|---|
| CPU Only | None (CPU only) | 16 GB | CPU offload |
| RTX 4060 Laptop | 8 GB | 16 GB | CPU offload |
| RTX 3060 (12GB) | 12 GB | 32 GB | Tight |
| RTX 4060 Ti (16GB) | 16 GB | 32 GB | Comfortable |
| RTX 3090 | 24 GB | 64 GB | Comfortable |
| Apple Silicon (Unified Memory) 36 GB | 27 GB of 36 GB | 36 GB | Comfortable |
Comfortable means VRAM clears the requirement by 2 GB or more. Tight means it covers the requirement with no margin. CPU offload means the model does not fit in VRAM but system RAM is at least 1.6× the weights, so it will run at reduced speed — expect roughly 1–5 tokens per second. Figures are weights plus a fixed runtime overhead and exclude KV-cache growth, which scales with context length.
Apple Silicon shares one pool of memory between the system and the GPU, so a model cannot use all of it. These rows apply the same 75% usable fraction the Compatibility Checker uses, which is why a 36 GB Mac is graded on less than 36 GB.
Need more hardware for Phi-3 Medium 128K Instruct? Open the PC Builder for the 13B / 14B tier →
Ready to run this model locally?
Find a compatible interface in our Local AI Tools directory →