Best list · Reviewed June 2026

Best Small Models for Consumer GPUs

Find small open models for consumer GPUs and local AI workflows, including Phi, Gemma, Mistral, Qwen, and embedding/reranking models.

Editorial review

Reviewed byOpenSourcesAI EditorialLast updatedJune 2026SourcesOfficial docs, GitHub repositories, vendor documentation, model cards, and source links listed on this page.

AI tools, model releases, pricing, licenses, and platform terms can change quickly. Verify the official source before production or commercial use.

Disclosure: OpenSourcesAI may earn a commission when you sign up through partner links. Our listings remain editorial unless specifically labeled as sponsored.

Top picks

  1. Phi-4 Mini
  2. Mistral Small 3.1
  3. Gemma 3 27B
  4. Multilingual E5 Large
  5. BGE Reranker v2 M3

Grouped recommendations

Best tiny/edge candidate

Phi-4 Mini

Best medium chat candidates

Mistral Small 3.1, Gemma 3 27B

Best retrieval helpers

Multilingual E5 Large, BGE Reranker v2 M3

How to choose

For consumer GPUs, start smaller than you think. Reliable speed beats a huge model that barely fits.

Related links

/models/phi-4-mini//models/mistral-small-3-1//models/gemma-3-27b//guides/how-much-vram-local-ai/

Related best-of pages

Best GPU for Local AIHow to think about VRAM, model size, quantization, and realistic local AI hardware choices.

Related resources

Continue comparing tools, models, stacks, and guides related to this category.

/models/phi-4-mini//models/mistral-small-3-1//models/gemma-3-27b//guides/how-much-vram-local-ai/Tools directory

Sources

Microsoft PhiMistral AI

Sponsorship note

Built an AI tool or open-source project? Submit it for review or sponsor a featured placement on OpenSourcesAI.

Sponsor or submit