Best list
Best Models for Local RAG
Pick models for local RAG using embedding models, rerankers, and generator models that fit your hardware and source material.
Top picks
- Multilingual E5 Large
- BGE Reranker v2 M3
- Qwen3 Embedding
- Mistral Small 3.1
Grouped recommendations
Embedding candidates
Multilingual E5 Large, Qwen3 Embedding
Reranking candidate
BGE Reranker v2 M3
Generator candidates
Qwen3 235B A22B, Mistral Small 3.1, Gemma 3 27B
How to choose
RAG quality depends on the whole pipeline: chunking, embeddings, retrieval, reranking, prompts, and evaluation.
Related links
Sources
Sponsorship note
Built an AI tool or open-source project? Submit it for review or sponsor a featured placement on OpenSourcesAI.
Sponsor or submit