Stack recipe · Reviewed June 2026
Enterprise RAG Stack with Access Control
A production RAG setup combining Qdrant, LlamaIndex, and a local LLM behind an authentication and role-based access control layer — for regulated enterprise environments.
Best for
Enterprise teams that need source-grounded AI answers over internal documents with per-user access control, per-query audit logging, and data residency guarantees.
Core tools
- Qdrant
- LlamaIndex
- Ollama
- n8n
- Keycloak (or JWT middleware)
Recommended models
- Mistral Small 3.1 (24B, Q4 — 15 GB VRAM) — reliable for structured document Q&A
- Qwen3 14B (Q8 — 16 GB VRAM) — strong reasoning with 128K context for large documents
- Llama 3.3 70B (Q4 — 41 GB VRAM) — near-frontier quality for high-stakes enterprise deployments
Hardware notes
16–32 GB VRAM for the LLM. Qdrant runs on CPU with fast NVMe storage. LlamaIndex orchestration runs on CPU. For concurrent user load, consider a dedicated vLLM inference endpoint rather than Ollama.
Setup steps
- Deploy Qdrant with persistent storage and configure collection-level access tokens.
- Set up LlamaIndex document pipeline: load documents, chunk, embed with a local embedding model, store in Qdrant with user-group metadata.
- Add authentication middleware: JWT token validation on every API endpoint (Keycloak, Auth0, or custom JWT middleware).
- Implement per-user Qdrant filters: each query only retrieves documents tagged with the user's access group.
- Deploy the LLM (Ollama for single-user dev, vLLM for concurrent production traffic).
- Wire n8n to log every query: user ID, timestamp, retrieved document IDs, and model response.
- Test access control: verify that user A cannot retrieve documents tagged for user B's access group.
- Add rate limiting and response monitoring before production rollout.
Trade-offs
Adding auth and access control significantly increases setup complexity vs. a basic RAG stack. Keycloak alone requires a running service and user management. The payoff is compliance auditability and per-user data isolation that basic RAG stacks cannot provide.
Alternatives
- Use Dify with its built-in knowledge base and API access controls for a lower-setup enterprise RAG alternative.
- Use a managed vector database with VPC isolation (Qdrant Cloud, Pinecone) when infrastructure management is the bottleneck.
- Use the private-sovereign-knowledge-base stack if compliance logging and data residency are the top requirements.
Related resources
Not sure if your PC has enough VRAM for this workflow?
Run the Local LLM Hardware Checker →FAQ
How is per-user access control enforced at the vector database level?
Qdrant supports payload-based filtering. Each document chunk is stored with a metadata field (e.g., access_group: ["legal", "finance"]). At query time, a server-side filter ensures only documents matching the authenticated user's groups are retrieved. The LLM never sees documents outside the user's access scope.
What is the minimum viable audit trail for compliance?
Log at minimum: authenticated user ID, query text, list of retrieved document IDs and their access labels, model response, and timestamp. n8n can capture this on every workflow execution. For HIPAA or SOC 2, also log model version, retrieval scores, and any flagged responses.
Get practical stack updates
Join the OpenSourcesAI update list for new stack recipes, tool notes, and developer-first comparisons.
For builders
Sponsor a clearly labeled stack placement
Sponsor and partner placements are labeled and reviewed separately from editorial recommendations. For sponsorship options, advertise with us. For submissions or corrections, use the submit page.