What Can 48GB VRAM Run? Workstation-Class Local AI

Model fit at 48GB VRAM

The table applies to any configuration with approximately 48 GB of contiguous accessible memory. For Apple Silicon 64GB, approximately 48 GB is accessible for AI workloads (~75% of total pool).

Use the Local LLM Compatibility Checker to match specific models against your exact hardware configuration and workflow.

What 48GB unlocks over 24GB

• 70B models at Q4: The critical unlock. A 70B model at Q4_K_M needs ~38–42 GB. This fits in a 48 GB pool with 6–10 GB of headroom. No 24 GB GPU can do this without CPU offload at slow speeds. 70B at Q4 is the practical best-quality 70B that fits here.
• 30B and 32B at Q8: Q8 on a 32B model needs ~34 GB — impossible at 24 GB, comfortable at 48 GB. Near-lossless quality on a 30B model is a meaningful step above 24 GB where Q4 is the ceiling.
• 14B at FP16 with full context headroom: 14B FP16 uses ~28 GB, leaving 20 GB of headroom for KV cache at large context windows. Very long context at FP16 precision on a 14B model becomes practical.

Hardware paths to 48GB: architecture constraints

Architecture constraints by path

• RTX A6000 (CUDA / Ampere): Standard CUDA compute path. Ollama, LM Studio, vLLM, TGI, and llama.cpp all work without modification. The 768 GB/s bandwidth is lower than the consumer RTX 4090 (1008 GB/s) but operates on a 48 GB pool — you trade tokens-per-second versus model ceiling. Professional pricing typically means used market is the most realistic purchase path.
• Dual RTX 3090 / NVLink (CUDA / Ampere, bridged): The RTX 3090 is the last consumer NVIDIA card to support NVLink, making it the only practical consumer dual-GPU path to a unified 48 GB pool. The constraint is runtime support: Ollama sees each card as a separate 24 GB device and will not automatically pool them. vLLM with --tensor-parallel-size 2 or a TensorRT-LLM build with explicit tensor parallel config is required. The setup also demands a 1000 W+ PSU, an NVLink 3.0 bridge, and careful thermal management. Not a beginner path.
• Apple Silicon 64GB (Metal / unified memory): Unified memory means the CPU and GPU share the same physical pool — there is no discrete VRAM; the OS dynamically allocates the accessible GPU allocation (raise the limit with sudo sysctl iogpu.wired_limit_mb=57344). The Metal backend replaces CUDA, so CUDA-only runtimes and tools that require CUDA libraries do not run. Ollama, LM Studio, and llama.cpp all support Metal. The tradeoff: lower tokens-per-second at any given model size vs a CUDA workstation GPU, but dramatically simpler setup and the best watt-per-gigabyte-of-memory ratio of any path at this tier.

What 48GB still cannot do

• 70B at Q8: Q8 on a 70B model requires ~74 GB. Needs 80 GB+ (NVIDIA A100 80GB, H100 80GB) or cloud inference.
• 30B at FP16: FP16 on 30B requires ~60 GB. Exceeds 48 GB. Q8 is the practical ceiling for 30B+ at this tier.
• Multiple large models simultaneously: 70B at Q4 uses most of the 48 GB pool. Running a second model alongside it is not practical at this tier.
• Frontier MoE models: Very large mixture-of-experts models (120B+ parameter counts) at Q4 exceed 70 GB. Requires multi-GPU nodes or cloud.

Recommended models for the 48GB tier

• Llama 3.1 70B Q4_K_M — the headline model: ollama pull llama3.1:70b. This is the model class that 48GB is built for. Strong instruction following, long context, and broad capability at Q4 quality.
• Qwen 2.5 32B Q8 — near-lossless 30B quality: ollama pull qwen2.5:32b-q8_0. The other headline unlock at this tier. Near-FP16 quality on a 32B model — impossible at 24 GB.
• DeepSeek-R1 70B Q4 — reasoning at scale: ollama pull deepseek-r1:70b. Multi-step reasoning at the 70B parameter class. Strong for structured output, chain-of-thought, and complex analysis tasks.
• Qwen 2.5 14B FP16 — fast and precise: ollama pull qwen2.5:14b-fp16. Full precision on a 14B model with plenty of VRAM headroom. The best daily driver when generation speed matters more than 70B depth.

Setup paths by hardware track

Track A: NVIDIA RTX A6000 / Linux CUDA path

# Install Ollama on Linux
curl -fsSL https://ollama.com/install.sh | sh

# Pull the headline 70B Q4 workload for this tier
ollama pull llama3.1:70b

# Pull a high-quality 32B option at Q8
ollama pull qwen2.5:32b-q8_0

# Run the 70B model locally
ollama run llama3.1:70b

Track B: Dual RTX 3090 / NVLink workstation path

# Install Ollama on Linux
curl -fsSL https://ollama.com/install.sh | sh

# Confirm both GPUs are visible
nvidia-smi

# Pull a practical 70B Q4 workload
ollama pull llama3.1:70b

# Run the model — Ollama uses one GPU at a time
# For unified 48GB pool, switch to vLLM:
# pip install vllm
# python -m vllm.entrypoints.openai.api_server \
#   --model meta-llama/Llama-3.1-70B-Instruct \
#   --tensor-parallel-size 2

Ollama and llama.cpp already reach all 48 GB across two RTX 3090s by layer split, with no NVLink and no flags — that is the default path, and it costs you bandwidth rather than capacity. Use vLLM with --tensor-parallel-size 2 when you want tensor parallelism instead, which is faster. Verify your NVLink bridge is seated and that both cards appear under the same NUMA node in nvidia-smi topo -m.

Track C: Apple Silicon 64GB unified memory path

# Raise the usable GPU memory allocation limit first
sudo sysctl iogpu.wired_limit_mb=57344

# Pull and run the 70B model via Metal backend
ollama pull llama3.1:70b
ollama run llama3.1:70b

Apple Silicon uses the Metal backend — no CUDA runtimes. Expect 10–18 tokens per second on 70B Q4 on a 64GB M-series Max or Ultra, compared to 20–35 t/s on a CUDA workstation card at the same model. The tradeoff is simplicity and price-per-gigabyte of memory. The Apple Silicon guide covers Metal-specific configuration in full.

Best local AI workflows for this tier

• 70B reasoning and analysis: Complex document analysis, legal review, multi-step reasoning chains, and structured output tasks where 30B-class models fall short. 70B Q4 on a 48GB setup runs at interactive speed without cloud dependency.
• Near-lossless 30B inference: 32B models at Q8 are effectively indistinguishable from FP16 for most inference tasks. This is the right tier for teams that want high model quality without the complexity of cloud-serving a 70B model.
• Private local inference server: An RTX A6000 running vLLM can serve a 70B Q4 model to multiple concurrent users at acceptable throughput. Viable for small teams that need a private, uncensored, offline inference endpoint.
• Long-context workloads: The KV cache headroom at this tier makes very long context windows practical on 14B and 30B models. Good for RAG over large document sets, full codebase analysis, and extended agent loops.

Cloud GPU fallback

Even 48GB-class systems have a ceiling. 70B Q8, very large MoE models, and 120B+ experiments still exceed local memory or run too slowly for production-style testing. 80GB and larger cloud GPUs are the practical next step.

Some outbound provider links may be affiliate or partner links. OpenSourcesAI recommends cloud fallback only when the local hardware limit is the blocker.

Related hardware

FAQ

Disclosure

OpenSourcesAI may earn a commission or referral fee from links to cloud GPU providers or partner tools on this page. Editorial assessments are produced independently and are not influenced by commercial relationships. Hardware specs are sourced from manufacturer documentation. Model VRAM estimates are derived from GGUF quantization formulas and may vary across runtime versions and model architectures. Verify before making purchasing decisions.

Last verified: June 2026