Hardware · Apple M4

Reviewed July 2026

Mac Mini M4 24GB for Local LLMs: The Small Always-On Pick

The base Mac Mini M4 with 24GB of unified memory is the smallest, cheapest, and quietest entry into Apple Silicon local AI — a genuinely different use case from the 64GB Mac Studio and MacBook Pro Max/Ultra machines covered elsewhere on this site. This is the base M4 chip, not Pro or Max, so its 120 GB/s memory bandwidth is a fraction of the Max-tier 400 GB/s — that affects generation speed significantly even though the underlying unified-memory architecture is identical. Think of this as an always-on home AI server, not a performance machine.

Unified memory24 GB LPDDR5 (~18 GB accessible for AI)
Memory bandwidth120 GB/s
ArchitectureApple M4 (base chip, not Pro/Max)
Local AI tierComfortable 7B-8B / selective 13B Q4
GPU computeMetal (not CUDA)
Best runtimesOllama · LM Studio

Editorial review

Reviewed byOpenSourcesAI EditorialLast updatedJuly 2026SourcesApple M4 specifications, llama.cpp Metal performance benchmarks, GGUF quantization documentation, Ollama model library size data, and OpenSourcesAI editorial review.

This page covers the base Mac Mini M4 (24GB configuration) specifically — not the M4 Pro Mac Mini, and not the M4 Max/Ultra machines covered in the separate 64GB Apple Silicon guide. Memory bandwidth and accessible memory limits differ significantly between these tiers.

Quick verdict

The small, silent, always-on pick — comfortable 7B–8B daily work and a 13B Q4 stretch, won on power draw and footprint rather than speed.

  • 7B–8BQ8 with solid headroom in the ~18 GB accessible pool
  • 13BQ4_K_M comfortable; Q8 fits but thin
  • 30B+Does not fit reliably — beyond the accessible ceiling
  • Power~20–30W under AI load vs 170W+ for a GPU PC

Good fit for

  • An always-on, silent local AI box with a tiny desk footprint
  • Daily 7B–8B chat where power draw matters more than tokens per second

Wrong fit for

  • Performance — base M4 bandwidth (120 GB/s) is about a third of Max-tier chips
  • Anything at 30B or beyond

The Mac Mini M4 24GB is a genuinely good pick for local AI if your priority is a small, silent, low-power machine that is comfortable running 7B-8B models daily and can stretch to 13B at Q4 when needed. It is not a performance pick — the base M4 chip has 120 GB/s of memory bandwidth, roughly a third of the Max-tier chips, so token generation is noticeably slower than higher-end Apple Silicon or a discrete NVIDIA GPU at the same model size.

Where it wins is everywhere else: low power draw (roughly 20-30W under AI load versus 170W+ for a GPU-equipped PC), completely silent operation for typical chat workloads, a tiny desk footprint, and no separate graphics card, PSU upgrade, or case to manage. If you want an always-on local AI box tucked in a corner, this is one of the most practical choices available.

Check Mac Mini M4 model fit →

What this hardware can run

With roughly 18GB accessible for AI out of 24GB total unified memory, the comfortable ceiling is 7B-8B at Q8 and 13B at Q4_K_M. Larger models and higher-precision quantizations at this size class either run with thin margins or do not fit at all.

Model sizeBest quantizationMemory usedVerdictNotes
1B-4BFP16 or Q82-8 GBComfortableFits easily within the ~18 GB accessible pool. Fast generation even on base M4 bandwidth.
7BQ8~7.7 GBComfortableThe recommended daily tier. Leaves ~10 GB of headroom for context and background apps.
8BQ8~8.7 GBComfortableLlama 3.1 8B, Qwen3 8B. Fits with solid headroom within ~18 GB accessible.
7B-8BFP16~14-16 GBTight but workableFits within ~18 GB accessible with a short context window. Little room left for other apps.
13BQ4_K_M~7.9 GBComfortableFits well within the accessible pool. The recommended tier for more reasoning depth than 7B.
13BQ8~14 GBTightFits within ~18 GB accessible but with thin margin for context. Q4_K_M is the safer default.
30B+Q4~18-20 GBDoes not fit reliablyAt or beyond the ~18 GB accessible ceiling. Not a realistic tier for this hardware.

A note on the ~18GB limit: macOS dynamically caps how much of the unified memory pool AI processes can use, typically around 75 percent of total system memory. On a 24GB system that is roughly 18GB. You can raise this with sudo sysctl iogpu.wired_limit_mb=20480 (20GB) in Terminal, but doing so leaves less memory for macOS and other applications — not recommended unless you are running the Mac Mini as a dedicated, headless AI box.

Best model sizes for this hardware

  • 7B-8B at Q8: The recommended daily tier. Qwen3 8B and Llama 3.1 8B at Q8 use roughly 7.7-8.7GB, leaving comfortable headroom within the ~18GB accessible pool for context and other running apps.
  • 13B at Q4_K_M: The ceiling model for this hardware. Roughly 7.9GB, fitting well with margin to spare. Good choice when you want more reasoning depth than 7B-8B but still need a reliable, comfortable fit.
  • 4B and smaller at FP16: Very fast relative to the larger tiers on this chip. Good for quick, low-latency tasks where the base M4's lower bandwidth matters less because the model is small to begin with.

Recommended models

  • Qwen3 8B Q8 — The recommended starting model. Pull with ollama pull qwen3:8b-q8_0. Strong instruction following at near-FP16 quality within the accessible memory budget.
  • Llama 3.1 8B Q8 — A reliable, well-documented alternative. Pull with ollama pull llama3.1:8b-instruct-q8_0.
  • 13B at Q4_K_M: Pull with ollama pull qwen2.5:13b-instruct-q4_K_M (or the equivalent 13B model of your choice). The ceiling tier for this hardware — larger than 8B but at reduced precision.
  • Phi-4 Mini or Gemma 4 4B: Small, fast models for quick tasks and low-latency tool use, where the base M4's bandwidth is less of a limiting factor.

Getting started: first setup on the Mac Mini M4 24GB

# Install Ollama (macOS)
curl -fsSL https://ollama.com/install.sh | sh

# Metal acceleration is automatic — no flags needed
ollama serve &
ollama list

# Start with 8B at Q8 — good balance of quality and speed
ollama pull qwen3:8b-q8_0
ollama run qwen3:8b-q8_0

# Try the 13B ceiling model at Q4 (~7.9 GB)
ollama pull qwen2.5:13b-instruct-q4_K_M
ollama run qwen2.5:13b-instruct-q4_K_M

# Optional: raise the accessible memory limit if running headless
sudo sysctl iogpu.wired_limit_mb=20480

Recommended runtimes

  • Ollama — Metal acceleration is automatic on Apple Silicon, including the base M4. The easiest path to running GGUF models on macOS with no special configuration required.
  • LM Studio — Desktop GUI with full Metal support and a built-in model browser. Good for comparing 7B-13B models without a terminal, especially useful on a small home-server machine you may run headless later.
  • Open WebUI — Browser-based chat interface over Ollama. Pairs well with the Mac Mini's always-on nature — leave it running and access it from any device on your network.

Not supported: vLLM, TGI, and other CUDA-dependent runtimes will not run on Apple Silicon. For this hardware tier, Ollama and LM Studio cover the realistic use cases well.

Best local AI workflows for this hardware

  • Always-on home AI server: The Mac Mini M4's low power draw and silent operation make it a natural fit for a machine that stays on 24/7 serving a family or personal chat interface via Open WebUI, without the fan noise or power bill of a GPU tower.
  • Beginner local AI setup: Ollama + LM Studio or Open WebUI is a clean first stack. The Mac Mini M4 handles 7B-8B models comfortably, making it a low-friction way to get started with local AI without buying a discrete GPU.
  • General chat and note summarization: 7B-8B at Q8 or 13B at Q4 at comfortable-if-modest speeds — well suited to daily Q&A, drafting, and summarization tasks that do not need instant responses.
  • Lightweight local RAG: Pair an 8B chat model with a small local vector database for personal document search. Manageable document volumes work well within the ~18GB budget.

What this hardware cannot do well

  • Fast token generation: At 120 GB/s bandwidth — a fraction of the Max-tier 400 GB/s or a discrete GPU's 300-1000+ GB/s — the Mac Mini M4 generates tokens noticeably slower than higher-end Apple Silicon or an equivalent-tier NVIDIA card at the same model size.
  • Models beyond ~18GB: The accessible memory ceiling rules out 30B+ models and FP16 at 13B and above. This is a 7B-8B-comfortable, 13B-selective machine, not a large-model box.
  • CUDA runtimes: vLLM, TGI, and other CUDA-dependent tools do not run on Apple Silicon at all, regardless of chip tier.
  • Long context at the 13B tier: KV cache growth competes with model weights for the ~18GB budget. Keep context windows modest at 13B to avoid running into the memory ceiling.

Mac Mini M4 vs a budget NVIDIA card: the honest comparison

MetricMac Mini M4 24GBRTX 3060 12GB (PC)
Accessible memory for AI~18 GB12 GB VRAM
Memory bandwidth120 GB/s360 GB/s
7B Q8 generation speedSlowerFaster (~2-3x)
Power draw under AI load~20-30W~170W+ (card alone)
NoiseSilentFan noise under load
Separate GPU/PSU neededNoYes

The RTX 3060 12GB is faster per model. The Mac Mini M4 uses far less power, needs no separate graphics card, and runs silently. Choose based on whether raw speed or a quiet always-on footprint matters more for your use case.

Upgrade path

  • Mac Mini M4 Pro or Mac Studio (M4 Max/Ultra): The natural upgrade within the Apple Silicon ecosystem. The Max-tier chips jump to roughly 400 GB/s of memory bandwidth and support 64GB+ unified memory configurations — see the separate 64GB Apple Silicon guide for what that unlocks, including 70B models at Q4.
  • Switch to a discrete NVIDIA GPU for speed: If token generation speed at 7B-13B is the bottleneck, an RTX 3060 12GB or RTX 4070 Ti delivers meaningfully faster generation at the same model sizes, at the cost of higher power draw and needing a full PC.

Cloud GPU fallback

When a model exceeds what the Mac Mini M4 24GB can run locally, cloud GPU rentals provide on-demand access to much larger memory pools — a practical option for 30B+ models or fine-tuning jobs that this hardware is not built for.

Best cloud GPU for on-demand inference and spot rentals

RunPod

RunPod offers on-demand and spot GPU rentals with RTX 4090 (24 GB), A100 (80 GB), and H100 instances. A good fit for running 30B+ models or fine-tuning experiments that exceed the ~18GB accessible ceiling of the Mac Mini M4.

Pros

  • Spot pricing makes large-model experiments affordable
  • RTX 4090 and A100 instances available on demand
  • Full-stack GPU pods with persistent storage and networking

Cons

  • Spot instances can be interrupted mid-run
  • Requires Docker familiarity for custom environments

Partner link: OpenSourcesAI may earn a commission if you sign up.

Visit RunPod

Best cloud GPU for ML training and large-model fine-tuning

Lambda

Lambda Labs provides A100 and H100 GPU cloud instances purpose-built for ML workloads. A strong choice for fine-tuning runs and larger-model inference that the Mac Mini M4 cannot handle locally.

Pros

  • A100 and H100 instances for serious ML workloads
  • Clean pricing, no hidden fees, ML-focused infrastructure
  • Strong fit for fine-tuning and batch inference jobs

Cons

  • Less flexible than RunPod for short spot experiments
  • GPU availability can vary by region and instance type
Visit Lambda

Related hardware

Apple Silicon 64GB — the Max/Ultra tier for larger modelsRTX 3060 12GB — a faster budget NVIDIA alternativeLocal AI Hardware Fit DirectoryCheck Mac Mini M4 model fitVRAM and unified memory reference guide

FAQ

Is the Mac Mini M4 24GB good for local AI?

It is a solid entry point, not a performance pick. With 24GB of unified memory, roughly 18GB is accessible for AI workloads by default, which comfortably fits 7B-8B models at Q8 and 13B models at Q4_K_M with a thin margin. This is the base M4 chip, not the Pro or Max, so memory bandwidth (120 GB/s) is much lower than the Max-tier chips (around 400 GB/s) covered elsewhere on this site — expect noticeably slower token generation than a Mac Studio or MacBook Pro with an M-series Max chip.

How much memory is actually usable for AI on a 24GB Mac Mini M4?

macOS dynamically limits how much of the unified memory pool AI processes can allocate — typically around 75 percent of total system memory. On a 24GB system that works out to roughly 18GB accessible for a model plus its context, with the rest reserved for macOS and other running applications. This limit can be raised with `sudo sysctl iogpu.wired_limit_mb=<value>` in Terminal, though doing so leaves less headroom for everything else running on the machine.

What is the difference between the Mac Mini M4 24GB and the 64GB Apple Silicon guide on this site?

They are different use cases on the same underlying architecture. The 64GB Mac Studio/MacBook Pro guide covers Max and Ultra chips with roughly 400 GB/s of memory bandwidth and enough accessible memory for 70B models at Q4. The base Mac Mini M4 24GB has about 120 GB/s of bandwidth and roughly 18GB accessible — it is built for 7B-8B models comfortably and 13B at Q4 with margin, not for large models. Both use Metal instead of CUDA and share the same unified-memory tradeoffs, but the Mac Mini M4 is a much smaller, cheaper, and slower machine.

Should I buy a Mac Mini M4 or an RTX 3060 12GB for local AI?

The RTX 3060 12GB will run the same class of models (7B-14B) meaningfully faster, since discrete GPU memory bandwidth and CUDA acceleration outperform a base-tier unified memory chip. The Mac Mini M4 wins on power draw (roughly 20-30W under load versus 170W+ for a 3060-equipped PC), noise (silent, no fan spin-up under typical load), size, and not needing a separate GPU, PC case, or PSU. Choose the Mac Mini if you want a quiet always-on home server; choose the RTX 3060 12GB if raw tokens-per-second is the priority.

Can I run 13B models on the Mac Mini M4 24GB?

Yes, at Q4_K_M quantization (roughly 7.9GB), which fits within the ~18GB accessible pool alongside a moderate context window. Q8 on a 13B model needs roughly 14GB, which is tight but often workable with a short context; FP16 (~26GB) does not fit. For the most reliable experience, stick to Q4_K_M for 13B models and reserve Q8 for 7B-8B models.

What is the best first model to try on a Mac Mini M4 24GB?

Start with Qwen3 8B or Llama 3.1 8B at Q8 for the best balance of quality and speed. Run `ollama pull qwen3:8b-q8_0` then `ollama run qwen3:8b-q8_0`. Metal acceleration is automatic on macOS — no configuration needed. Once that feels comfortable, try a 13B model at Q4_K_M for more reasoning depth.

Disclosure

OpenSourcesAI may earn a commission or referral fee from links to hardware retailers, cloud GPU providers, or partner tools on this page. Editorial assessments are produced independently and are not influenced by commercial relationships. Hardware specs are sourced from Apple documentation. Model memory estimates are derived from GGUF quantization formulas and may vary across runtime versions and model architectures. Verify before making purchasing decisions.

Last verified: July 2026

Related guides

OllamaRun open models locally with a simple CLI, model library, desktop app, and local API.LM StudioDesktop app for discovering, downloading, chatting with, and serving local LLMs.Qwen3 8B8B open-weight model. Good for: coding, chat.Llama 3 8B8B open-weight model. Good for: chat, coding.Ollama + Open WebUI Starter StackOpen-source AI stack recipeWhat Is VRAM and How Much Do You Need for Local AI?Full VRAM reference guide: formulas, quantization tradeoffs, and hardware tier tables.What Is Quantization? GGUF, Q4, Q8, and Local Model CompressionHow quantization reduces model size and when Q4_K_M is the right tradeoff.Fine-Tuning Open-Weight ModelsPractical guide to fine-tuning and customizing local LLMs for specific tasks.

Check model fit for your exact setup

Use the compatibility checker with your unified memory amount to see which models and quantization levels are recommended for your specific Mac.

Check your hardwareVRAM and unified memory guide