Qwen Models

Qwen3 235B A22B

Alibaba Qwen · Qwen

Best for: Builders testing frontier-style open-weight reasoning and coding in hosted or multi-GPU environments.

Qwen3 235B A22B Thinking

Alibaba Qwen · Qwen

Best for: Multilingual chat, assistant workflows, and Qwen-family comparisons.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen3 30B A3B

Alibaba Qwen · Qwen

Best for: High-quality local chat, reasoning, and agent workflows on mid-range GPUs - sparsity keeps split-mode performance usable.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen3 32B

Alibaba Qwen · Qwen

Best for: Multilingual chat, assistant workflows, and Qwen-family comparisons.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen3 14B

Alibaba Qwen · Qwen

Best for: Multilingual chat, assistant workflows, and Qwen-family comparisons.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen3 8B

Alibaba Qwen · Qwen

Best for: Multilingual chat, assistant workflows, and Qwen-family comparisons.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen2.5 Coder 14B

Alibaba Qwen · Qwen

Best for: Coding assistants, repository help, and developer workflow evaluation.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen2.5 14B

Alibaba Qwen · Qwen

Best for: Builders hosting local AI on a single RTX 4080 (16 GB) or RTX 4090 (24 GB) who want near-frontier multilingual reasoning, mathematical accuracy, and agentic coding performance without multi-GPU infrastructure. The sweet spot between consumer accessibility and frontier-class open-weight capability for 2026 local deployment.

Local: Start with `ollama run qwen2.5:14b` — Ollama handles the download and quantization selection automatically. On a 16 GB GPU, Q4_K_M runs with comfortable KV cache headroom for 128K-context sessions. On an RTX 4090 (24 GB) or MacBook Pro M3 Max (48 GB), use Q8_0 for near-reference output quality at only a modest latency cost. For a private browser-based interface, connect Open WebUI to the Ollama API at localhost:11434. For agentic coding workflows, wire the endpoint into Continue or Cline — at 14B, the model handles multi-file context, tool-call planning, and structured JSON output with notably more reliability than 7B alternatives.

Qwen3 4B

Alibaba Cloud · Qwen3

Best for: Qwen3 4B is a compact model from Alibaba's Qwen3 family, small enough to fit comfortably on entry-level consumer GPUs while still handling everyday chat, coding help, and summarization.

Local: Compact Qwen3 variant. Q4 fits in 4 GB VRAM. Best compact pick for chat and coding on constrained hardware.

Qwen3.6 27B

Alibaba · Qwen 3.6

Best for: Qwen 3.6 — chat, reasoning, long-context workflows.

Local: The dense flagship of the Qwen3.6 generation - 27.8B parameters, 256K context, image input, Apache 2.0. The Ollama Q4_K_M is ~17 GB: treat a 24 GB card as the practical floor, because dense models pay the full split-mode cost - measured 3.1 tok/s on an RTX 3080 (10 GB) at 46% VRAM residency (2026-08-01). On hardware that fits it fully, this is the current Qwen quality pick.

Qwen3.6 35B (A3B)

Alibaba · Qwen 3.6

Best for: Qwen 3.6 — chat, reasoning, long-context workflows.

Local: Sparse mixture-of-experts - 36B total across 256 experts with ~3B active per token. 256K context, image input, Apache 2.0. The Ollama Q4_K_M is ~24 GB, yet sparsity keeps it genuinely usable even heavily split: measured 32.8 tok/s on an RTX 3080 (10 GB) with only 35% of the model in VRAM (2026-08-01). The strongest capability-per-VRAM entry in the current catalog for machines with fast system RAM.

Qwen3.5 9B

Alibaba · Qwen 3.5

Best for: Qwen 3.5 — chat, reasoning, long-context workflows.

Local: The mid-small model of the Qwen3.5 generation — 9.7B parameters, 256K context, image input, Apache 2.0. Carries the same hybrid attention stack as the 27B checkpoints (a full-attention layer every fourth block), so no architectureNotes are published for this record. The Ollama library Q4_K_M is 6.6 GB, Q8_0 is 11 GB and BF16 is 19 GB; with the 1.5 GB runtime overhead this site grades against, Q4 asks 8.1 GB, so a 10 GB card is a tight fit and 12 GB is comfortable. Worth knowing why this generation is still current at this size: neither Qwen3.6 nor Qwen3.8 released a checkpoint below 27B, so Qwen3.5 remains the newest Qwen line under that mark. No throughput measured on our rigs yet. Figures verified against the Hugging Face model card, its config.json, and the Ollama library on 2026-08-19.

Qwen3 Coder

Alibaba Qwen · Qwen

Best for: Developers comparing open coding models for Continue, Aider, Cline, and Roo Code workflows.

Local: Smaller or quantized coder variants can be tested locally for IDE and coding-agent workflows.

Qwen2.5 72B Instruct

Alibaba Qwen · Qwen

Best for: Teams requiring frontier-class open-weight reasoning and multilingual performance with local control. Practical for multi-GPU setups (2×RTX 3090/4090) or Mac Studio M2/M3 Ultra with 192 GB unified memory.

Local: Multi-GPU required for CUDA local inference at Q4. Mac Studio M2 Ultra (192 GB) handles Q4 well in Ollama. Use vLLM tensor parallelism for production multi-GPU serving.

Qwen2.5 Coder 32B

Alibaba Qwen · Qwen

Best for: Coding assistants, repository help, and developer workflow evaluation.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen2.5 Coder 7B

Alibaba Qwen · Qwen

Best for: Coding assistants, repository help, and developer workflow evaluation.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen2.5 7B

Alibaba Qwen · Qwen

Best for: Developers running local AI on 6–8 GB VRAM consumer GPUs who need a fast, highly capable 7B model for code completion, agentic function calling, and structured JSON output generation. The recommended 7B starting point for any local inference pipeline in 2026 before stepping to 14B or 72B.

Local: Start with `ollama run qwen2.5:7b` — the model downloads automatically and is immediately available at the Ollama API on localhost:11434. For structured JSON output, set the format parameter in the Ollama API request body (`"format": "json"`) or use the `--format json` flag in the CLI. For agentic function calling, wire the Ollama endpoint into Continue or Cline inside VS Code to get inline code completions and repository-level chat backed by local inference. For a browser-based chat UI, pair with Open WebUI against the same localhost endpoint.

Qwen2.5 72B

Alibaba Qwen · Qwen 2.5

Best for: Qwen2.5 72B is Alibaba's largest widely-deployed Qwen2.5 checkpoint, offering near-frontier reasoning, coding, and multilingual performance among open-weight models.

Local: 72B with 128K context. Q4 requires 45 GB VRAM — multi-GPU (2×RTX 3090/4090) or Mac Studio with 64 GB+ unified memory. Near-frontier open-weight reasoning and multilingual performance.

Qwen2.5 32B

Alibaba Qwen · Qwen 2.5

Best for: Qwen2.5 32B sits in the sweet spot between consumer-friendly VRAM requirements and near-frontier reasoning and coding performance, fitting Q4 quantization in a single 24 GB GPU like an RTX 4090 or 3090.

Local: 32B with 128K context. Q4 fits in 24 GB VRAM. Near-frontier reasoning and coding performance at consumer GPU scale. Compare with Qwen3 32B — Qwen3 often wins on reasoning benchmarks; Qwen2.5 32B has strong broad task coverage.

Qwen3-Coder 30B (A3B)

Alibaba · Qwen3 Coder

Best for: Qwen3 Coder — coding, agents, tool-use workflows.

Local: The coding variant of the 30B sparse MoE - same 128-expert, ~3B-active architecture tuned for agentic coding, with native 256K context for large repos. Q4_K_M is ~19 GB; measured 31.4 tok/s on an RTX 3080 (10 GB) at 46% VRAM residency (2026-08-01), so it stays responsive even when split. Apache 2.0. The natural upgrade path from Qwen2.5-Coder.

Qwen2.5 Math 72B

Alibaba Qwen · Qwen

Best for: Math-heavy prompts, reasoning tests, and educational workflow evaluation.

Local: Use the exact checkpoint and quantization that matches your hardware and latency target.

Qwen3.8 27B

Alibaba · Qwen 3.8

Best for: Qwen 3.8 — chat, reasoning, long-context workflows.

Local: The dense flagship of the Qwen3.8 generation — 27.8B parameters, 256K context, image input, Apache 2.0. Architecturally a hybrid rather than a plain GQA stack: config.json declares a full-attention layer every fourth block with linear-attention and gated-delta components between them, so ordinary KV-cache arithmetic does not transfer and no architectureNotes are published for this record. The Ollama library Q4_K_M is 18 GB, Q8_0 is 30 GB and BF16 is 56 GB; with the 1.5 GB runtime overhead this site grades against, Q4 asks 19.5 GB, so 20 GB is a tight fit and 24 GB clears it comfortably. Shares a parameter count and architecture string with Qwen3.6 27B and is a different checkpoint — do not read one record's measurements onto the other. No throughput measured on our rigs yet. An official FP8 checkpoint (Qwen/Qwen3.8-27B-FP8) exists for vLLM and SGLang serving, which is a server path, not a llama.cpp one. Figures verified against the Hugging Face model card, its config.json, and the Ollama library on 2026-08-19.

Qwen3 VL

Alibaba Qwen · Qwen

Best for: Builders adding visual understanding to open AI workflows.

Local: Cap image resolution in the processor if context is tight — reducing max pixels is the most direct lever on both token cost and latency.

Qwen2.5 VL

Alibaba Qwen · Qwen

Best for: Vision-language assistants, document images, and multimodal agent workflows.

Local: The 12.8M-pixel ceiling is a processor setting, not a hard limit of the model: lower it to trade visual detail for context budget and speed.

Qwen2 VL

Alibaba Qwen · Qwen

Best for: Vision-language baseline comparisons and multimodal prototypes.

Local: Because preprocessing is identical to Qwen2.5-VL, moving between the two is a checkpoint swap rather than a pipeline change — worth measuring before staying on this generation.

Qwen3 Embedding

Alibaba Qwen · Qwen

Best for: Builders who want a newer embedding family to compare against E5, BGE, and Jina.

Local: Smaller embedding variants are practical for local RAG and retrieval experiments.