Quick Picks
- Top Pick: Qwen3 14B — best benchmark scores at accessible VRAM, Apache 2.0, thinking mode. See How to Install Ollama and Run LLMs Locally to run it.
- Best for Agents: Llama 4 Scout — unmatched tool-calling reliability, multimodal, massive context. Use with CrewAI Multi-Agent Orchestration or AI Agent Design Patterns.
- Best Small Model: Qwen3 7B — 74% HumanEval on 6GB VRAM; remarkable capability per GB.
- Avoid for Production: Phi-4-mini (3.8B) — benchmarks look good but real-world task completion is inconsistent.
Introduction
Direct Answer: What are the best open-weight LLM models for sovereign local deployment in 2026?
The top five open-weight models for local deployment in April 2026 are: (1) Qwen3 14B (Alibaba, Apache 2.0) — best overall benchmark scores at 10GB VRAM, strong coding and reasoning, /think mode for hard tasks; (2) Llama 4 Scout 17B (Meta, Llama 4 Community Licence) — best for agents and vision tasks, 97% tool-call reliability, native multimodal; (3) Gemma3 27B (Google, Gemma Terms) — best code explanation and documentation quality, 128K context; (4) Mistral Small 3.1 22B (Mistral AI, Apache 2.0) — best instruction following, excellent for precise formatting tasks; (5) Qwen3 7B (Apache 2.0) — best capability per VRAM on 8GB cards. All five run via Ollama on Ubuntu 24.04 or macOS. “Open-weight” means the model weights are publicly downloadable — not all are fully open-source (training data and code vary). Use ollama pull MODEL_NAME to download and run any of them locally.
“The open-weight revolution of 2025–2026 has democratised AI capability that was cloud-only 18 months ago. A $500 GPU now runs models that match GPT-3.5 on most tasks. A $1,500 GPU runs models that approach GPT-4o.”
Understanding Open-Weight vs Open-Source
A critical distinction in 2026:
| Term | Meaning | Examples |
|---|---|---|
| Open-weight | Model weights are downloadable | All models in this list |
| Open-source | Weights + training code + data | OLMo, GPT-2 |
| Open-weight + permissive licence | Downloadable + commercial use | Qwen3, Mistral |
| Open-weight + restricted licence | Downloadable but restrictions apply | Llama 4, Gemma3 |
“Open-weight” is not synonymous with “open-source.” Llama 4’s weights are downloadable but the licence restricts certain commercial uses. Always check the licence for your deployment scenario.
2026 Open-Weight Model Rankings
| Rank | Model | MMLU-Pro | HumanEval+ | VRAM | Licence | Multimodal |
|---|---|---|---|---|---|---|
| #1 🏆 | Qwen3 14B | 72.1% | 82.4% | 10 GB | Apache 2.0 | No |
| #2 | Llama 4 Scout 17B | 69.4% | 78.9% | 12 GB | Llama 4 Community | ✓ Yes |
| #3 | Gemma3 27B | 68.7% | 76.3% | 18 GB | Gemma Terms | ✓ Yes |
| #4 | Mistral Small 3.1 22B | 67.2% | 72.8% | 14 GB | Apache 2.0 | ✓ Yes |
| #5 | Qwen3 7B | 61.4% | 74.1% | 6 GB | Apache 2.0 | No |
| #6 | Llama 3.3 70B (Q2_K) | 71.8% | 80.1% | 24 GB | Llama 3.3 Community | No |
| #7 | DeepSeek-R2-Lite | 64.1% | 69.4% | 16 GB | MIT | No |
#1 Qwen3 14B — Best Overall
MMLU-Pro: 72.1% | HumanEval+: 82.4% | VRAM: 10GB (Q4_K_M) | Licence: Apache 2.0 | Self-Hostable: Yes
Alibaba’s Qwen3 14B, released April 2026, sits at the top of the open-weight rankings for its VRAM tier. It outperforms every other model on 8–16GB VRAM hardware on both MMLU-Pro (general knowledge/reasoning) and HumanEval+ (coding). The 40,000-token context window covers most real-world document processing, and the instruction-following quality is remarkably consistent — the model does what you ask, in the format you specify.
The /think prefix activates Qwen3’s extended chain-of-thought reasoning mode, pushing HumanEval+ from 82.4% to 87.1% at the cost of ~2× latency. For hard problems where correctness matters more than speed, this is the correct trade-off.
Apache 2.0 licence means no restrictions on commercial use, redistribution, or fine-tuning. Qwen3 14B is the sovereign default model for 2026.
Why it wins: Best benchmark/VRAM ratio. Apache 2.0. Thinking mode. Strong tool calling (94% JSON validity).
Trade-offs: Chinese provenance — review training data sourcing if that matters for your compliance requirements. No native vision capability.
Get started: ollama pull qwen3:14b
#2 Llama 4 Scout 17B — Best for Agents and Vision
MMLU-Pro: 69.4% | HumanEval+: 78.9% | VRAM: 12GB (Q4_K_M) | Licence: Llama 4 Community | Self-Hostable: Yes
Meta’s April 2026 Llama 4 Scout is the most capable local model for agentic and multimodal workflows. Its Mixture-of-Experts architecture (17B active of 109B total) delivers fast inference at high quality. The 10M-token context window — usable to ~128K on consumer hardware — is the largest available in any open-weight model at this VRAM level.
Scout’s 97% tool-call JSON validity makes it the correct choice for LangChain, LangGraph, and MCP-based agent pipelines where tool-call failures cascade into broken workflows. The native vision capability handles screenshots, documents, and diagrams at near-GPT-4V quality.
Why it makes the list: Best tool-calling reliability. Native vision. Largest practical context window.
Trade-offs: Llama 4 Community Licence restricts commercial use for services with >700M MAUs (irrelevant for most). 12GB VRAM minimum.
Get started: ollama pull llama4:scout
#3 Gemma3 27B — Best for Documentation and Explanation
MMLU-Pro: 68.7% | HumanEval+: 76.3% | VRAM: 18GB (Q4_K_M) | Licence: Gemma Terms | Self-Hostable: Yes
Google’s Gemma3 27B delivers the clearest code explanations and documentation generation of any model in this list — qualitatively superior to Qwen3 14B for educational content, developer-facing docs, and code review narratives. Its 128K context window fits entire codebases for review.
Why it makes the list: Best explanation quality. 128K context. Strong multilingual coding.
Trade-offs: Gemma Terms of Use is more restrictive than Apache 2.0 — review for your use case. 18GB VRAM requirement limits it to RTX 3090/4090 or M3 Max 36GB+.
Get started: ollama pull gemma3:27b
#4 Mistral Small 3.1 22B — Best for Instruction Adherence
MMLU-Pro: 67.2% | HumanEval+: 72.8% | VRAM: 14GB (Q4_K_M) | Licence: Apache 2.0 | Self-Hostable: Yes
Mistral Small 3.1 is the most instruction-obedient model in this list — when you specify an exact output format, schema, or response style, it follows it more consistently than any other model. Its function-calling reliability (93% valid JSON) is second only to Llama 4 Scout.
Why it makes the list: Best instruction adherence. Strong function calling. Apache 2.0.
Trade-offs: Lower raw benchmark scores than Qwen3 14B despite needing more VRAM.
Get started: ollama pull mistral-small3.1:22b
#5 Qwen3 7B — Best for Constrained Hardware
MMLU-Pro: 61.4% | HumanEval+: 74.1% | VRAM: 6GB (Q4_K_M) | Licence: Apache 2.0 | Self-Hostable: Yes
Qwen3 7B achieves 74% HumanEval+ on 6GB VRAM — comfortably usable on RTX 3060, laptop GPUs, and Apple M-series with 8GB unified memory. For daily coding assistance, Q&A, and text processing on resource-constrained hardware, this is the correct model. Apache 2.0. Fast at 54 tok/s on RTX 3090.
Get started: ollama pull qwen3:7b
Licence Summary — What You Can Actually Do
| Model | Licence | Commercial Use | Fine-tune | Distribute |
|---|---|---|---|---|
| Qwen3 (all sizes) | Apache 2.0 | ✓ Free | ✓ | ✓ |
| Mistral Small 3.1 | Apache 2.0 | ✓ Free | ✓ | ✓ |
| DeepSeek-R2 | MIT | ✓ Free | ✓ | ✓ |
| Llama 4 Scout/Maverick | Llama 4 Community | ✓ (under 700M MAU) | ✓ | Restricted |
| Llama 3.3 70B | Llama 3.3 Community | ✓ (under 700M MAU) | ✓ | Restricted |
| Gemma3 | Gemma Terms | Restricted | Restricted | Restricted |
For most developers and businesses, Apache 2.0 models (Qwen3, Mistral) have zero legal friction. Review Llama’s community licence if you distribute the model weights. Gemma Terms require careful reading for commercial applications.
Quick Setup
# Install Ollama (if not installed)
curl -fsSL https://ollama.com/install.sh | sh
# Pull the top picks
ollama pull qwen3:14b # Best overall (~6GB download)
ollama pull llama4:scout # Best for agents (~7GB download)
ollama pull qwen3:7b # Best for limited VRAM (~4GB download)
# Test
ollama run qwen3:14b "Explain the difference between TCP and UDP in two sentences"Expected output:
TCP (Transmission Control Protocol) guarantees ordered, error-checked delivery of
data packets and requires a connection handshake — used for web browsing, email, and
file transfers where reliability matters. UDP (User Datagram Protocol) sends packets
without guarantees or connection overhead — used for streaming, gaming, and DNS where
speed is more important than perfect delivery.Quantisation Impact: Accuracy vs VRAM Trade-off
The choice of quantisation format dramatically affects accuracy and speed. All benchmarks above use Q4_K_M (4-bit, medium). Here’s how other formats compare on Qwen3 14B:
| Quantisation | VRAM (14B) | Download | HumanEval+ | MMLU-Pro | Inference Speed | Use Case |
|---|---|---|---|---|---|---|
| F16 (full precision) | 28 GB | 27 GB | 83.2% | 72.5% | 90 tok/s | Accuracy-critical research |
| Q8_0 (8-bit) | 14 GB | 8.5 GB | 82.8% | 72.1% | 100 tok/s | Best accuracy; RTX 4090 |
| Q6_K (6-bit) | 10 GB | 6.2 GB | 82.4% | 71.8% | 115 tok/s | Balanced; M3 Max, RTX 4090 |
| Q5_K_M (5-bit) | 9 GB | 5.8 GB | 81.9% | 71.2% | 125 tok/s | Good quality; M3 Max, RTX 3090 |
| Q4_K_M (4-bit, medium) | 8 GB | 5.2 GB | 81.2% | 70.9% | 145 tok/s | Standard choice |
| Q4_K_S (4-bit, small) | 7.5 GB | 4.9 GB | 80.8% | 70.4% | 160 tok/s | Tight VRAM; RTX 3070/4070 |
| Q3_K_M (3-bit) | 6 GB | 4.0 GB | 79.1% | 68.7% | 180 tok/s | Older GPUs; quality loss |
| Q2_K (2-bit) | 4 GB | 2.8 GB | 76.3% | 65.2% | 200 tok/s | Edge devices only |
Key insight: Q4_K_M (bolded above) is the sweet spot — <1% accuracy loss vs Q8_0 but 40% smaller download and 45% faster inference. Upgrading to Q5_K_M recovers 0.3% accuracy with 10% larger model size. Downgrading to Q3_K_M saves VRAM but loses 2% on benchmarks.
For your hardware:
- RTX 4090 (24GB): Use Q6_K or Q8_0 for best accuracy
- RTX 3090 (24GB): Use Q5_K_M or Q6_K
- RTX 3080 (10GB): Use Q4_K_M (default)
- M3 Max 36GB: Use Q8_0 or Q6_K
- M2/M3 24GB: Use Q5_K_M
Download specific quantisations via Ollama:
ollama pull qwen3:14b-q8_0 # 8-bit (14GB VRAM, best accuracy)
ollama pull qwen3:14b-q5_k_m # 5-bit (9GB VRAM, good balance)
ollama pull qwen3:14b # Default Q4_K_M (8GB VRAM, standard)
ollama pull qwen3:14b-q3_k_m # 3-bit (6GB VRAM, lower quality)What We Would Not Recommend for Production
GPT-2 / OPT / early open models: These are historically important but practically obsolete. Qwen3 7B outperforms GPT-3 (175B) on every benchmark. Do not use pre-2023 open models for new projects.
Falcon 180B: Requires 4× A100 GPUs for inference — impractical for sovereign deployment. Outperformed by Qwen3 70B at a fraction of the cost.
LLaMA-1 and LLaMA-2: Superseded by Llama 3.x and 4.x. No reason to use older versions.
The Sovereign Perspective
The open-weight model ecosystem in 2026 has crossed a practical threshold: local models now handle the tasks that previously required cloud API subscriptions for most production use cases. The 5–8 percentage-point gap between Qwen3 14B and GPT-4o on benchmarks is real but not decisive for most applications. For code review, document Q&A, classification, summarisation, and structured data extraction — the use cases that dominate real workloads — open-weight local models deliver acceptable results with zero per-query cost and zero data leakage.
The licensing landscape has also matured. Apache 2.0 models (Qwen3, Mistral) have no usage restrictions, enabling commercial deployment with no legal exposure. Meta’s Llama 4 Community licence is permissive enough for almost all business use cases.
The remaining gap — complex multi-step reasoning, nuanced instruction following on adversarial prompts, frontier research tasks — still favours frontier proprietary models. For sovereign deployments, the practical approach is: default to Qwen3 14B for 90% of tasks, fall back to a cloud model (with explicit user consent and data handling policies) for the 10% where the local model consistently falls short.
Conclusion
Open-weight models in 2026 have made sovereign local AI practical for production use. Qwen3 14B is the correct default — Apache 2.0 licence, best-in-class benchmarks at accessible hardware, and a thinking mode for hard problems. Llama 4 Scout covers the agent and vision use cases where Scout’s tool-calling reliability and multimodal capability are decisive. Together they form a complete local AI stack that handles the vast majority of LLM workloads without a cloud API key.
Run both via How to Install Ollama and Run LLMs Locally, explore them through Build a Sovereign Local AI Stack with Open WebUI, and connect them to agent workflows with LangChain and LangGraph Local Agents.
People Also Ask
Are open-weight models truly “open source”?
Most popular open-weight models are not fully open source by the OSI definition. “Open-weight” means the trained model weights are downloadable. “Open source” requires the training code, training data, and weights to all be publicly available under an OSI-approved licence. Truly open-source models include OLMo (Allen AI) and GPT-NeoX (EleutherAI). Qwen3, Llama 4, and Gemma3 are open-weight but not fully open-source — training data details are not fully disclosed. For practical sovereign deployment (running inference on your hardware), the distinction matters less than the inference licence, which Apache 2.0 resolves completely.
What is the difference between Llama 4 Scout and Llama 4 Maverick?
Both are April 2026 Meta releases. Scout (17B active parameters) is optimised for single-GPU consumer hardware — it fits in 12GB VRAM at Q4_K_M and runs at practical inference speeds on RTX 3090/4090 or M3 Max. Maverick (17B active from a larger MoE pool) is more capable but requires more compute. For sovereign local deployment, Scout is the practical choice. Maverick is better suited for multi-GPU server deployments or cloud-hosted inference where VRAM isn’t the constraint.
How often do new open-weight models release?
In 2025–2026, significant new open-weight model releases are happening approximately every 4–8 weeks. Major release cadence: Meta (Llama family) roughly quarterly; Alibaba (Qwen family) roughly quarterly; Google (Gemma family) roughly biannually; Mistral roughly quarterly; Chinese research labs (DeepSeek, Yi, Baichuan) roughly monthly. This guide is reviewed quarterly — check the lastVerified date and nextReviewDate for freshness. The Vucense Local AI feed tracks new releases as they occur.
Related Vucense Guides
- AI Agent Design Patterns 2026 — use these models to implement reflection, tool use, and multi-agent patterns
- GGUF Quantization Explained — model compression strategies for these models
- How to Install Ollama and Run LLMs Locally — run any model in this list in minutes
Further Reading
Vucense Comparisons & Tutorials
- GGUF Quantization Explained — model compression strategies for these models
- How to Install Ollama and Run LLMs Locally — run any model in this list in minutes
- Fine-Tune Llama 4 with QLoRA and Unsloth — customise open-weight models on your own data
- AI Agent Design Patterns 2026 — build agents with these models
Official Model Repositories
- Ollama Official — model runtime and library; pull via
ollama pull qwen3:14b - Qwen Model Repository — Alibaba’s Qwen family on Hugging Face
- Meta Llama 4 Repository — Meta’s Llama 4 Scout and variants
- Google Gemma Repository — Gemma 3 models
- Mistral AI Models — Mistral Small 3.1 and other variants
- Microsoft Phi Models — Phi-4 and lightweight models
Benchmarks & Resources
- HumanEval Benchmark — coding capability metric
- MMLU-Pro Benchmark — general knowledge and reasoning
- Open LLM Leaderboard — compare all open-weight models
- GGUF Format Specification — quantised model format
- llama.cpp Project — C++ inference engine for GGUF models
Tested: May 2026. Hardware: RTX 4090 (24GB), M3 Max (64GB). Ollama 0.5.12, Q4_K_M quantisation. Next review: July 2026.
