Open models: VRAM needed at Q4 and Q8
Estimates: real GGUF file size (Hugging Face, 16 Oct 2026) ×1.1 + 1 GB for ~4k context. gpt-oss ships natively in ~4-bit MXFP4, so Q4 = Q8 there.
| Model | Params | Family | Type | Q4 est. | Q8 est. | Source |
|---|---|---|---|---|---|---|
| Llama 3.2 3B | 3.2B | Llama | Dense | 3.2 GB | 4.7 GB | GGUF files |
| Qwen3 4B | 4B | Qwen | Dense | 3.8 GB | 5.7 GB | GGUF files |
| Gemma 3 4B | 4.3B | Gemma | Dense | 3.8 GB | 5.5 GB | GGUF files |
| Mistral 7B | 7.2B | Mistral | Dense | 5.8 GB | 9.5 GB | GGUF files |
| Llama 3.1 8B | 8B | Llama | Dense | 6.4 GB | 10.4 GB | GGUF files |
| Qwen3 8B | 8.2B | Qwen | Dense | 6.5 GB | 10.6 GB | GGUF files |
| Gemma 3 12B | 12B | Gemma | Dense | 9 GB | 14.8 GB | GGUF files |
| Phi-4 14B | 14.7B | Phi | Dense | 11 GB | 18.2 GB | GGUF files |
| Qwen3 14B | 14.8B | Qwen | Dense | 10.9 GB | 18.3 GB | GGUF files |
| DeepSeek-R1-Distill-Qwen 14B | 14.8B | DeepSeek | Dense | 10.9 GB | 18.3 GB | GGUF files |
| gpt-oss 20B | 21B | gpt | Dense | 16.2 GB | 16.2 GB | GGUF files |
| Mistral Small 3.1 24B | 24B | Mistral | Dense | 16.7 GB | 28.6 GB | GGUF files |
| Gemma 3 27B | 27B | Gemma | Dense | 19.2 GB | 32.6 GB | GGUF files |
| Qwen3 30B-A3B (MoE) | 30.5B | Qwen | MoE | 21.5 GB | 36.8 GB | GGUF files |
| Qwen3 32B | 32.8B | Qwen | Dense | 22.8 GB | 39.3 GB | GGUF files |
| Qwen2.5-Coder 32B | 32.8B | Qwen | Dense | 22.9 GB | 39.3 GB | GGUF files |
| Llama 3.3 70B | 70B | Llama | Dense | 47.8 GB | 83.5 GB | GGUF files |
| DeepSeek-R1-Distill-Llama 70B | 70B | DeepSeek | Dense | 47.8 GB | 83.5 GB | GGUF files |
| Llama 4 Scout 109B-A17B (MoE) | 109B | Llama | MoE | 72.9 GB | 127 GB | GGUF files |
| gpt-oss 120B | 117B | gpt | Dense | 72.9 GB | 72.9 GB | GGUF files |
| DeepSeek V3/R1 671B (MoE) | 671B | DeepSeek | MoE | 445.8 GB | 785.6 GB | GGUF files |