What hardware runs DeepSeek-R1-Distill-Llama 70B?
70B parameters. GGUF files: Q4_K_M 42.5 GB, Q8_0 75.0 GB (GGUF repo, model card). Est. VRAM incl. ~4k context: 47.8 GB (Q4) / 83.5 GB (Q8).
Hardware that fits at Q4 (real UK prices first, then cheapest sample price)
- Ryzen AI Max+ 395 mini-PC (128GB unified) (128 GB) ~$1999 sample
- RTX A6000 (48 GB) ~$4500 sample
- Mac Studio M3 Ultra (max 512GB unified) (512 GB) ~$9499 sample