DeepSeek V3 / R1 671B (MoE)

DeepSeek · 671B parameters (MoE, 37B active per token) · 131,072 token max context · Model card

Memory needed

QuantWeightsTotal @ 8K ctx
Q3_K_M327 GB339 GB
Q4_K_M407 GB421 GB
Q5_K_M478 GB494 GB
Q6_K554 GB572 GB
Q8_0713 GB736 GB
F161342 GB1384 GB

Approximate GGUF sizes from bits-per-weight; totals include fp16 KV cache and ~0.8 GB overhead.

Can my hardware run it?

DeviceVerdictBest quantEst. tok/s (Q4)
Apple M3 Ultra (80-core GPU, 512GB)Yes, usableQ3_K_M~9.3
NVIDIA GeForce RTX 5090No——
NVIDIA GeForce RTX 4090No——
NVIDIA GeForce RTX 3090No——
NVIDIA GeForce RTX 3090 TiNo——
NVIDIA GeForce RTX 5080No——
NVIDIA GeForce RTX 5070 TiNo——
NVIDIA GeForce RTX 5070No——
NVIDIA GeForce RTX 5060 Ti 16GBNo——
NVIDIA GeForce RTX 4080 SUPERNo——
NVIDIA GeForce RTX 4070 Ti SUPERNo——
NVIDIA GeForce RTX 4060 Ti 8GBNo——
NVIDIA GeForce RTX 3060 12GBNo——
NVIDIA GeForce RTX 3080 10GBNo——
AMD Radeon RX 7900 XTXNo——
NVIDIA DGX SparkNo——
Apple M4 (10-core GPU, 24GB)No——
Apple M4 Pro (20-core GPU, 48GB)No——
Apple M4 Max (40-core GPU, 128GB)No——
Apple M3 Max (40-core GPU, 64GB)No——
Apple M2 Ultra (76-core GPU, 192GB)No——
Apple M1 Max (32-core GPU, 64GB)No——
Apple M5 Max (40-core GPU, 128GB)No——
Apple M5 Pro (20-core GPU, 64GB)No——

Speeds estimated from memory bandwidth (calibrated per device with a measured llama.cpp benchmark where available).

Rent a GPU instead

Don't own the hardware for DeepSeek V3 / R1 671B (MoE)? Rent a cloud GPU with enough VRAM by the hour and run it there.

Vast.ai — cheap marketplace GPUs  ·  RunPod — on-demand GPU pods

Affiliate links: we may earn a commission at no extra cost to you.