What can the Apple M3 Ultra (80-core GPU, 512GB) run?
512 GB unified memory (~384 GB usable for models) · 800 GB/s memory bandwidth
Measured: 92.14 tok/s on Llama 2 7B Q4_0 (llama.cpp tg128). Source: llama.cpp Apple Silicon benchmarks (Llama 2 7B Q4_0, tg128)
| Model | Verdict | Best quant that fits | Est. tok/s (Q4) |
|---|---|---|---|
| Llama 3.1 8B Instruct | Yes | F16 | ~72.6 |
| Llama 3.3 70B Instruct | Yes, usable | F16 | ~8.2 |
| Llama 3.2 3B Instruct | Yes | F16 | ~182 |
| Llama 4 Scout (17B active, 16 experts) | Yes | F16 | ~20.7 |
| Qwen3 8B | Yes | F16 | ~70.8 |
| Qwen3 14B | Yes | F16 | ~39.3 |
| Qwen3 32B | Yes, usable | F16 | ~17.7 |
| Qwen3 30B-A3B (MoE) | Yes | F16 | ~86.5 |
| Qwen3 235B-A22B (MoE) | Yes, usable | Q8_0 | ~14.6 |
| Qwen2.5 Coder 32B Instruct | Yes, usable | F16 | ~17.7 |
| Gemma 3 4B | Yes | F16 | ~135 |
| Gemma 3 12B | Yes | F16 | ~47.6 |
| Gemma 3 27B | Yes | F16 | ~21.2 |
| Mistral Small 3.1 24B | Yes | F16 | ~24.2 |
| Mistral Nemo 12B | Yes | F16 | ~47.6 |
| Phi-4 14B | Yes | F16 | ~39.5 |
| DeepSeek R1 Distill Qwen 32B | Yes, usable | F16 | ~17.7 |
| DeepSeek R1 Distill Llama 70B | Yes, usable | F16 | ~8.2 |
| gpt-oss-20b (MoE) | Yes | MXFP4 (native) | ~84.7 |
| gpt-oss-120b (MoE) | Yes | MXFP4 (native) | ~50.0 |
| DeepSeek V3 / R1 671B (MoE) | Yes, usable | Q3_K_M | ~9.3 |
Estimated tok/s = calibrated bandwidth ÷ active weight size. Real results vary with runtime, context and drivers.
No suitable GPU? Rent one by the hour on Vast.ai or RunPod (affiliate links).