NVIDIA GeForce RTX 4070 Ti SUPER × Mistral Small 3.1 24B
Can the RTX 4070 Ti SUPER run Mistral Small 3.1 24B?
Yes
Yes. The RTX 4070 Ti SUPER can run Mistral Small 3.1 24B at up to Q3_K_M (needs ~14.2 GB of 15.2 GB usable), at an estimated ~43.2 tokens/sec at Q3_K_M.
Speed is estimated from memory bandwidth calibrated with a measured 132.26 tok/s Llama 2 7B Q4_0 result (source). No direct public benchmark of this exact pair is recorded yet.
By quantization (8K context)
| Quant | Memory needed | Fits? | Est. tok/s |
|---|---|---|---|
| Q3_K_M | 14.2 GB | ✅ | ~43.2 |
| Q4_K_M | 17.1 GB | ❌ | — |
| Q5_K_M | 19.8 GB | ❌ | — |
| Q6_K | 22.5 GB | ❌ | — |
| Q8_0 | 28.4 GB | ❌ | — |
| F16 | 51.6 GB | ❌ | — |
Fastest hardware for Mistral Small 3.1 24B
- NVIDIA GeForce RTX 5090 — ~76.2 tok/s at Q4_K_M
- NVIDIA GeForce RTX 5080 — ~59.5 tok/s at Q3_K_M
- NVIDIA GeForce RTX 5070 Ti — ~57.8 tok/s at Q3_K_M
- NVIDIA GeForce RTX 4090 — ~48.9 tok/s at Q4_K_M
- NVIDIA GeForce RTX 4080 SUPER — ~48.5 tok/s at Q3_K_M
No suitable GPU? Rent one by the hour on Vast.ai or RunPod (affiliate links).
Get new models & benchmarks in your inbox
No spam. Unsubscribe anytime by replying to any email.