NVIDIA GeForce RTX 3060 12GB × gpt-oss-20b (MoE)
Can the RTX 3060 12GB run gpt-oss-20b (MoE)?
No
No. gpt-oss-20b (MoE) needs at least ~14.6 GB, but the RTX 3060 12GB has ~11.4 GB usable. You would need CPU offloading (much slower) or more memory.
By quantization (8K context)
| Quant | Memory needed | Fits? | Est. tok/s |
|---|---|---|---|
| MXFP4 (native) | 14.6 GB | ❌ | — |
Fastest hardware for gpt-oss-20b (MoE)
- NVIDIA GeForce RTX 5090 — ~267 tok/s at MXFP4 (native)
- NVIDIA GeForce RTX 4090 — ~171 tok/s at MXFP4 (native)
- AMD Radeon RX 7900 XTX — ~169 tok/s at MXFP4 (native)
- NVIDIA GeForce RTX 5080 — ~167 tok/s at MXFP4 (native)
- NVIDIA GeForce RTX 5070 Ti — ~163 tok/s at MXFP4 (native)
Rent a GPU instead
gpt-oss-20b (MoE) doesn't fit here? Rent a cloud GPU with enough VRAM by the hour and run it there.
Vast.ai — cheap marketplace GPUs · RunPod — on-demand GPU pods
Affiliate links: we may earn a commission at no extra cost to you.
Get new models & benchmarks in your inbox
No spam. Unsubscribe anytime by replying to any email.