gpt-oss-20b (MoE)

OpenAI · 21B parameters (MoE, 3.6B active per token) · 131,072 token max context · Model card

Memory needed

QuantWeightsTotal @ 8K ctx
MXFP4 (native)13.0 GB14.6 GB

Approximate GGUF sizes from bits-per-weight; totals include fp16 KV cache and ~0.8 GB overhead.

Can my hardware run it?

DeviceVerdictBest quantEst. tok/s (Q4)
NVIDIA GeForce RTX 5090YesMXFP4 (native)~267
NVIDIA GeForce RTX 4090YesMXFP4 (native)~171
AMD Radeon RX 7900 XTXYesMXFP4 (native)~169
NVIDIA GeForce RTX 5080YesMXFP4 (native)~167
NVIDIA GeForce RTX 5070 TiYesMXFP4 (native)~163
NVIDIA GeForce RTX 3090 TiYesMXFP4 (native)~157
NVIDIA GeForce RTX 3090YesMXFP4 (native)~145
NVIDIA GeForce RTX 4080 SUPERYesMXFP4 (native)~136
NVIDIA GeForce RTX 4070 Ti SUPERYesMXFP4 (native)~122
Apple M5 Max (40-core GPU, 128GB)YesMXFP4 (native)~110
Apple M2 Ultra (76-core GPU, 192GB)YesMXFP4 (native)~86.7
Apple M3 Ultra (80-core GPU, 512GB)YesMXFP4 (native)~84.7
NVIDIA GeForce RTX 5060 Ti 16GBYesMXFP4 (native)~83.6
Apple M4 Max (40-core GPU, 128GB)YesMXFP4 (native)~76.4
Apple M5 Pro (20-core GPU, 64GB)YesMXFP4 (native)~61.0
Apple M3 Max (40-core GPU, 64GB)YesMXFP4 (native)~61.0
Apple M1 Max (32-core GPU, 64GB)YesMXFP4 (native)~56.2
NVIDIA DGX SparkYesMXFP4 (native)~52.6
Apple M4 Pro (20-core GPU, 48GB)YesMXFP4 (native)~46.6
Apple M4 (10-core GPU, 24GB)YesMXFP4 (native)~22.2
NVIDIA GeForce RTX 5070No——
NVIDIA GeForce RTX 4060 Ti 8GBNo——
NVIDIA GeForce RTX 3060 12GBNo——
NVIDIA GeForce RTX 3080 10GBNo——

Speeds estimated from memory bandwidth (calibrated per device with a measured llama.cpp benchmark where available).

Rent a GPU instead

Don't own the hardware for gpt-oss-20b (MoE)? Rent a cloud GPU with enough VRAM by the hour and run it there.

Vast.ai — cheap marketplace GPUs  ·  RunPod — on-demand GPU pods

Affiliate links: we may earn a commission at no extra cost to you.