Memory requirements calculator

Preset model
Total params (B)
Active params (B, MoE)
Quantization
Context (tokens)
Layers / KV heads / head dim
Hardware

No suitable GPU? Rent one by the hour on Vast.ai or RunPod (affiliate links).