Qwen3 14B

Alibaba Qwen · 14.8B parameters · 32,768 token max context · Model card

Memory needed

QuantWeightsTotal @ 8K ctx
Q3_K_M7.2 GB9.6 GB
Q4_K_M9.0 GB11.4 GB
Q5_K_M10.5 GB13.0 GB
Q6_K12.2 GB14.7 GB
Q8_015.7 GB18.3 GB
F1629.6 GB32.6 GB

Approximate GGUF sizes from bits-per-weight; totals include fp16 KV cache and ~0.8 GB overhead.

Can my hardware run it?

DeviceVerdictBest quantEst. tok/s (Q4)
NVIDIA GeForce RTX 5090YesQ8_0~124
NVIDIA GeForce RTX 4090YesQ8_0~79.3
AMD Radeon RX 7900 XTXYesQ8_0~78.5
NVIDIA GeForce RTX 5080YesQ6_K~77.5
NVIDIA GeForce RTX 5070 TiYesQ6_K~75.3
NVIDIA GeForce RTX 3080 10GBYesQ3_K_M (4K ctx)~74.0
NVIDIA GeForce RTX 3090 TiYesQ8_0~72.9
NVIDIA GeForce RTX 3090YesQ8_0~67.4
NVIDIA GeForce RTX 4080 SUPERYesQ6_K~63.2
NVIDIA GeForce RTX 4070 Ti SUPERYesQ6_K~56.3
NVIDIA GeForce RTX 5070YesQ4_K_M~54.3
Apple M5 Max (40-core GPU, 128GB)YesF16~51.1
Apple M2 Ultra (76-core GPU, 192GB)YesF16~40.2
Apple M3 Ultra (80-core GPU, 512GB)YesF16~39.3
NVIDIA GeForce RTX 5060 Ti 16GBYesQ6_K~38.7
Apple M4 Max (40-core GPU, 128GB)YesF16~35.4
NVIDIA GeForce RTX 3060 12GBYesQ4_K_M~32.2
Apple M5 Pro (20-core GPU, 64GB)YesF16~28.3
Apple M3 Max (40-core GPU, 64GB)YesF16~28.2
Apple M1 Max (32-core GPU, 64GB)YesF16~26.1
NVIDIA DGX SparkYesF16~24.4
Apple M4 Pro (20-core GPU, 48GB)YesF16~21.6
Apple M4 (10-core GPU, 24GB)Yes, usableQ8_0 (4K ctx)~10.3
NVIDIA GeForce RTX 4060 Ti 8GBNo——

Speeds estimated from memory bandwidth (calibrated per device with a measured llama.cpp benchmark where available).

Rent a GPU instead

Don't own the hardware for Qwen3 14B? Rent a cloud GPU with enough VRAM by the hour and run it there.

Vast.ai — cheap marketplace GPUs  ·  RunPod — on-demand GPU pods

Affiliate links: we may earn a commission at no extra cost to you.