Phi-4 14B

Microsoft · 14.7B parameters · 16,384 token max context · Model card

Memory needed

QuantWeightsTotal @ 8K ctx
Q3_K_M7.2 GB9.9 GB
Q4_K_M8.9 GB11.7 GB
Q5_K_M10.5 GB13.3 GB
Q6_K12.1 GB15.0 GB
Q8_015.6 GB18.6 GB
F1629.4 GB32.8 GB

Approximate GGUF sizes from bits-per-weight; totals include fp16 KV cache and ~0.8 GB overhead.

Can my hardware run it?

DeviceVerdictBest quantEst. tok/s (Q4)
NVIDIA GeForce RTX 5090YesQ8_0~124
NVIDIA GeForce RTX 4090YesQ8_0~79.9
AMD Radeon RX 7900 XTXYesQ8_0~79.1
NVIDIA GeForce RTX 5080YesQ6_K~78.1
NVIDIA GeForce RTX 5070 TiYesQ6_K~75.9
NVIDIA GeForce RTX 3080 10GBYesQ3_K_M (4K ctx)~74.5
NVIDIA GeForce RTX 3090 TiYesQ8_0~73.4
NVIDIA GeForce RTX 3090YesQ8_0~67.8
NVIDIA GeForce RTX 4080 SUPERYesQ6_K~63.6
NVIDIA GeForce RTX 4070 Ti SUPERYesQ6_K~56.7
NVIDIA GeForce RTX 5070YesQ4_K_M (4K ctx)~54.7
Apple M5 Max (40-core GPU, 128GB)YesF16~51.4
Apple M2 Ultra (76-core GPU, 192GB)YesF16~40.4
Apple M3 Ultra (80-core GPU, 512GB)YesF16~39.5
NVIDIA GeForce RTX 5060 Ti 16GBYesQ6_K~39.0
Apple M4 Max (40-core GPU, 128GB)YesF16~35.6
NVIDIA GeForce RTX 3060 12GBYesQ4_K_M (4K ctx)~32.4
Apple M5 Pro (20-core GPU, 64GB)YesF16~28.4
Apple M3 Max (40-core GPU, 64GB)YesF16~28.4
Apple M1 Max (32-core GPU, 64GB)YesF16~26.2
NVIDIA DGX SparkYesF16~24.5
Apple M4 Pro (20-core GPU, 48GB)YesF16~21.8
Apple M4 (10-core GPU, 24GB)Yes, usableQ8_0 (4K ctx)~10.3
NVIDIA GeForce RTX 4060 Ti 8GBNo——

Speeds estimated from memory bandwidth (calibrated per device with a measured llama.cpp benchmark where available).

Rent a GPU instead

Don't own the hardware for Phi-4 14B? Rent a cloud GPU with enough VRAM by the hour and run it there.

Vast.ai — cheap marketplace GPUs  ·  RunPod — on-demand GPU pods

Affiliate links: we may earn a commission at no extra cost to you.