Hardware Match

Which model for NVIDIA L40S · 48 Go?

48 GB usable for the model. 86 open-weight models fit in memory.

The margin goes to context: count ~1 GB per 8k tokens for a 7B model. A small margin means a short context.

Model	Elo	Quantization	RAM required	Margin	Speed	Run
Gemma 4 31BGoogle · 31B	1451	Q8_0	39 Go	+9 Go	tight	Hugging Face ↗
Gemma 4 26B-A4BGoogle · 26B	1438	Q8_0	33 Go	+15 Go	comfortable	Hugging Face ↗
Gemma 3 27BGoogle · 27B	1365	Q8_0	34 Go	+14 Go	tight	ollama run gemma3:27b
Qwen3 32BAlibaba · 32.8B	1347	Q8_0	41 Go	+7 Go	tight	ollama run qwen3:32b
Gemma 3 12BGoogle · 12B	1342	FP16	28 Go	+20 Go	comfortable	ollama run gemma3:12b
QwQ 32BAlibaba · 32.8B	1336	Q8_0	41 Go	+7 Go	tight	ollama run qwq
Gemma 3n E4BGoogle · 8B	1317	FP16	20 Go	+28 Go	comfortable	ollama run gemma3n:e4b
Llama 3.3 70BMeta · 70.6B	1317	Q4_K_M	46 Go	+2 Go	tight	ollama run llama3.3:70b
gpt-oss-20bOpenAI · 21B	1317	FP16	48 Go	+0 Go	tight	ollama run gpt-oss:20b
Nemotron 3 Nano 30B-A3BNVIDIA · 30B	1314	Q8_0	37 Go	+11 Go	tight	Hugging Face ↗
Granite 4.1 8BIBM · 8B	1306	FP16	20 Go	+28 Go	comfortable	Hugging Face ↗
Gemma 3 4BGoogle · 4B	1303	FP16	11 Go	+37 Go	comfortable	ollama run gemma3:4b
Qwen2.5 72BAlibaba · 72.7B	1303	Q4_K_M	48 Go	+0 Go	tight	ollama run qwen2.5:72b
Llama 3.1 Nemotron 70BNVIDIA · 70.6B	1299	Q4_K_M	46 Go	+2 Go	tight	ollama run nemotron:70b
Qwen2.5-Coder 32BAlibaba · 32.8B	1270	Q8_0	41 Go	+7 Go	tight	ollama run qwen2.5-coder:32b
Phi-4Microsoft · 14.7B	1256	FP16	34 Go	+14 Go	tight	ollama run phi4
Phi-4 ReasoningMicrosoft · 14.7B	1256	FP16	34 Go	+14 Go	tight	ollama run phi4-reasoning
OLMo 2 32BAllen AI · 32B	1251	Q8_0	40 Go	+8 Go	tight	Hugging Face ↗
CodestralMistral AI · 22.2B	1240	Q8_0	28 Go	+20 Go	comfortable	ollama run codestral
Ministral 8BMistral AI · 8.02B	1237	FP16	20 Go	+28 Go	comfortable	Hugging Face ↗
Llama 3.1 8BMeta · 8.03B	1211	FP16	20 Go	+28 Go	comfortable	ollama run llama3.1:8b
Mixtral 8x7BMistral AI · 46.7B	1197	Q4_K_M	31 Go	+17 Go	comfortable	ollama run mixtral:8x7b
Llama 3.2 3BMeta · 3.21B	1166	FP16	9 Go	+39 Go	comfortable	ollama run llama3.2:3b
Llama 3.2 1BMeta · 1.24B	1111	FP16	5 Go	+43 Go	comfortable	ollama run llama3.2:1b
Mistral 7BMistral AI · 7.25B	1110	FP16	18 Go	+30 Go	comfortable	ollama run mistral:7b
Kokoro-82MHexgrad · 0.08B	—	FP16	2 Go	+46 Go	comfortable	Hugging Face ↗
ChatterboxResemble AI · 0.5B	—	FP16	3 Go	+45 Go	comfortable	Hugging Face ↗
Parakeet TDT 0.6B v3NVIDIA · 0.6B	—	FP16	3 Go	+45 Go	comfortable	Hugging Face ↗
Qwen3 0.6BAlibaba · 0.6B	—	FP16	3 Go	+45 Go	comfortable	ollama run qwen3:0.6b
TripoSRStability AI · 0.5B	—	FP16	3 Go	+45 Go	comfortable	Hugging Face ↗
Gemma 3 1BGoogle · 1B	—	FP16	4 Go	+44 Go	comfortable	ollama run gemma3:1b
Stable Fast 3DStability AI · 1B	—	FP16	4 Go	+44 Go	comfortable	Hugging Face ↗
TRELLISMicrosoft · 1.2B	—	FP16	5 Go	+43 Go	comfortable	Hugging Face ↗
Whisper large-v3OpenAI · 1.55B	—	FP16	5 Go	+43 Go	comfortable	Hugging Face ↗
Dia 1.6BNari Labs · 1.6B	—	FP16	6 Go	+42 Go	comfortable	Hugging Face ↗
Gemma 4 E2BGoogle · 2B	—	FP16	6 Go	+42 Go	comfortable	Hugging Face ↗
Granite 4.1 3BIBM · 3B	—	FP16	9 Go	+39 Go	comfortable	Hugging Face ↗
Hunyuan3D 2.1Tencent · 3B	—	FP16	9 Go	+39 Go	comfortable	Hugging Face ↗
Ministral 3 3BMistral AI · 3B	—	FP16	9 Go	+39 Go	comfortable	Hugging Face ↗
Orpheus 3BCanopy Labs · 3B	—	FP16	9 Go	+39 Go	comfortable	Hugging Face ↗
SmolLM3 3BHugging Face · 3.1B	—	FP16	9 Go	+39 Go	comfortable	Hugging Face ↗
Voxtral Mini 3BMistral AI · 3B	—	FP16	9 Go	+39 Go	comfortable	Hugging Face ↗
Phi-4 MiniMicrosoft · 3.8B	—	FP16	10 Go	+38 Go	comfortable	ollama run phi4-mini
FLUX.2 [klein] 4BBlack Forest Labs · 4B	—	FP16	11 Go	+37 Go	comfortable	Hugging Face ↗
Gemma 4 E4BGoogle · 4B	—	FP16	11 Go	+37 Go	comfortable	Hugging Face ↗
Qwen3 4BAlibaba · 4.02B	—	FP16	11 Go	+37 Go	comfortable	ollama run qwen3:4b
TRELLIS.2Microsoft · 4B	—	FP16	11 Go	+37 Go	comfortable	Hugging Face ↗
CogVideoX 5BZhipu AI · 5B	—	FP16	13 Go	+35 Go	comfortable	Hugging Face ↗
Wan 2.2 5BAlibaba · 5B	—	FP16	13 Go	+35 Go	comfortable	Hugging Face ↗
Z-Image TurboAlibaba · 6B	—	FP16	15 Go	+33 Go	comfortable	Hugging Face ↗
Lucie 7BOpenLLM France · 6.7B	—	FP16	17 Go	+31 Go	comfortable	Hugging Face ↗
OLMo 3 7BAllen AI · 7B	—	FP16	17 Go	+31 Go	comfortable	Hugging Face ↗
Apertus 8BSwiss AI · 8B	—	FP16	20 Go	+28 Go	comfortable	Hugging Face ↗
Command R7BCohere · 8B	—	FP16	20 Go	+28 Go	comfortable	ollama run command-r7b
DeepSeek-R1 Distill 8BDeepSeek · 8.03B	—	FP16	20 Go	+28 Go	comfortable	ollama run deepseek-r1:8b
Granite 3.3 8BIBM · 8.1B	—	FP16	20 Go	+28 Go	comfortable	ollama run granite3.3:8b
Hermes 3 8BNous Research · 8.03B	—	FP16	20 Go	+28 Go	comfortable	ollama run hermes3:8b
HunyuanVideo 1.5Tencent · 8.3B	—	FP16	20 Go	+28 Go	comfortable	Hugging Face ↗
Ministral 3 8BMistral AI · 8B	—	FP16	20 Go	+28 Go	comfortable	Hugging Face ↗
Qwen3 8BAlibaba · 8.2B	—	FP16	20 Go	+28 Go	comfortable	ollama run qwen3:8b
CodeGemma 7BGoogle · 8.5B	—	FP16	21 Go	+27 Go	comfortable	ollama run codegemma
InternLM3 8BShanghai AI Lab · 8.8B	—	FP16	21 Go	+27 Go	comfortable	Hugging Face ↗
EuroLLM 9BUTTER Project · 9.15B	—	FP16	22 Go	+26 Go	comfortable	Hugging Face ↗
GLM-ImageZhipu AI · 9B	—	FP16	22 Go	+26 Go	comfortable	Hugging Face ↗
Mochi 1Genmo · 10B	—	FP16	24 Go	+24 Go	comfortable	Hugging Face ↗
Falcon 3 10BTII · 10.3B	—	FP16	25 Go	+23 Go	comfortable	ollama run falcon3:10b
FLUX.1 [dev]Black Forest Labs · 12B	—	FP16	28 Go	+20 Go	comfortable	Hugging Face ↗
Gemma 4 12BGoogle · 12B	—	FP16	28 Go	+20 Go	comfortable	Hugging Face ↗
LTX-2.3Lightricks · 22B	—	Q8_0	28 Go	+20 Go	comfortable	Hugging Face ↗
Mistral Nemo 12BMistral AI · 12.2B	—	FP16	29 Go	+19 Go	comfortable	ollama run mistral-nemo
Devstral SmallMistral AI · 24B	—	Q8_0	30 Go	+18 Go	comfortable	ollama run devstral
Magistral SmallMistral AI · 24B	—	Q8_0	30 Go	+18 Go	comfortable	ollama run magistral
Mistral Small 3.2Mistral AI · 24B	—	Q8_0	30 Go	+18 Go	comfortable	ollama run mistral-small3.2
Voxtral Small 24BMistral AI · 24B	—	Q8_0	30 Go	+18 Go	comfortable	Hugging Face ↗
Ministral 3 14BMistral AI · 14B	—	FP16	33 Go	+15 Go	comfortable	Hugging Face ↗
Qwen3.6 27BAlibaba · 27B	—	Q8_0	34 Go	+14 Go	tight	Hugging Face ↗
Wan 2.2 A14BAlibaba · 27B	—	Q8_0	34 Go	+14 Go	tight	Hugging Face ↗
Qwen3 14BAlibaba · 14.8B	—	FP16	35 Go	+13 Go	tight	ollama run qwen3:14b
DeepSeek-Coder-V2 LiteDeepSeek · 15.7B	—	FP16	37 Go	+11 Go	tight	ollama run deepseek-coder-v2:16b
StarCoder 2BigCode · 16B	—	FP16	37 Go	+11 Go	tight	ollama run starcoder2:15b
HiDream-I1HiDream · 17B	—	FP16	39 Go	+9 Go	tight	Hugging Face ↗
Aya Expanse 32BCohere · 32.3B	—	Q8_0	40 Go	+8 Go	tight	ollama run aya-expanse:32b
FLUX.2 [dev]Black Forest Labs · 32B	—	Q8_0	40 Go	+8 Go	tight	Hugging Face ↗
DeepSeek-R1 Distill 32BDeepSeek · 32.8B	—	Q8_0	41 Go	+7 Go	tight	ollama run deepseek-r1:32b
Qwen3.6 35B-A3BAlibaba · 35B	—	Q8_0	43 Go	+5 Go	tight	Hugging Face ↗
Qwen-Image 2512Alibaba · 20B	—	FP16	46 Go	+2 Go	tight	Hugging Face ↗

Refine with the Finder Other configurations