What can the RTX 4090 run?
| Text models: 258 | Visual models: 17 | Strongest first |
- Memory
- 24 GB
- Bandwidth
- 1,008 GB/s
- Maker
- NVIDIA
Open the calculator on this machineEvery model for 24 GB of VRAM
Text models that run on it
| Model | Best fit | Context | Speed | Strongest at |
|---|---|---|---|---|
| Qwen3.8 27B | 5-bit | 48K | 35 tok/s | Chat, Coding, Agents, Docs |
| Muse Glimmer 30B | 5-bit | 128K | 33 tok/s | Docs |
| Qwen3.6 35B A3B | 4-bit | 72K | 197 tok/s | Chat, Coding, Agents, Docs |
| Gemma 4 31B | 4-bit | 80K | 35 tok/s | Chat, Coding, Agents, Docs |
| Gemma 4 26B A4B | 6-bit | 184K | 161 tok/s | Chat, Coding, Agents, Docs |
| Qwen3.5 9B | 8-bit | 256K | 71 tok/s | Chat, Coding, Agents, Docs |
| GLM-4.7 Flash | 5-bit | 24K | 171 tok/s | Chat, Coding, Agents |
| Gemma 4 12B | 8-bit | 256K | 51 tok/s | Chat, Coding, Docs |
| Nemotron 3.5 Lightning 30B A3B | 5-bit | 176K | 161 tok/s | Agents, Docs |
| Qwen3.5 4B | 8-bit | 256K | 124 tok/s | Agents, Docs |
| Devstral Small 2 24B | 6-bit | 20K | 35 tok/s | Coding |
| LFM2.5 24B A2B | 6-bit | 128K | 212 tok/s | Agents |
| Mistral Small 3.2 24B | 6-bit | 20K | 35 tok/s | Chat, Coding, Agents |
| Ministral 3 14B | 8-bit | 48K | 47 tok/s | Coding, Agents, Docs |
| Gemma 4 E4B | 8-bit | 128K | 110 tok/s | Chat, Coding, Docs |
| gpt-oss 20B | MXFP4 | 128K | 197 tok/s | Agents |
| Ministral 3 8B | 8-bit | 96K | 73 tok/s | Docs |
| Gemma 4 E2B | 8-bit | 128K | 191 tok/s | Chat |
| Olmo 3.1 32B | 4-bit | 48K | 34 tok/s | Chat, Coding, Agents, Docs |
| Llama 3.1 8B | 8-bit | 112K | 82 tok/s | Chat, Coding, Agents, Docs |
| Qwen3 14B | 8-bit | 40K | 45 tok/s | Not rated |
| Qwen3 VL 8B Instruct | 8-bit | 88K | 75 tok/s | Text and images |
| Qwen3 8B | 8-bit | 40K | 80 tok/s | Not rated |
| Qwen2.5 7B Instruct | 8-bit | 32K | 87 tok/s | Not rated |
| Qwen2.5 VL 7B Instruct | 8-bit | 125K | 80 tok/s | Text and images |
| Qwen3 VL 4B Instruct | 8-bit | 120K | 129 tok/s | Text and images |
| Qwen3 1.7B | 8-bit | 40K | 256 tok/s | Not rated |
| Qwen3 4B | 8-bit | 40K | 140 tok/s | Not rated |
| Qwen3 32B | 3-bit | 24K | 42 tok/s | Not rated |
| Qwen3 4B Instruct 2507 | 8-bit | 128K | 140 tok/s | Not rated |
| Qwen3.6 27B | 5-bit | 48K | 35 tok/s | Text and images |
| Qwen2.5 Coder 14B Instruct | 8-bit | 32K | 44 tok/s | Coding |
| Qwen3 Coder 30B A3B Instruct | 5-bit | 16K | 169 tok/s | Coding |
| Qwen3.5 27B | 5-bit | 48K | 35 tok/s | Text and images |
| Qwen3.5 35B A3B | 4-bit | 72K | 197 tok/s | Text and images |
| NVIDIA Nemotron 3 Nano 30B A3B BF16 | 5-bit | 176K | 161 tok/s | Not rated |
| Qwen2.5 Coder 7B Instruct | 8-bit | 32K | 87 tok/s | Coding |
| Mistral 7B Instruct V0.2 | 8-bit | 32K | 86 tok/s | Not rated |
| Chandra OCR 2 | 8-bit | 256K | 111 tok/s | Reading documents |
| Qwen3 30B A3B | 5-bit | 16K | 169 tok/s | Not rated |
| Qwen3 VL 2B Instruct | 8-bit | 176K | 247 tok/s | Text and images |
| Gemma 3 4B IT | 8-bit | 128K | 154 tok/s | Text and images |
| Qwen3 30B A3B Instruct 2507 | 5-bit | 16K | 169 tok/s | Not rated |
| Llama 3.2 3B Instruct | 8-bit | 128K | 176 tok/s | Not rated |
| Laguna XS.2 | 4-bit | 80K | 200 tok/s | Not rated |
| OTel 2.0 LLM 31B IT | 4-bit | 80K | 35 tok/s | Not rated |
| Qwen2.5 32B Instruct | 3-bit | 24K | 42 tok/s | Not rated |
| MiniCPM V 4.7 35B A3B | 4-bit | 88K | 197 tok/s | Text and images |
| Qwen2.5 14B Instruct | 8-bit | 32K | 44 tok/s | Not rated |
| Laguna XS 2.1 | 4-bit | 80K | 200 tok/s | Not rated |
| Qwen3 1.7B Base | 8-bit | 32K | 291 tok/s | Not rated |
| Diffusiongemma 26B A4B IT | 6-bit | 88K | 155 tok/s | Text and images |
| Qwen2.5 VL 32B Instruct | 3-bit | 24K | 41 tok/s | Text and images |
| Qwen2.5 Coder 32B Instruct | 3-bit | 24K | 42 tok/s | Coding |
| Qwen2 VL 7B Instruct | 8-bit | 32K | 80 tok/s | Text and images |
| Ornith 1.5 9B | 8-bit | 256K | 71 tok/s | Not rated |
| Granite 4.1 3B | 8-bit | 128K | 158 tok/s | Not rated |
| Qwen3 VL 30B A3B Instruct | 4-bit | 44K | 183 tok/s | Text and images |
| Mistral 7B V0.1 | 8-bit | 32K | 86 tok/s | Not rated |
| Meta Llama 3 8B Instruct Abliterated V3 | 8-bit | 8K | 82 tok/s | Not rated |
Best fit is the highest quality, up to 8-bit, that leaves the use case’s context. Ranked by our use-case scores, then by Hugging Face downloads. Speeds are rough, plus or minus 25%. The top 60 of 258 are shown.
Image and video models
| Model | Best fit | Per output | Strongest at |
|---|---|---|---|
| LTX-2.5 | 8-bit | 54 s | Video |
| Z-Image Turbo | 16-bit | 6.4 s | Images |
| FLUX.1 [dev] | 16-bit | 12 s | Images |
| Qwen-Image | 8-bit | 3.5 min | Images |
| FLUX.2 [dev] | 5-bit | 1.8 min | Images |
| Qwen-Image 2.1 | 16-bit | 1.5 min | Images |
| FLUX.3 Klein | 16-bit | 8.9 s | Images |
| Wan2.2 I2V A14B | 8-bit | 31 min | Video |
| FLUX.2 [klein] 9B | 16-bit | 3.5 s | Images |
| Wan2.1 I2V 14B | 8-bit | 31 min | Video |
| Wan 2.1 T2V-14B | 8-bit | 39 min | Video |
| Krea 2 Turbo | 8-bit | 12 s | Images |
| Stable Diffusion 3.5 Large | 16-bit | 48 s | Images |
| Qwen Image 2512 | 8-bit | 3.5 min | Images |
| Wan 2.2 TI2V-5B | 16-bit | 8.6 min | Video |
| Wan 2.2 A14B | 8-bit | 31 min | Video |
| HunyuanVideo 1.5 | 16-bit | 38 min | Video |
Only just too big
These need up to a quarter more memory than the RTX 4090 has. Spilling into system RAM or a lower precision can get them running, slowly.
Questions
What is the best LLM for the RTX 4090?
Of the open models we track, the strongest that run well on the RTX 4090 are Qwen3.8 27B, Muse Glimmer 30B, Qwen3.6 35B A3B. 258 text models fit it in all.
Calculated from published specs, not benchmarked. All machines