Best LLMs for 96 GB of VRAM
| Text models: 119 | Visual models: 1 | Sized for RTX PRO 6000 Blackwell | Strongest first |
Text models for 96 GB
| Model | Best fit | Context | Speed | Strongest at |
|---|---|---|---|---|
| Qwen3.8 Flash-Next | 3-bit | 256K | 188 tok/s | Chat, Coding, Agents |
| MiniMax M3 | 1.6-bit | 64K | 135 tok/s | Docs |
| Qwen3.8 27B | 8-bit | 256K | 41 tok/s | Chat, Coding, Agents, Docs |
| Qwen3.5 397B A17B | 1.6-bit | 256K | 153 tok/s | Agents, Docs |
| MiniMax M2.7 | 2-bit | 40K | 149 tok/s | Coding, Agents |
| Muse Glimmer 30B | 8-bit | 128K | 39 tok/s | Docs |
| Mistral Medium 3.5 128B | 4-bit | 52K | 16 tok/s | Chat, Coding, Agents, Docs |
| Qwen3.5 122B A10B | 5-bit | 256K | 115 tok/s | Chat, Coding, Agents, Docs |
| Nemotron 3 Super 120B A12B | 5-bit | 256K | 82 tok/s | Chat, Coding, Agents, Docs |
| Nemotron 3.5 Bolt 74B A9B | 8-bit | 256K | 77 tok/s | Chat, Coding, Agents, Docs |
| Qwen3.6 35B A3B | 8-bit | 256K | 200 tok/s | Chat, Coding, Agents, Docs |
| Gemma 4 31B | 8-bit | 256K | 35 tok/s | Chat, Coding, Agents, Docs |
| Gemma 4 26B A4B | 8-bit | 256K | 199 tok/s | Chat, Coding, Agents, Docs |
| GLM-4.7 Flash | 8-bit | 198K | 187 tok/s | Chat, Coding, Agents |
| Mistral Small 4 119B | 5-bit | 256K | 171 tok/s | Chat, Docs |
| gpt-oss 120B | MXFP4 | 128K | 197 tok/s | Chat, Coding |
| Qwen3 Coder Next 80B | 8-bit | 256K | 182 tok/s | Coding, Agents |
| Nemotron 3.5 Lightning 30B A3B | 8-bit | 256K | 175 tok/s | Agents, Docs |
| Devstral Small 2 24B | FP8 | 384K | 47 tok/s | Coding |
| LFM2.5 24B A2B | 8-bit | 128K | 244 tok/s | Agents |
| Mistral Small 3.2 24B | 8-bit | 128K | 48 tok/s | Chat, Coding, Agents |
| Llama 3.3 70B | 8-bit | 64K | 16 tok/s | Chat, Coding, Agents, Docs |
| Olmo 3.1 32B | 8-bit | 64K | 35 tok/s | Chat, Coding, Agents, Docs |
| Qwen3 32B | 8-bit | 40K | 34 tok/s | Not rated |
| Qwen3.6 27B | 8-bit | 256K | 41 tok/s | Text and images |
| Qwen3 Coder 30B A3B Instruct | 8-bit | 256K | 184 tok/s | Coding |
| Qwen3.5 27B | 8-bit | 256K | 41 tok/s | Text and images |
| Qwen3.5 35B A3B | 8-bit | 256K | 200 tok/s | Text and images |
| NVIDIA Nemotron 3 Nano 30B A3B BF16 | 8-bit | 256K | 175 tok/s | Not rated |
| GLM 4.5 Air | 6-bit | 32K | 80 tok/s | Not rated |
| Inkling Small | 2-bit | 4K | 168 tok/s | Text and images |
| Qwen3 30B A3B | 8-bit | 40K | 184 tok/s | Not rated |
| MiniMax M2.5 | 2-bit | 40K | 139 tok/s | Not rated |
| Qwen3 30B A3B Instruct 2507 | 8-bit | 256K | 184 tok/s | Not rated |
| Laguna XS.2 | 8-bit | 256K | 202 tok/s | Not rated |
| OTel 2.0 LLM 31B IT | 8-bit | 256K | 35 tok/s | Not rated |
| Laguna M.1 | 2-bit | 40K | 89 tok/s | Not rated |
| Qwen2.5 32B Instruct | 8-bit | 32K | 34 tok/s | Not rated |
| MiniCPM V 4.7 35B A3B | 8-bit | 256K | 200 tok/s | Text and images |
| Laguna XS 2.1 | 8-bit | 256K | 202 tok/s | Not rated |
| Diffusiongemma 26B A4B IT | 8-bit | 256K | 193 tok/s | Text and images |
| Qwen2.5 VL 32B Instruct | 8-bit | 125K | 34 tok/s | Text and images |
| Qwen2.5 Coder 32B Instruct | 8-bit | 32K | 34 tok/s | Coding |
| Laguna S 2.1 | 5-bit | 256K | 130 tok/s | Not rated |
| Qwen3 VL 30B A3B Instruct | 8-bit | 256K | 184 tok/s | Text and images |
| DeepSeek R1 Distill Qwen 32B | 8-bit | 128K | 34 tok/s | Not rated |
| Gemma 3 27B IT | 8-bit | 128K | 42 tok/s | Text and images |
| Qwen3 VL 235B A22B Instruct | 2-bit | 40K | 82 tok/s | Text and images |
| Qwen3 235B A22B | 2-bit | 40K | 82 tok/s | Not rated |
| Ornith 1.5 35B A3B | 8-bit | 256K | 200 tok/s | Not rated |
| Ornith 1.5 122B | 5-bit | 256K | 98 tok/s | Not rated |
| Ornith 1.5 397B | 1.6-bit | 256K | 129 tok/s | Not rated |
| Qwen3 VL 32B Instruct | 8-bit | 224K | 34 tok/s | Text and images |
| Step 3.5 Flash | 2-bit | 52K | 181 tok/s | Not rated |
| Qwen2.5 72B Instruct | 8-bit | 32K | 16 tok/s | Not rated |
| Qwen3 Next 80B A3B Instruct | 8-bit | 256K | 182 tok/s | Not rated |
| Llama 4 Scout 17B 16E Instruct | 6-bit | 36K | 68 tok/s | Text and images |
| MiniMax M2 | 2-bit | 40K | 139 tok/s | Not rated |
| Qwen2.5 VL 72B Instruct | 8-bit | 56K | 16 tok/s | Text and images |
| Qwen1.5 7B | 8-bit | 32K | 143 tok/s | Not rated |
Fits, context and speed on the RTX PRO 6000 Blackwell. A slower card with the same memory holds the same models at lower speed. Ranked by our use-case scores, then by Hugging Face downloads. The top 60 of 119 are shown.
Image and video models
| Model | Best fit | Per output | Strongest at |
|---|---|---|---|
| FLUX.2 [dev] | 16-bit | 31 s | Images |
Machines with 96 GB
Questions
What is the best LLM for 96 GB of VRAM?
Of the open models we track, the strongest that run well on 96 GB are Qwen3.8 Flash-Next, MiniMax M3, Qwen3.8 27B. 119 text models fit in all.
Calculated from published specs, not benchmarked. Every machine and tier