VRAM calculator
Choose your hardware and an open model to see the memory it needs, how fast it runs, and what to change if it doesn’t fit.
Hardware
RTX 509032.0 GB usable+56.0 GB system RAM
Model
390 of 479 fit · 44 more with RAM spill- Fits
- Tight
- Spills
- Too big
- Spills: part of the model sits in system RAM. Mixture-of-experts models (A3B means 3B active) stay quick, since each token reads only its active experts. Slow means under reading speed, usually a dense model. Too big means it won’t fit even with RAM.
ollama run qwen3.8:27bOllamaGGUFMLXHugging FaceApache 2.0
Quality
Fewer bits per weight means a smaller model. 4-bit is the usual pick. Below that, small models lose quality fast; big ones hold up better.Context
How much text the model sees at once: your messages, its replies, pasted documents. It needs memory on top of the model.Chats at onceEach conversation keeps its own context, so memory multiplies.
Running cost
Power at the wall while it works, at your rate. Cloud is the cheapest on-demand GPU that runs the same job, at RunPod’s on-demand prices as of Oct 21, 2026. Add the hardware’s MSRP to see when owning pays off; the PDF report carries it into a budget table.Hours a day
Power at home
$9.30/mo
425 W
Cloud
$58/mo
2× RTX 2000 Ada at $0.48/h
Result
- GPU memory
- 29.9/ 32.0 GB
- System RAM
- 0.0/ 56.0 GB
- Context
- 208K
- Speed
- 67tok/s
- First reply
- 0.5s
- Model
- 15.7 GB
- Context
- 13.1 GB
- Overhead
- 1.1 GB
- OS
- 0.0 GB
- Free
- 2.1 GB
- In RAM
- 0.0 GB
Suggested changes
- 6-bit weightsBetter answers, 120K context, 52 tok/s
How the math works
- The model
- parameters × bits ÷ 8
- Qwen3.8 27B at 4-bit: 27.8B × 4.85 ÷ 8 = 15.7 GB.
- Context
- tokens × layers × KV heads × head size × 2 × bytes
- Llama 3.1 8B: 128 KB per token, 16.0 GB at 128K. Newer models cache far less, so we read each one’s config.
- Overhead
- driver + scratch buffers
- About 0.6 to 1.5 GB.
- Speed
- bandwidth ÷ bytes read per token
- Mixture-of-experts models read only their active experts, so they run faster than their size suggests.
- First reply
- 2 × active weights × prompt tokens ÷ TFLOPS
- Reading the prompt is compute, not bandwidth. A Mac can write as fast as a GeForce card and still read a long document several times slower.
- Images and video
- steps × weights × tokens × 2 ÷ TFLOPS
- Memory is the model plus one block’s working set. Time grows faster than resolution, since attention compares every patch with every other.
- Running cost
- watts × hours × your rate
- Set against the cheapest on-demand cloud GPU that runs the same job.
How much memory each model needs
Weights at each level, plus 32K tokens of context.
| Model | Parameters | 4-bit | 8-bit | As released | + 32K context |
|---|---|---|---|---|---|
| Nemotron 3 Nano 4B | 4B | 2.2 GB | 3.9 GB | BF167.4 GB | 592 MB |
| Qwen3.5 4B | 4.7B | 2.6 GB | 4.6 GB | BF168.7 GB | 1.0 GB |
| Gemma 4 E2B | 5.1B (2.3B active) | 2.9 GB | 5.1 GB | BF169.5 GB | 198 MB |
| Gemma 4 E4B | 8B (4.5B active) | 4.5 GB | 7.9 GB | BF1614.9 GB | 532 MB |
| Llama 3.1 8B | 8B | 4.5 GB | 7.9 GB | BF1615.0 GB | 4.0 GB |
| Ministral 3 8B | 8.9B | 5.0 GB | 8.8 GB | FP89.7 GB | 4.3 GB |
| Qwen3.5 9B | 9.7B | 5.4 GB | 9.5 GB | BF1618.0 GB | 1.0 GB |
| Gemma 4 12B | 12B | 6.8 GB | 11.8 GB | BF1622.3 GB | 576 MB |
| Ministral 3 14B | 13.9B | 7.9 GB | 13.8 GB | FP814.6 GB | 5.0 GB |
| gpt-oss 20B | 20.9B (3.6B active) | 11.8 GB | n/a | MXFP411.3 GB | 771 MB |
| LFM2.5 24B A2B | 24B (2B active) | 13.5 GB | 23.7 GB | BF1644.6 GB | 640 MB |
| Mistral Small 3.2 24B | 24B | 13.6 GB | 23.8 GB | BF1644.7 GB | 5.0 GB |
| Devstral Small 2 24B | 24B | 13.6 GB | 23.8 GB | FP824.0 GB | 5.0 GB |
| Gemma 4 26B A4B | 25.8B (3.8B active) | 14.6 GB | 25.5 GB | BF1648.1 GB | 520 MB |
| Qwen3.8 27B | 27.8B | 15.7 GB | 27.5 GB | BF1651.7 GB | 2.1 GB |
| Muse Glimmer 30B | 29.8B | 16.8 GB | 29.5 GB | BF1655.5 GB | 494 MB |
| GLM-4.7 Flash | 31.2B (3B active) | 17.6 GB | 30.9 GB | BF1658.2 GB | 1.7 GB |
| Gemma 4 31B | 31.3B | 17.7 GB | 30.9 GB | BF1658.2 GB | 2.0 GB |
| Nemotron 3.5 Lightning 30B A3B | 31.6B (3B active) | 17.8 GB | 31.2 GB | BF1658.8 GB | 239 MB |
| Olmo 3.1 32B | 32.2B | 18.2 GB | 31.9 GB | BF1660.0 GB | 2.8 GB |
| Qwen3.6 35B A3B | 36B (3B active) | 20.3 GB | 35.6 GB | BF1667.0 GB | 701 MB |
| Llama 3.3 70B | 70.5B | 39.8 GB | 69.8 GB | BF16131 GB | 10.0 GB |
| Nemotron 3.5 Bolt 74B A9B | 74.9B (9B active) | 42.3 GB | 74.1 GB | BF16139 GB | 418 MB |
| Qwen3 Coder Next 80B | 79.7B (3B active) | 45.0 GB | 78.8 GB | BF16148 GB | 842 MB |
| gpt-oss 120B | 117B (5.1B active) | 66.0 GB | n/a | MXFP459.0 GB | 1.1 GB |
| Mistral Small 4 119B | 119B (6.5B active) | 67.4 GB | n/a | FP8113 GB | 720 MB |
| Nemotron 3 Super 120B A12B | 124B (12B active) | 69.8 GB | 122 GB | BF16230 GB | 418 MB |
| Qwen3.5 122B A10B | 125B (10B active) | 70.6 GB | 124 GB | BF16233 GB | 915 MB |
| Mistral Medium 3.5 128B | 128B | 72.1 GB | n/a | FP8124 GB | 11.0 GB |
| Qwen3.8 Flash-Next | 180B (6B active) | 102 GB | 178 GB | BF16335 GB | 878 MB |
| MiniMax M2.7 | 229B (10B active) | 129 GB | n/a | FP8214 GB | 7.8 GB |
| DeepSeek V4 Flash | 284B (13B active) | 160 GB | n/a | FP4 + FP8155 GB | 199 MB |
| Hy3 | 295B (21B active) | 167 GB | 292 GB | BF16549 GB | 10.0 GB |
| MiMo V2.6 Flash | 309B (15B active) | 174 GB | n/a | FP4 + FP8161 GB | 744 MB |
| GLM-5.3 Flash | 321B (18B active) | 181 GB | n/a | FP8306 GB | 537 MB |
| Qwen3.5 397B A17B | 403B (17B active) | 228 GB | 399 GB | BF16751 GB | 1.1 GB |
| MiniMax M3 | 427B (23B active) | 241 GB | 423 GB | BF16795 GB | 3.8 GB |
| Nemotron 3 Ultra 550B A55B | 561B (55B active) | 316 GB | 555 GB | BF161,044 GB | 773 MB |
| DeepSeek V3.2 | 671B (37B active) | 379 GB | n/a | FP8642 GB | 2.4 GB |
| GLM-5.3 | 753B (40B active) | 425 GB | n/a | FP8704 GB | 3.0 GB |
| DeepSeek V4.1 Flash | 763B (16B active) | 431 GB | n/a | FP4 + FP8475 GB | 89 MB |
| Muse Spark Open 816B A42B | 822B (42B active) | 464 GB | 813 GB | BF161,531 GB | 2.4 GB |
| MiMo V2.6 Pro | 1T (42B active) | 578 GB | n/a | FP4 + FP8527 GB | 1.6 GB |
| Kimi K2.6 | 1T (32B active) | 580 GB | n/a | INT4554 GB | 2.1 GB |
| DeepSeek V4 Pro | 1.6T (49B active) | 903 GB | n/a | FP4 + FP8805 GB | 285 MB |
| Qwen3.8 2.4T A95B | 2.4T (95B active) | 1,381 GB | 2,421 GB | BF164,556 GB | 3.4 GB |
| Kimi K3 | 2.8T (104B active) | 1,570 GB | n/a | INT41,454 GB | 1.3 GB |
395 more models from Hugging Face
- A.X 4.0 Light
- Academic Ds 9B
- Agents A1
- AI21 Jamba Reasoning 3B
- AIN
- ALLaM 7B Instruct Preview
- ALMA 13B R
- ALMA 7B R
- Apertus 8B Instruct 2509
- Beam 1.0 501B A23B
- Chandra OCR 2
- CodeLlama 13B Instruct HF
- CodeLlama 34B Instruct HF
- CodeLlama 7B HF
- Command A Plus 05 2026 Bf16
- D1 2B
- D1 3B
- D1 Omni 600M
- Deepseek Coder 1.3B Instruct
- Deepseek Coder 33B Instruct
- Deepseek Coder 6.7B Base
- Deepseek Coder 7B Instruct V1.5
- DeepSeek Coder V2 Lite Instruct
- Deepseek MOE 16B Chat
- DeepSeek OCR
- DeepSeek OCR 2
- DeepSeek R1
- DeepSeek R1 0528
- DeepSeek R1 0528 Qwen3 8B
- DeepSeek R1 Distill Llama 70B
- DeepSeek R1 Distill Llama 8B
- DeepSeek R1 Distill Qwen 1.5B
- DeepSeek R1 Distill Qwen 14B
- DeepSeek R1 Distill Qwen 32B
- DeepSeek R1 Distill Qwen 7B
- DeepSeek V2 Chat
- DeepSeek V2 Lite
- DeepSeek V3
- DeepSeek V3 0324
- DiegoCropper
- Diffusiongemma 26B A4B IT
- Dots.mocr
- Dots.ocr
- Dream V0 Instruct 7B
- EuroLLM 1.7B Instruct
- EXAONE 3.5 2.4B Instruct
- EXAONE 3.5 32B Instruct
- EXAONE 3.5 7.8B Instruct
- EXAONE 4.0 1.2B
- EXAONE 4.0 32B
- EXAONE 4.5 33B
- Falcon 40B
- Falcon 7B
- Falcon 7B Instruct Sharded
- Falcon3 7B Instruct
- FastVLM 0.5B
- Flint
- Foundation Sec 8B Instruct
- Gemma 2 27B IT
- Gemma 2 2B IT
- Gemma 2 9B IT
- Gemma 3 12B IT
- Gemma 3 1B IT
- Gemma 3 27B IT
- Gemma 3 4B IT
- Gemma SEA LION V3 9B IT
- Ghost 8B Beta 1608
- Glm 4 9B
- GLM 4 9B 0414
- GLM 4.1V 9B Thinking
- GLM 4.5
- GLM 4.5 Air
- GLM 4.5V
- GLM 4.6
- GLM 4.6V Flash
- GLM 4.7
- GLM 5
- GLM 5.1
- GLM 5.2
- GLM 5.3 BF16
- GLM 5.3 Flash BF16
- GLM Z1 32B 0414
- Gnani Evon V3.3 30B A3B
- GOT OCR2 0
- GPT Neox 20B
- Gpt2 Large
- Gpt2 XL
- Granite 3.0 8B Instruct
- Granite 3.1 1B A400m Instruct
- Granite 3.1 2B Instruct
- Granite 3.1 3B A800m Instruct
- Granite 3.1 8B Instruct
- Granite 3.2 8B Instruct
- Granite 3.3 2B Instruct
- Granite 3.3 8B Instruct
- Granite 3B Code Base 2K
- Granite 4.0 1B
- Granite 4.0 H Small
- Granite 4.0 Micro
- Granite 4.1 30B
- Granite 4.1 3B
- Granite 4.1 8B
- Granite 4.1 8B Base
- Granite 4.2 30B
- Granite 4.2 3B
- Granite 4.2 8B
- Granite Vision 4.1 4B
- H2o Danube3 500M Chat
- H2ovl Mississippi 2B
- HarmBench Llama 2 13B Cls
- HunyuanOCR
- Hy3 Preview
- Hy4 Preview
- HyperCLOVAX SEED Think 14B
- Idefics3 8B Llama3
- Infinity Parser2 Flash
- Inkling
- Inkling Small
- Intern S1
- Internlm2 5 7B Chat
- Internlm2 Chat 20B
- Internlm2 Chat 20B ExPO
- Internlm2 Chat 7B
- Internlm2 Chat 7B ExPO
- Internlm3 8B Instruct
- InternVL2 5 4B MPO
- InternVL3 5 1B HF
- InternVL3 5 4B HF
- InternVL3 5 8B
- InternVL3 78B
- InternVL3 8B
- K EXAONE 2.0 750B A37B
- K2 Horizon 0.9B
- K2 Horizon 3.7B
- K2 Horizon 375B A23B
- K2 Horizon 7B
- K2 Horizon MoVA 36B A4B
- Kanana 1.5 8B Instruct 2505
- Kanana 2 3B Instruct
- Kimi K2 Instruct
- Kimi Linear 48B A3B Instruct
- Kimi VL A3B Instruct
- Kimi VL A3B Thinking 2506
- KONI Llama3.1 8B Instruct 20241024
- Laguna M.1
- Laguna S 2.1
- Laguna XS 2.1
- Laguna XS.2
- Le Triomphant ECE TW3
- LFM2 1.2B
- LFM2 8B A1B
- LFM2.5 1.2B Instruct
- LFM2.5 2.6B
- LFM2.5 8B A1B
- LFM2.5 VL 1.6B
- LFM2.5 VL 3B
- LFM3 1.7B
- LFM3 VL 2B
- Lift
- LightOnOCR 2 1B
- LightOnOCR 3 0.8B
- LightOnOCR 3 4B
- LLaDA 8B Instruct
- LLaDA2.0 Mini
- LLaDA2.1 Mini
- Llama 3 8B Instruct Gradient 1048K
- Llama 3 SauerkrautLM 8B Instruct
- Llama 3.1 70B
- Llama 3.1 8B Instruct
- Llama 3.1 SauerkrautLM 8B Instruct
- Llama 3.2 11B Vision Instruct
- Llama 3.2 1B Instruct
- Llama 3.2 3B Instruct
- Llama 4 Scout 17B 16E Instruct
- Llama 7B
- Llama Guard 4 12B
- Llama xLAM 2 8B Fc R
- Llama2 13B Chat HF
- Llama3 ChatQA 1.5 8B
- Llama3 Chinese 8B Instruct
- Llava 1.5 13B HF
- Llava V1.6 Vicuna 7B
- Llm Jp 4 8B Base
- Llm Jp 4.1 8B Thinking
- Malaysian TTS 0.6B V1
- Medgemma 1.5 4B IT
- Medgemma 27B Text IT
- Mellum2.1 12B A2.5B Thinking
- Meta Llama 3 8B Instruct Abliterated V3
- Midm 2.0 Base Instruct
- Midm 2.0 Mini Instruct
- MiMo 7B RL
- MiMo V2 Flash
- MiMo V2.5
- MinerU2.5 2509 1.2B
- MinerU2.5 Pro 2604 1.2B
- MinerU2.5 Pro 2605 1.2B
- MiniCPM Llama3 V 2 5
- MiniCPM OCR 1.0
- MiniCPM V 4
- MiniCPM V 4 5
- MiniCPM V 4.6
- MiniCPM V 4.6 Thinking
- MiniCPM V 4.7 1B
- MiniCPM V 4.7 35B A3B
- MiniCPM5 1B
- MiniCPM5 2B
- MiniMax M2
- MiniMax M2.1
- MiniMax M2.5
- Ministral 3B Instruct
- Mistral 7B Instruct V0.2
- Mistral 7B V0.1
- Molmo2 O 7B
- MonkeyOCRv2 B Parsing
- Moonlight 16B A3B Instruct
- Nanbeige4.2 3B
- Nanbeige4.3 33B
- Nemotron Labs Diffusion 3B
- Neural Chat 7B V3 3
- NeuralDaredevil 8B Abliterated
- NeuralMonarch 7B
- North Micro Vision Instruct
- NVIDIA Nemotron 3 Nano 30B A3B BF16
- NVIDIA Nemotron 3.5 Super VL 120B A12B BF16
- NVIDIA Nemotron Labs Dory1.0 4B BF16
- OLMo 1B HF
- OLMo 2 0425 1B
- OLMo 2 1124 13B Instruct
- OLMo 2 1124 7B Instruct
- Olmo 3 1025 7B
- Olmo 3 1125 32B
- Olmo 3 32B Think
- Olmo 3 7B Instruct
- Olmo 3 7B Instruct DPO
- Olmo 3 7B Instruct SFT
- Olmo 3 7B Think
- Olmo 3.1 32B Instruct DPO
- OLMoE 1B 7B 0125 Instruct
- OLMoE 1B 7B 0924
- OneRec 1.7B
- Ornith 1.0 397B
- Ornith 1.5 122B
- Ornith 1.5 35B A3B
- Ornith 1.5 397B
- Ornith 1.5 9B
- OTel 2.0 LLM 31B IT
- Ouro 1.4B
- Ouro 1.4B Thinking
- Ouro 2.6B
- PaddleOCR VL 1.5
- PaddleOCR VL 1.6
- Phi 1 5
- Phi 2
- Phi 3 Medium 128K Instruct
- Phi 3 Medium 4K Instruct
- Phi 3 Mini 128K Instruct
- Phi 3 Mini 4K Instruct
- Phi 3.5 Mini Instruct
- Phi 3.5 MOE Instruct
- Phi 4
- Phi 4 Mini Instruct
- Phi 4 Mini Reasoning
- Phi 4 Reasoning
- Pixtral 12B
- PowerMoE 3B
- Pythia 1.4B Deduped
- Qari OCR V0.3 VL 2B Instruct
- Qianfan OCR
- Qwen 7B Chat
- Qwen Image Bench
- Qwen1.5 0.5B Chat
- Qwen1.5 1.8B Chat
- Qwen1.5 14B Chat
- Qwen1.5 32B Chat
- Qwen1.5 4B Chat
- Qwen1.5 72B
- Qwen1.5 7B
- Qwen1.5 MOE A2.7B
- Qwen2 1.5B
- Qwen2 57B A14B Instruct
- Qwen2 72B Instruct
- Qwen2 7B Instruct
- Qwen2 VL 2B Instruct
- Qwen2 VL 72B Instruct
- Qwen2 VL 7B Instruct
- Qwen2.5 1.5B Instruct
- Qwen2.5 14B Instruct
- Qwen2.5 14B Instruct 1M
- Qwen2.5 32B Instruct
- Qwen2.5 3B Instruct
- Qwen2.5 72B Instruct
- Qwen2.5 7B Instruct
- Qwen2.5 7B Instruct 1M
- Qwen2.5 Coder 1.5B Instruct
- Qwen2.5 Coder 14B Instruct
- Qwen2.5 Coder 32B Instruct
- Qwen2.5 Coder 3B Instruct
- Qwen2.5 Coder 7B Instruct
- Qwen2.5 Math 1.5B Instruct
- Qwen2.5 Math 7B Instruct
- Qwen2.5 VL 32B Instruct
- Qwen2.5 VL 3B Instruct
- Qwen2.5 VL 72B Instruct
- Qwen2.5 VL 7B Instruct
- Qwen3 0.6B
- Qwen3 0.6B Base
- Qwen3 1.7B
- Qwen3 1.7B Base
- Qwen3 14B
- Qwen3 235B A22B
- Qwen3 235B A22B Instruct 2507
- Qwen3 30B A3B
- Qwen3 30B A3B Instruct 2507
- Qwen3 30B A3B Thinking 2507
- Qwen3 32B
- Qwen3 4B
- Qwen3 4B DFlash B16
- Qwen3 4B Instruct 2507
- Qwen3 4B Thinking 2507
- Qwen3 8B
- Qwen3 Coder 30B A3B Instruct
- Qwen3 Coder 480B A35B Instruct
- Qwen3 Next 80B A3B Instruct
- Qwen3 Next 80B A3B Thinking
- Qwen3 VL 235B A22B Instruct
- Qwen3 VL 235B A22B Thinking
- Qwen3 VL 2B Instruct
- Qwen3 VL 2B Thinking
- Qwen3 VL 30B A3B Instruct
- Qwen3 VL 32B Instruct
- Qwen3 VL 4B Instruct
- Qwen3 VL 4B Thinking
- Qwen3 VL 8B Instruct
- Qwen3 VL 8B Thinking
- Qwen3.5 0.8B
- Qwen3.5 27B
- Qwen3.5 2B
- Qwen3.5 35B A3B
- Qwen3.6 27B
- Qwen3Guard Gen 0.6B
- Qwen3Guard Gen 4B
- Qwen3Guard Gen 8B
- QwQ 32B
- Rax 4.5
- Sage A1.5
- Saiga Llama3 8B
- Salamandra 7B Instruct
- Sarashina2.2 3B Instruct V0.1
- Seed OSS 36B Instruct
- SmolLM 1.7B
- SmolLM2 1.7B
- SmolLM3 3B
- SmolVLM2 2.2B Instruct
- SOLAR 10.7B Instruct V1.0
- Solar Open 100B
- Spark X2.5 4B
- Spark X2.6 4B
- Stablelm 3B 4e1t
- Stablelm Zephyr 3B
- Starcoder2 15B
- Starcoder2 3B
- Starcoder2 7B
- Step 3.5 Flash
- Step 3.7 Flash
- Surya OCR 2
- Surya OCR 2.5
- Suzume Llama 3 8B Multilingual Orpo Borda Top25
- SWE Gym OpenHands 7B Agent
- TeleOCR
- Tess 2.0 Llama 3 8B
- Teutonic I Reliquary RL
- TinyLlama 1.1B Chat V1.0
- TinyLlama 1.1B Intermediate Step 1431K 3T
- TinyLlama 1.1B Step 50K 105B
- Tongyi DeepResearch 30B A3B
- TW3 JRGL V2
- Typhoon2.5 Qwen3 4B
- UI TARS 1.5 7B
- Unlimited OCR
- Unlimited OCR 2
- VLX Seek 1.5 10B
- WeVisOne VL 2B
- WeVisOne VL 4B
- WizardLM 2 8x22B
- XCurOS1.2 8B VLBF16 Instruct
- YanoljaNEXT EEVE Instruct 10.8B
- Yi 1.5 34B Chat
- Yi 1.5 6B
- Yi 1.5 9B Chat
- Yi 34B 200K
- Yi 34B Chat
- Yi 6B 200K
- Yi 6B Chat
- Yi Coder 9B
By machine and VRAM
- What can my machine run?
- Best LLMs for 8 GB
- Best LLMs for 12 GB
- Best LLMs for 16 GB
- Best LLMs for 24 GB
- Best LLMs for 32 GB
- Best LLMs for 48 GB
- Best LLMs for 96 GB
Image and video models
Questions people ask
How much VRAM do I need to run an LLM?
About 0.6 GB per billion parameters at 4-bit, plus context. An 8B model fits an 8 GB card, 27B to 32B wants 24 GB, and a dense 70B needs about 48 GB.
Which quantization should I use?
4-bit (Q4_K_M) unless you have room to spare. Below 4-bit, small models degrade fast; models over 100B hold up.
How much context do I need?
8K covers a normal chat, 32K a long one, 128K about a novel. Coding agents want 64K or more.
Why can my Mac only use part of its memory?
macOS lets the GPU use about 75% of memory. Raise it with sudo sysctl iogpu.wired_limit_mb=<MB>; it resets on reboot.
Can I run a model bigger than my GPU?
Yes, by spilling part of it into system RAM. Dense models slow down badly; mixture-of-experts models hold up.
Do two GPUs double my memory?
For fitting a model, yes. For speed, mostly no: with the default layer split the cards take turns.
Why does a Mac take longer to start answering?
Writing tokens depends on memory bandwidth, where Macs are strong. Reading the prompt depends on raw compute, where NVIDIA cards are several times faster, so long documents and coding agents wait longer on a Mac.
Ollama, LM Studio or vLLM?
Ollama or LM Studio for one person: easy, and they can spill into system RAM. vLLM for a team: it serves many chats at once but needs the whole model on the GPU.
How much VRAM do image and video models need?
FLUX.1 dev wants about 14 GB at FP8 and runs on a 12 GB card as a 6-bit GGUF. 14B video models want about 21 GB at FP8, and a 5-second 720p clip takes around half an hour on an RTX 4090. Distilled models like LTX-2.5 take about a minute.
How accurate is this?
Memory lands within about a gigabyte. Speed is a rough guide, plus or minus 25%.
Where does the model data come from?
Each model’s config.json on Hugging Face, with active-parameter counts from the model cards.
Need a machine that runs it?
Tell us the model and how many people use it. We’ll quote the hardware.