Skip to content

Best LLMs for 8 GB of VRAM

What fits in 8 GB of VRAM
Text models: 234Visual models: 6Sized for RTX 4060Strongest first

Text models for 8 GB

Text models that fit 8 GB of VRAM, best first
ModelBest fitContextSpeedStrongest at
Qwen3.5 9B4-bit32K35 tok/sChat, Coding, Agents, Docs
Gemma 4 12B3-bit96K31 tok/sChat, Coding, Docs
Qwen3.5 4B8-bit60K37 tok/sAgents, Docs
Gemma 4 E4B6-bit28K42 tok/sChat, Coding, Docs
Ministral 3 8B3-bit20K44 tok/sDocs
Gemma 4 E2B8-bit128K60 tok/sChat
Nemotron 3 Nano 4B8-bit176K46 tok/sDocs
Llama 3.1 8B4-bit16K40 tok/sChat, Coding, Agents, Docs
Qwen3 0.6B8-bit40K199 tok/sNot rated
Qwen3 VL 8B Instruct3-bit16K45 tok/sText and images
Qwen3 8B3-bit20K49 tok/sNot rated
Qwen2.5 7B Instruct6-bit16K32 tok/sNot rated
Qwen2.5 VL 7B Instruct5-bit20K34 tok/sText and images
Unlimited OCR8-bit32K141 tok/sReading documents
Qwen2.5 1.5B Instruct8-bit32K106 tok/sNot rated
Qwen3.5 2B8-bit256K74 tok/sText and images
Qwen3 VL 4B Instruct8-bit16K39 tok/sText and images
Qwen3 1.7B8-bit40K82 tok/sNot rated
Qwen3.5 0.8B8-bit256K176 tok/sText and images
Qwen3 4B8-bit16K43 tok/sNot rated
Qwen3 4B Instruct 25078-bit16K43 tok/sNot rated
Qwen2.5 3B Instruct8-bit32K55 tok/sNot rated
DeepSeek OCR8-bit8K141 tok/sReading documents
Qwen2.5 VL 3B Instruct8-bit80K45 tok/sText and images
Qwen2.5 Coder 7B Instruct6-bit16K32 tok/sCoding
Mistral 7B Instruct V0.25-bit16K36 tok/sNot rated
Chandra OCR 28-bit40K32 tok/sReading documents
Llama 3.2 1B Instruct8-bit128K136 tok/sNot rated
Qwen3 VL 2B Instruct8-bit40K79 tok/sText and images
Qwen2 VL 2B Instruct8-bit32K76 tok/sText and images
MiniCPM5 2B8-bit104K72 tok/sNot rated
Gemma 3 4B IT8-bit112K47 tok/sText and images
Llama 3.2 3B Instruct8-bit32K53 tok/sNot rated
LightOnOCR 2 1B8-bit16K155 tok/sText and images
Qwen3 1.7B Base8-bit32K96 tok/sNot rated
HunyuanOCR8-bit60K143 tok/sText and images
Qwen3 0.6B Base8-bit32K240 tok/sNot rated
Rax 4.58-bit256K74 tok/sText and images
Qwen2 VL 7B Instruct5-bit20K34 tok/sText and images
Ornith 1.5 9B4-bit32K35 tok/sNot rated
Unlimited OCR 28-bit60K140 tok/sReading documents
Granite 4.1 3B8-bit44K50 tok/sNot rated
DeepSeek R1 Distill Qwen 1.5B8-bit128K106 tok/sNot rated
Mistral 7B V0.16-bit32K32 tok/sNot rated
Qwen2.5 Coder 3B Instruct8-bit32K55 tok/sCoding
Meta Llama 3 8B Instruct Abliterated V35-bit8K34 tok/sNot rated
EXAONE 3.5 7.8B Instruct4-bit16K41 tok/sNot rated
Saiga Llama3 8B5-bit8K34 tok/sNot rated
MiniCPM V 4.7 1B8-bit256K124 tok/sText and images
Qwen2.5 Coder 1.5B Instruct8-bit32K106 tok/sCoding
DeepSeek R1 0528 Qwen3 8B3-bit20K49 tok/sNot rated
Qwen3 4B Thinking 25078-bit16K43 tok/sNot rated
Phi 4 Mini Instruct8-bit20K45 tok/sNot rated
Dots.ocr8-bit128K61 tok/sReading documents
DeepSeek R1 Distill Qwen 7B6-bit16K32 tok/sNot rated
Phi 3 Mini 4K Instruct8-bit4K46 tok/sNot rated
DeepSeek OCR 28-bit8K135 tok/sReading documents
SmolLM3 3B8-bit52K55 tok/sNot rated
Qwen2 7B Instruct6-bit16K32 tok/sNot rated
Apertus 8B Instruct 25094-bit16K40 tok/sNot rated

Fits, context and speed on the RTX 4060. A slower card with the same memory holds the same models at lower speed. Ranked by our use-case scores, then by Hugging Face downloads. The top 60 of 234 are shown.

Image and video models

Image and video models that fit 8 GB
ModelBest fitPer outputStrongest at
Stable Diffusion XL 1.016-bit21 sImages
Z-Image Turbo8-bit30 sImages
FLUX.1 [dev]4-bit69 sImages
FLUX.3 Klein8-bit44 sImages
Stable Diffusion 3.5 Large5-bit4.9 minImages
Wan 2.2 TI2V-5B8-bit46 minVideo

Machines with 8 GB

Questions

What is the best LLM for 8 GB of VRAM?

Of the open models we track, the strongest that run well on 8 GB are Qwen3.5 9B, Gemma 4 12B, Qwen3.5 4B. 234 text models fit in all.

Calculated from published specs, not benchmarked. Every machine and tier