Skip to content

Best LLMs for 12 GB of VRAM

What fits in 12 GB of VRAM
Text models: 246Visual models: 12Sized for RTX 3060 12 GBStrongest first

Text models for 12 GB

Text models that fit 12 GB of VRAM, best first
ModelBest fitContextSpeedStrongest at
Qwen3.5 9B8-bit24K27 tok/sChat, Coding, Agents, Docs
Gemma 4 12B6-bit112K25 tok/sChat, Coding, Docs
Qwen3.5 4B8-bit176K48 tok/sAgents, Docs
Ministral 3 14B4-bit16K30 tok/sCoding, Agents, Docs
Gemma 4 E4B8-bit128K43 tok/sChat, Coding, Docs
gpt-oss 20B3-bit40K119 tok/sAgents
Ministral 3 8B6-bit28K35 tok/sDocs
Gemma 4 E2B8-bit128K78 tok/sChat
Nemotron 3 Nano 4B8-bit256K60 tok/sDocs
Llama 3.1 8B8-bit20K31 tok/sChat, Coding, Agents, Docs
Qwen3 0.6B8-bit40K249 tok/sNot rated
Qwen3 14B3-bit24K35 tok/sNot rated
Qwen3 VL 8B Instruct6-bit24K36 tok/sText and images
Qwen3 8B8-bit16K30 tok/sNot rated
Qwen2.5 7B Instruct8-bit32K33 tok/sNot rated
Qwen2.5 VL 7B Instruct8-bit40K30 tok/sText and images
Unlimited OCR8-bit32K181 tok/sReading documents
Qwen2.5 1.5B Instruct8-bit32K136 tok/sNot rated
Qwen3.5 2B8-bit256K96 tok/sText and images
Qwen3 VL 4B Instruct8-bit40K51 tok/sText and images
Qwen3 1.7B8-bit40K106 tok/sNot rated
Qwen3 4B8-bit40K55 tok/sNot rated
Qwen3 4B Instruct 25078-bit44K55 tok/sNot rated
Qwen2.5 3B Instruct8-bit32K71 tok/sNot rated
DeepSeek OCR8-bit8K181 tok/sReading documents
Qwen2.5 Coder 14B Instruct3-bit20K35 tok/sCoding
Qwen2.5 VL 3B Instruct8-bit125K59 tok/sText and images
Qwen2.5 Coder 7B Instruct8-bit32K33 tok/sCoding
Mistral 7B Instruct V0.28-bit28K32 tok/sNot rated
Chandra OCR 28-bit160K43 tok/sReading documents
Qwen3 VL 2B Instruct8-bit72K102 tok/sText and images
Qwen2 VL 2B Instruct8-bit32K98 tok/sText and images
MiniCPM5 2B8-bit128K93 tok/sNot rated
Gemma 3 4B IT8-bit128K61 tok/sText and images
Llama 3.2 3B Instruct8-bit64K70 tok/sNot rated
LightOnOCR 2 1B8-bit16K197 tok/sText and images
Qwen2.5 14B Instruct3-bit20K35 tok/sNot rated
Qwen3 1.7B Base8-bit32K124 tok/sNot rated
HunyuanOCR8-bit96K184 tok/sText and images
Qwen3 0.6B Base8-bit32K298 tok/sNot rated
Rax 4.58-bit256K96 tok/sText and images
Qwen2 VL 7B Instruct8-bit32K30 tok/sText and images
Ornith 1.5 9B8-bit20K27 tok/sNot rated
Unlimited OCR 28-bit120K180 tok/sReading documents
Granite 4.1 3B8-bit88K64 tok/sNot rated
DeepSeek R1 Distill Qwen 1.5B8-bit128K136 tok/sNot rated
Mistral 7B V0.18-bit32K32 tok/sNot rated
Qwen2.5 Coder 3B Instruct8-bit32K71 tok/sCoding
Meta Llama 3 8B Instruct Abliterated V38-bit8K31 tok/sNot rated
EXAONE 3.5 7.8B Instruct8-bit20K31 tok/sNot rated
Saiga Llama3 8B8-bit8K31 tok/sNot rated
Qwen2.5 Coder 1.5B Instruct8-bit32K136 tok/sCoding
DeepSeek R1 0528 Qwen3 8B8-bit16K30 tok/sNot rated
Gemma 3 12B IT5-bit28K30 tok/sText and images
Phi 3.5 Mini Instruct8-bit16K60 tok/sNot rated
Qwen3 4B Thinking 25078-bit44K55 tok/sNot rated
Phi 4 Mini Instruct8-bit52K58 tok/sNot rated
Dots.ocr8-bit128K79 tok/sReading documents
DeepSeek R1 Distill Qwen 7B8-bit52K33 tok/sNot rated
Phi 3 Mini 4K Instruct8-bit4K60 tok/sNot rated

Fits, context and speed on the RTX 3060 12 GB. A slower card with the same memory holds the same models at lower speed. Ranked by our use-case scores, then by Hugging Face downloads. The top 60 of 246 are shown.

Image and video models

Image and video models that fit 12 GB
ModelBest fitPer outputStrongest at
Stable Diffusion XL 1.016-bit25 sImages
Z-Image Turbo8-bit36 sImages
FLUX.1 [dev]6-bit82 sImages
Qwen-Image 2.18-bit9.7 minImages
FLUX.3 Klein16-bit52 sImages
FLUX.2 [klein] 9B8-bit17 sImages
Krea 2 Turbo5-bit81 sImages
Stable Diffusion 3.5 Large8-bit5.1 minImages
MOSS-Image16-bit6.3 minImages
Wan 2.2 TI2V-5B16-bit54 minVideo
MOSS Image Base16-bit6.3 minImages
HunyuanVideo 1.56-bit4.7 hVideo

Machines with 12 GB

Questions

What is the best LLM for 12 GB of VRAM?

Of the open models we track, the strongest that run well on 12 GB are Qwen3.5 9B, Gemma 4 12B, Qwen3.5 4B. 246 text models fit in all.

Calculated from published specs, not benchmarked. Every machine and tier