Skip to content

What can the RTX 4060 run?

What fits on the RTX 4060
Text models: 234Visual models: 6Strongest first
Memory
8 GB
Bandwidth
272 GB/s
Maker
NVIDIA

Open the calculator on this machineEvery model for 8 GB of VRAM

Text models that run on it

Text models that run on the RTX 4060, best first
ModelBest fitContextSpeedStrongest at
Qwen3.5 9B4-bit32K35 tok/sChat, Coding, Agents, Docs
Gemma 4 12B3-bit96K31 tok/sChat, Coding, Docs
Qwen3.5 4B8-bit60K37 tok/sAgents, Docs
Gemma 4 E4B6-bit28K42 tok/sChat, Coding, Docs
Ministral 3 8B3-bit20K44 tok/sDocs
Gemma 4 E2B8-bit128K60 tok/sChat
Nemotron 3 Nano 4B8-bit176K46 tok/sDocs
Llama 3.1 8B4-bit16K40 tok/sChat, Coding, Agents, Docs
Qwen3 0.6B8-bit40K199 tok/sNot rated
Qwen3 VL 8B Instruct3-bit16K45 tok/sText and images
Qwen3 8B3-bit20K49 tok/sNot rated
Qwen2.5 7B Instruct6-bit16K32 tok/sNot rated
Qwen2.5 VL 7B Instruct5-bit20K34 tok/sText and images
Unlimited OCR8-bit32K141 tok/sReading documents
Qwen2.5 1.5B Instruct8-bit32K106 tok/sNot rated
Qwen3.5 2B8-bit256K74 tok/sText and images
Qwen3 VL 4B Instruct8-bit16K39 tok/sText and images
Qwen3 1.7B8-bit40K82 tok/sNot rated
Qwen3.5 0.8B8-bit256K176 tok/sText and images
Qwen3 4B8-bit16K43 tok/sNot rated
Qwen3 4B Instruct 25078-bit16K43 tok/sNot rated
Qwen2.5 3B Instruct8-bit32K55 tok/sNot rated
DeepSeek OCR8-bit8K141 tok/sReading documents
Qwen2.5 VL 3B Instruct8-bit80K45 tok/sText and images
Qwen2.5 Coder 7B Instruct6-bit16K32 tok/sCoding
Mistral 7B Instruct V0.25-bit16K36 tok/sNot rated
Chandra OCR 28-bit40K32 tok/sReading documents
Llama 3.2 1B Instruct8-bit128K136 tok/sNot rated
Qwen3 VL 2B Instruct8-bit40K79 tok/sText and images
Qwen2 VL 2B Instruct8-bit32K76 tok/sText and images
MiniCPM5 2B8-bit104K72 tok/sNot rated
Gemma 3 4B IT8-bit112K47 tok/sText and images
Llama 3.2 3B Instruct8-bit32K53 tok/sNot rated
LightOnOCR 2 1B8-bit16K155 tok/sText and images
Qwen3 1.7B Base8-bit32K96 tok/sNot rated
HunyuanOCR8-bit60K143 tok/sText and images
Qwen3 0.6B Base8-bit32K240 tok/sNot rated
Rax 4.58-bit256K74 tok/sText and images
Qwen2 VL 7B Instruct5-bit20K34 tok/sText and images
Ornith 1.5 9B4-bit32K35 tok/sNot rated
Unlimited OCR 28-bit60K140 tok/sReading documents
Granite 4.1 3B8-bit44K50 tok/sNot rated
DeepSeek R1 Distill Qwen 1.5B8-bit128K106 tok/sNot rated
Mistral 7B V0.16-bit32K32 tok/sNot rated
Qwen2.5 Coder 3B Instruct8-bit32K55 tok/sCoding
Meta Llama 3 8B Instruct Abliterated V35-bit8K34 tok/sNot rated
EXAONE 3.5 7.8B Instruct4-bit16K41 tok/sNot rated
Saiga Llama3 8B5-bit8K34 tok/sNot rated
MiniCPM V 4.7 1B8-bit256K124 tok/sText and images
Qwen2.5 Coder 1.5B Instruct8-bit32K106 tok/sCoding
DeepSeek R1 0528 Qwen3 8B3-bit20K49 tok/sNot rated
Qwen3 4B Thinking 25078-bit16K43 tok/sNot rated
Phi 4 Mini Instruct8-bit20K45 tok/sNot rated
Dots.ocr8-bit128K61 tok/sReading documents
DeepSeek R1 Distill Qwen 7B6-bit16K32 tok/sNot rated
Phi 3 Mini 4K Instruct8-bit4K46 tok/sNot rated
DeepSeek OCR 28-bit8K135 tok/sReading documents
SmolLM3 3B8-bit52K55 tok/sNot rated
Qwen2 7B Instruct6-bit16K32 tok/sNot rated
Apertus 8B Instruct 25094-bit16K40 tok/sNot rated

Best fit is the highest quality, up to 8-bit, that leaves the use case’s context. Ranked by our use-case scores, then by Hugging Face downloads. Speeds are rough, plus or minus 25%. The top 60 of 234 are shown.

Image and video models

Image and video models that run on the RTX 4060
ModelBest fitPer outputStrongest at
Stable Diffusion XL 1.016-bit21 sImages
Z-Image Turbo8-bit30 sImages
FLUX.1 [dev]4-bit69 sImages
FLUX.3 Klein8-bit44 sImages
Stable Diffusion 3.5 Large5-bit4.9 minImages
Wan 2.2 TI2V-5B8-bit46 minVideo

Only just too big

Questions

What is the best LLM for the RTX 4060?

Of the open models we track, the strongest that run well on the RTX 4060 are Qwen3.5 9B, Gemma 4 12B, Qwen3.5 4B. 234 text models fit it in all.

Calculated from published specs, not benchmarked. All machines