Qwen3.8 2.4T A95B on M5 Ultra 512 GB: VRAM, speed and fit
Qwen3.8 2.4T A95B at 4-bit needs 1,383 GB; the M5 Ultra 512 GB can use 384 GB.
Text generation · Chat · Ollama / LM Studio
Won’t fit · on M5 Ultra 512 GB · 4-bit (Q4_K_M) · Ollama / LM Studio · Chat
- GPU memory
- 1,511 GB
- of 512 GB
- Context
- 8K
- auto
- Speed
- n/a
- First reply
- n/a
Where the memory goes
- Model
- 1,381 GB
- Context
- 1.3 GB
- Overhead
- 1.0 GB
- OS
- 128 GB
- Over by
- 999 GB
- Memory
- 1,511 GB of 512 GB
- Context
- 8K tokens · auto
- Speed
- Doesn’t run here
- Running cost
- Doesn’t run here · $2,325/mo on a cloud 8× MI300X
Validation: Not validated
| Check | Target | Result | Status |
|---|---|---|---|
| Fits in GPU memory | 384 GB usable | 1,383 GB needed | Fails |
| Headroom | 5% free | none | Fails |
| Context for chat | 16K or more | – | Not run |
| Writing speed | 15 tok/s or more | – | Not run |
| First reply | under 3 s, 2K prompt | – | Not run |
Targets: a reply you can read along with, a first reply the use case tolerates, and 5% of memory left free. Calculated from published specs, not benchmarked.
The setup
- Model
- Qwen3.8 2.4T A95B
- Parameters
- 2.4T, 95B active
- Precision
- 4-bit (Q4_K_M)
- Context
- 8K, auto
- Chats at once
- 1
- Use
- Chat
- Runs in
- Ollama / LM Studio
- Hardware
- M5 Ultra 512 GB
- GPU can use
- 384 GB of 512 GB
Running cost
- At home
- n/a
- Doesn’t run on this machine.
- Renting instead
- $2,325/mo
- 8× MI300X at $19/h
4 h a day, power at $0.18 per kWh, cloud at RunPod on-demand prices as of Nov 22, 2026.
How to run it
- llama.cpp
llama-server -hf unsloth/Qwen3.8-2.4T-A95B-GGUF- vLLM
vllm serve Qwen/Qwen3.8-2.4T-A95B- Model card
- huggingface.co/Qwen/Qwen3.8-2.4T-A95B
Qwen3.8 2.4T A95B on similar machines
- M5 Max 128 GBToo big
- M3 Ultra 96 GBToo big
- M3 Ultra 256 GBToo big
- M3 Ultra 512 GBJust misses
- Ryzen AI Max+ 395 64 GBToo big
- Ryzen AI Max+ 395 128 GBToo big
- RTX SparkToo big
- DGX SparkToo big
All machines: Qwen3.8 2.4T A95B VRAM requirements. Everything that runs here: what can the M5 Ultra 512 GB run?
Other models that run on M5 Ultra 512 GB
Questions
Can Qwen3.8 2.4T A95B run on M5 Ultra 512 GB?
Qwen3.8 2.4T A95B at 4-bit needs 1,383 GB; the M5 Ultra 512 GB can use 384 GB.
Shown at the closest setup. Report VR-0WE4O9D, calculated Nov 22, 2026 from published specs, not benchmarked.