LLMs · DeepSeek

DeepSeek V3.2 Exp

Parameters: 685B (37.0B active)
Architecture: Mixture of experts
Max context: 160K tokens
License: mit
Released: 2025-09

How to run it

Original weights · Hugging Face

Full-precision files from DeepSeek, for fine-tuning, vLLM, Transformers.

huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp (opens in a new tab)

No sign-in needed.

Links checked 6 days ago against Hugging Face, Ollama and GitHub.

Apps that run DeepSeek V3.2 Exp

Our pick for each kind of computer, assuming the model fits in its memory. Pick your own GPU or device in the Model Finder for an exact answer.

Mac (Apple silicon)

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

Windows PC, NVIDIA GPU

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

Windows PC, AMD Radeon

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

Linux, NVIDIA GPU

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

Linux server, 80 GB+ GPUs

Docker Model Runner

Handy if you already work in Docker.

Also good:

Text Generation WebUI

App	Type	Opens it as	Systems	Engine	Start
vLLM	Server	Original weights	WSLWindowsLinux	vLLM	`vllm serve deepseek-ai/DeepSeek-V3.2-Exp`
SGLang	Server	Original weights	WSLWindowsLinux	SGLang	`python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3.2-Exp`
Docker Model Runner	CLI + API	Original weights	macOSWindowsLinux	llama.cpp, vLLM	`docker model run hf.co/deepseek-ai/DeepSeek-V3.2-Exp`
Text Generation WebUI	Desktop app	Original weights	macOSWindowsLinux	llama.cpp, Transformers, ExLlamaV3	Get Text Generation WebUI

Which apps can open a model depends on the files published for it: GGUF for llama.cpp-based apps, MLX for Apple’s engine, the original weights for vLLM and SGLang. Compare every app.

Memory needed

Weights plus KV cache and runtime overhead. Leave a little extra free for the operating system and other apps. Try another context length.

KV cache: latent (MLA) KV cache.

Quantization	Quality	Weights	Total at 8K context	Total at 32K context	Download
`Q2_K`	Low	274 GB	275 GB	277 GB	All files (opens in a new tab)
`Q3_K_M`	Low	336 GB	337 GB	339 GB	All files (opens in a new tab)
`Q4_K_M`	Good	418 GB	419 GB	421 GB	All files (opens in a new tab)
`Q5_K_M`	High	487 GB	488 GB	490 GB	All files (opens in a new tab)
`Q6_K`	High	562 GB	563 GB	565 GB	All files (opens in a new tab)
`Q8_0`	Very high	727 GB	728 GB	729 GB	All files (opens in a new tab)
`F16`	Full	1,371 GB	1,372 GB	1,374 GB	Original weights (opens in a new tab)

Best hardware for DeepSeek V3.2 Exp

Quality	Hardware	Quant	Memory	Speed	Runs
77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	Apple M3 Ultra (512 GB)	`Q3_K_M`	368 GB / 384 GB	27.5 tok/s	Runs welltight fit
77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	Apple M5 Ultra (512 GB)	`Q3_K_M`	368 GB / 384 GB	38.8 tok/s	Runs welltight fit
64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	AMD Instinct MI350X	`Q2_K`	290 GB / 288 GB	133 tok/s	Runs welloffloads to RAM
87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	AMD EPYC 7002 / 7003 series NAS (512 GB)	`Q4_K_M`	457 GB / 471 GB	3.7 tok/s	Slowtight fit
64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	Intel Xeon Silver (4100 to 4300 series) NAS (512 GB)	`Q2_K`	313 GB / 471 GB	3.0 tok/s	Slow
87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	Intel Xeon W-2100 series NAS (512 GB)	`Q4_K_M`	457 GB / 471 GB	1.8 tok/s	Very slowtight fit
87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	Intel Xeon D-2700 series NAS (512 GB)	`Q4_K_M`	457 GB / 471 GB	1.4 tok/s	Very slowtight fit

Check it on your own hardware

Run it on a cluster

Devices that can’t run DeepSeek V3.2 Exp alone can run it together. These example clusters do:

Two Mac Studios (M3 Ultra, 256 GB)34.2 tok/s at Q3_K_M with EXO

Try it with your own devices

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.

Was this page helpful?