Intel Xeon E-2100 / E-2200 NAS (16 GB) vs Intel Arc A380 for local AI

Pick two to four systems or devices to see which runs more models, faster, for the kind of work you do.

Hardware 1

Hardware 2

Hardware 3 (optional)

Hardware 4 (optional)

Use

Winner: Intel Arc A380

For general chat: Its ten best models average a score of 87 vs 82; it runs 37 more LLMs (434 vs 397); it is about 7.3× faster on its top models.

Side by side

Swap sides All models on the Xeon E-2100 / E-2200 NAS All models on the Arc A380

	Intel Xeon E-2100 / E-2200 NAS (16 GB)	Intel Arc A380
Memory	16.0 GB unified	6.0 GB VRAM
Usable for models	12.0 GB	6.0 GB
Memory bandwidth	43 GB/s	186 GB/s
FP16 compute	0.3 TFLOPS	8.2 TFLOPS
Launch price	—	$139
Released	2018	2022
LLMs that run	397 of 622	434 of 622
LLMs that run well	173	297
Image models that run	54 of 61	43 of 61
Video models that run	14 of 43	3 of 43
Decision models that run	65 of 70	63 of 70
Top-10 LLM average score	82	87
Top-10 LLM average speed	6.5 tok/s	47.8 tok/s
Best LLM	Gemma 4 12B	Nanbeige4.2 3B

Highlighted values are the best of the two. Model results assume 8K of context and the default macOS GPU memory limit on Macs.

LLMs

397 of 622 LLMs run on the Intel Xeon E-2100 / E-2200 NAS (16 GB). 12.0 GB usable for models.

434 of 622 LLMs run on the Intel Arc A380. 6.0 GB usable for models.

Model	QualityXeon E-2100 / E-2200 NAS	SpeedXeon E-2100 / E-2200 NAS	RunsXeon E-2100 / E-2200 NAS	QualityArc A380	SpeedArc A380	RunsArc A380
	Intel Xeon E-2100 / E-2200 NAS (16 GB)			Intel Arc A380
Gemma 4 12BGoogle · 12B	78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	3.0 tok/s	Slow	69Good quality (B) · 69 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	14.8 tok/s	Runs welloffloads to RAM
K2 Horizon 7BInstitute of Foundation Models · 9.0B	77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	3.3 tok/s	Slow	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	31.8 tok/s	Runs welltight fit
SenseNova Loop 9B PreviewSenseTime · 9.7B	75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	29.7 tok/s	Runs welltight fit
Ornith 1.5 9BDeepReinforce · 9.7B	74Great quality (A) · 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	64Good quality (B) · 64 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	23.4 tok/s	Runs welltight fit
Qwen3.5 9BAlibaba · 9.7B	73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	23.4 tok/s	Runs welltight fit
Ornith 1.0 9BDeepReinforce · 9.7B	72Great quality (A) · 72 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	23.4 tok/s	Runs welltight fit
TMax 9BAi2 · 9.0B	71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	25.2 tok/s	Runs welltight fit
Ling 3.0 Tiny (8B-A1.3B)Ant Group (inclusionAI) · 7.9B (1.3B active)	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	25.6 tok/s	Runs well	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	92.4 tok/s	Runs welltight fit
Granite 4.2 8BIBM · 8.8B	65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	32.6 tok/s	Runs welltight fit
Mellum2.1 Thinking (12B-A2.5B)JetBrains · 12B (2.5B active)	66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	13.9 tok/s	Runs well	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	84.4 tok/s	Runs welltight fit
LFM2.5 24B-A2BLiquid AI · 24B (2.5B active)	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	22.7 tok/s	Runs welltight fit	70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	35.8 tok/s	Slowheavy RAM offload
MiMo V2.6 Distill Qwen 9BXiaomi · 9.4B	75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit
Maple Preview (20B-A1B)DeepGrove · 20B (1.5B active)	58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	28.6 tok/s	Runs welltight fit	69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	39.0 tok/s	Slowheavy RAM offload
Ling Mini 2.0 (16B-A1.4B)Ant Group (inclusionAI) · 16B (1.4B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	23.9 tok/s	Runs welltight fit	36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	105 tok/s	Runs welloffloads to RAM
Kimi-VL A3B Thinking 2506Moonshot AI · 16B (3.0B active)	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	11.7 tok/s	Runs welltight fit	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	59.9 tok/s	Runs welloffloads to RAM
LFM2.5 8B-A1BLiquid AI · 8.5B (1.5B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	22.4 tok/s	Runs well	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	101 tok/s	Runs welltight fit
InternVL3.5 8BShanghai AI Lab · 8.5B	60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	33.6 tok/s	Runs welltight fit
Qwen3-VL 8B InstructAlibaba · 8.8B	60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	32.7 tok/s	Runs welltight fit
Qwen3 8BAlibaba · 8.2B	59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	35.0 tok/s	Runs welltight fit
MiniCPM-V 4.5 8BOpenBMB · 8.7B	58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	32.9 tok/s	Runs welltight fit
Granite 4.1 8BIBM · 8.8B	58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	32.6 tok/s	Runs welltight fit
Nemotron Nano 9B v2NVIDIA · 8.9B	58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	25.4 tok/s	Runs welltight fit
Apriel 1.5 15B ThinkerServiceNow · 15B	60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slowheavy RAM offload
GLM-4.1V 9B ThinkingZhipu AI · 10B	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	27.8 tok/s	Runs welltight fit
Gemma 4 E2BGoogle · 5.1B	58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	15.8 tok/s	Runs well	60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.	62.6 tok/s	Runs great
MiMo-VL 7B RLXiaomi · 8.3B	57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	3.0 tok/s	Slow	40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	34.5 tok/s	Runs welltight fit
Qwen3 14BAlibaba · 15B	57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	7.1 tok/s	Slowheavy RAM offload
Nemotron 3.5 Bolt E4BNVIDIA · 8.0B	58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	10.5 tok/s	Runs well	66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	38.8 tok/s	Runs welltight fit
UI-Mate 9BTencent · 9.4B	67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit
UI-Venus 2 9BAnt Group (inclusionAI) · 9.4B	67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit
Groot 19B A3B 261023GrootTeam · 19B (3.0B active)	47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	14.9 tok/s	Runs welltight fit	54Fair quality (C) · 54 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	31.3 tok/s	Slowheavy RAM offload
Phi-4 Reasoning Vision 15BMicrosoft · 15B	58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slowtight fit	69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	4.4 tok/s	Slowheavy RAM offload
ERNIE 4.5 21B-A3BBaidu · 22B (3.0B active)	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	18.8 tok/s	Runs welltight fit	60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	30.0 tok/s	Slowheavy RAM offload
MiMo 7B RLXiaomi · 7.8B	55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	28.8 tok/s	Runs welltight fit
Hermes 4 14BNous Research · 15B	55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	7.1 tok/s	Slowheavy RAM offload
InternVL3 8BShanghai AI Lab · 7.9B	55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	28.4 tok/s	Runs welltight fit
Gemma 4 E4BGoogle · 8.0B	57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	10.5 tok/s	Runs well	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.	38.8 tok/s	Runs welltight fit
Ministral 3 8BMistral AI · 8.9B	54Fair quality (C) · 54 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	32.1 tok/s	Runs welltight fit
Granite 4.0 H Tiny (7B-A1B)IBM · 6.9B (1.0B active)	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	32.4 tok/s	Runs great	42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	95.8 tok/s	Runs welltight fit
Ministral 3 14BMistral AI · 14B	53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	7.7 tok/s	Slowheavy RAM offload
Kimi-VL A3B InstructMoonshot AI · 16B (3.0B active)	47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	11.7 tok/s	Runs welltight fit	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	59.9 tok/s	Runs welloffloads to RAM
Nanbeige4.2 3BBOSS Zhipin (Nanbeige) · 4.2B	57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	11.3 tok/s	Runs well	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.	41.8 tok/s	Runs great
LFM2 24B-A2BLiquid AI · 24B (2.3B active)	39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit	53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	37.9 tok/s	Slowheavy RAM offload
Llama 3.1 Nemotron Nano 8B v1NVIDIA · 8.0B	53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
MiniCPM4 8BOpenBMB · 8.2B	53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	27.5 tok/s	Runs welltight fit
Hunyuan 7B InstructTencent · 7.5B	51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	30.1 tok/s	Runs welltight fit
Qwen2.5 7B InstructAlibaba · 7.6B	51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	3.3 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	29.6 tok/s	Runs welltight fit
Phi-4 (14B)Microsoft · 15B	53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slowheavy RAM offload
InternVL3 14BShanghai AI Lab · 15B	53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slowtight fit	60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	6.8 tok/s	Slowheavy RAM offload
Gemma 3 12B ITGoogle · 12B	50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slow	50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	14.3 tok/s	Runs welloffloads to RAM
Qwen2.5-VL 7B InstructAlibaba · 8.3B	51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	3.0 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	27.2 tok/s	Runs welltight fit
MiniCPM-o 2.6 8BOpenBMB · 8.7B	50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	26.0 tok/s	Runs welltight fit
GLM-4 9B 0414Zhipu AI · 9.4B	50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit
NorthCUA Nanobeipo-ai · 9.4B	50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit
DeepSeek R1 0528 Qwen3 8BDeepSeek · 8.2B	61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	35.0 tok/s	Runs welltight fit
Moonlight 16B-A3B InstructMoonshot AI · 16B (3.0B active)	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	11.7 tok/s	Runs welltight fit	32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	60.8 tok/s	Runs welloffloads to RAM
Olmo 3 7B InstructAi2 · 7.3B	49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	30.9 tok/s	Runs welltight fit
Falcon 3 10B InstructTII · 10B	49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	17.0 tok/s	Runs welloffloads to RAM
Qwen2.5 14B InstructAlibaba · 15B	52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slowheavy RAM offload
Falcon-H1 7B InstructTII · 7.6B	49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.3 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	29.7 tok/s	Runs well
Qwen2.5 7B Instruct 1MAlibaba · 7.6B	49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.3 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	29.6 tok/s	Runs welltight fit
Nemotron Labs Dory 1.0 4BNVIDIA · 4.0B	57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	11.8 tok/s	Runs well	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	43.8 tok/s	Runs great
GLiDE 9BFastino · 7.9B	49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	28.4 tok/s	Runs well
Foundation-Sec 1.1 8B InstructCisco Foundation AI · 8.0B	49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
InternVL2.5 8BShanghai AI Lab · 8.1B	49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	27.9 tok/s	Runs welltight fit
ERNIE 4.5 21B-A3B ThinkingBaidu · 22B (3.0B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	18.8 tok/s	Runs welltight fit	66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	30.1 tok/s	Slowheavy RAM offload
MiniCPM5 2BOpenBMB · 2.5B	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	14.4 tok/s	Runs well	59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	48.0 tok/s	Runs great
InternLM3 8B InstructShanghai AI Lab · 8.8B	48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	25.6 tok/s	Runs welltight fit
Gemma 2 9B ITGoogle · 9.2B	48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	3.3 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	19.4 tok/s	Runs welloffloads to RAM
Qwen2.5 14B Instruct 1MAlibaba · 15B	50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slowheavy RAM offload
Hermes 3 (Llama 3.1 8B)Nous Research · 8.0B	47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Llama 3.1 8B InstructMeta · 8.0B	47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
MiniCPM-V 2.6 8BOpenBMB · 8.1B	47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	27.8 tok/s	Runs welltight fit
Phi-4 Reasoning Plus (14B)Microsoft · 15B	60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	4.6 tok/s	Slowheavy RAM offload
LFM2 8B-A1BLiquid AI · 8.3B (1.5B active)	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	22.4 tok/s	Runs well	31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	101 tok/s	Runs great
Spark-X2.6 4BSparkLLM · 4.1B	55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	11.4 tok/s	Runs well	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.	42.4 tok/s	Runs great
DeepSeek V2 Lite ChatDeepSeek · 16B (2.4B active)	33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	18.4 tok/s	Runs well	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	73.1 tok/s	Runs welloffloads to RAM
DeepSeek Coder V2 Lite (16B)DeepSeek · 16B (2.4B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	14.4 tok/s	Runs welltight fit	36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	73.1 tok/s	Runs welloffloads to RAM
GLM-4 9B Chat (2024)Zhipu AI · 9.4B	46Fair quality (C) · 46 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit
GLM-4 9B Chat 1MZhipu AI · 9.5B	46Fair quality (C) · 46 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	30.2 tok/s	Runs welltight fit
LFM3 1.7BLiquid AI · 1.7B	53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	21.0 tok/s	Runs well	56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	51.3 tok/s	Runs great
Seed-Coder 8B ReasoningByteDance · 8.3B	57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	34.7 tok/s	Runs welltight fit
Phi-4 Reasoning (14B)Microsoft · 15B	58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	4.6 tok/s	Slowheavy RAM offload
Granite 3.2 8B InstructIBM · 8.2B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	35.1 tok/s	Runs welltight fit
Granite 3.3 8BIBM · 8.2B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	35.1 tok/s	Runs welltight fit
GLM-Z1 9B (0414)Zhipu AI · 9.4B	56Good quality (B) · 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit
Apertus v1.5 8BSwiss AI · 8.9B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	4.1 tok/s	Slow	32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	32.2 tok/s	Runs welltight fit
Ministral 8B (2410)Mistral AI · 8.0B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	35.7 tok/s	Runs welltight fit
Mistral Nemo 12B InstructMistral AI · 12B	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	18.6 tok/s	Runs welloffloads to RAM
Ministral 3 14B ReasoningMistral AI · 14B	55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	7.7 tok/s	Slowheavy RAM offload
Falcon-H1R 7BTII · 7.6B	55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.3 tok/s	Slow	53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	23.0 tok/s	Runs welltight fit
EXAONE Deep 7.8BLG AI Research · 7.8B	55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	28.8 tok/s	Runs welltight fit
DeepHermes 3 Llama 3 8B PreviewNous Research · 8.0B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Dolphin 3.0 Llama 3.1 8BCognitive Computations · 8.0B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Tülu 3 8BAi2 · 8.0B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Seed-Coder 8B InstructByteDance · 8.3B	55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	34.7 tok/s	Runs welltight fit
Qwen2.5-Omni 7BAlibaba · 11B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	26.7 tok/s	Runs welltight fit
EXAONE 3.5 7.8BLG AI Research · 7.8B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	4.7 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	28.8 tok/s	Runs welltight fit
Qwen3.5 4BAlibaba · 4.7B	52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	10.1 tok/s	Runs well	58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	37.4 tok/s	Runs great
Qwen2.5 Coder 7B InstructAlibaba · 7.6B	54Fair quality (C) · 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.3 tok/s	Slow	46Fair quality (C) · 46 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	29.6 tok/s	Runs welltight fit
Granite 3.1 8B InstructIBM · 8.2B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	35.1 tok/s	Runs welltight fit
Pixtral 12B (2409)Mistral AI · 12B	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slow	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	18.1 tok/s	Runs welloffloads to RAM
TMax 4BAi2 · 4.2B	51Fair quality (C) · 51 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	11.2 tok/s	Runs well	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.	41.4 tok/s	Runs great
Olmo 3 7B ThinkAi2 · 7.3B	53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slow	45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	30.9 tok/s	Runs welltight fit
Falcon 3 7B InstructTII · 7.5B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	4.9 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	30.2 tok/s	Runs welltight fit
InternVL2 8BShanghai AI Lab · 8.1B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	27.9 tok/s	Runs welltight fit
OpenReasoning Nemotron 7BNVIDIA · 7.6B	53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.3 tok/s	Slow	45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	29.6 tok/s	Runs welltight fit
Dolphin 2.9.4 Llama 3.1 8BCognitive Computations · 8.0B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Hermes 2 Pro Llama 3 8BNous Research · 8.0B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
OpenChat 3.6 8BOpenChat · 8.0B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
AceReason Nemotron 14BNVIDIA · 15B	55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slowheavy RAM offload
OpenReasoning Nemotron 14BNVIDIA · 15B	55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slowheavy RAM offload
Qwen2.5 Coder 14B InstructAlibaba · 15B	55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slowheavy RAM offload
Apriel Nemotron 15B ThinkerServiceNow · 15B	55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slowtight fit	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	6.8 tok/s	Slowheavy RAM offload
Agents-A1 4BInternScience · 4.5B	50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	10.4 tok/s	Runs well	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.	38.4 tok/s	Runs great
Mistral-NeMo-Minitron 8B InstructNVIDIA · 8.4B	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	4.3 tok/s	Slow	29Basic quality (D) · 29 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	34.1 tok/s	Runs welltight fit
Phi-3 Small 8K InstructMicrosoft · 7.4B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	4.9 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	30.5 tok/s	Runs welltight fit
Unlimited OCR 2 (3B-A0.6B)Baidu · 3.3B (0.6B active)	42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	42.6 tok/s	Runs great	44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	94.7 tok/s	Runs great
JanusCoder 14BShanghai AI Lab · 15B	52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slow	58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	7.1 tok/s	Slowheavy RAM offload
Qwen3-VL 4B InstructAlibaba · 4.4B	45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	10.6 tok/s	Runs well	51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	39.3 tok/s	Runs great
Qwen3 4B Instruct 2507Alibaba · 4.0B	45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	11.7 tok/s	Runs well	51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	43.4 tok/s	Runs great
Gemma 4 26B-A4BGoogle · 26B (3.8B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	12.7 tok/s	Runs wellover memory limit	87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	23.6 tok/s	Slowheavy RAM offload
Aya Vision 8BCohere · 8.6B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	4.2 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	26.1 tok/s	Runs welltight fit
Llama 3 8B InstructMeta · 8.0B	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
DeepCoder 14B PreviewAgentica · 15B	53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slowheavy RAM offload
Qwen2-VL 7B InstructAlibaba · 8.3B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	4.4 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	27.2 tok/s	Runs welltight fit
Foundation-Sec 8B ReasoningCisco Foundation AI · 8.0B	51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Fara 1.5 4BMicrosoft · 4.5B	48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	10.4 tok/s	Runs well	55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	38.4 tok/s	Runs great
Llama 3.2 11B Vision InstructMeta · 11B	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	16.0 tok/s	Runs welloffloads to RAM
LFM3-VL 2BLiquid AI · 1.9B	47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	19.0 tok/s	Runs well	50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	46.2 tok/s	Runs great
Command R7BCohere · 8.0B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Qwen2 7B InstructAlibaba · 7.6B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	4.8 tok/s	Slow	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	29.6 tok/s	Runs welltight fit
Solar Pro Preview 22BUpstage · 22B	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	2.7 tok/s	Very slowtight fit	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	4.3 tok/s	Slowheavy RAM offload
Phi-3 Medium (14B)Microsoft · 14B	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	32Basic quality (D) · 32 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	16.2 tok/s	Runs welloffloads to RAM
Phi-3 Medium 128K InstructMicrosoft · 14B	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	7.5 tok/s	Slowheavy RAM offload
DeepSeek R1 Distill Qwen 14BDeepSeek · 15B	52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowtight fit	58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slowheavy RAM offload
Apertus 8B InstructSwiss AI · 8.1B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	28.0 tok/s	Runs welltight fit
Aya Expanse 8BCohere · 8.0B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
DeepSeek R1 Distill Llama 8BDeepSeek · 8.0B	49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Nous Hermes 2 SOLAR 10.7BNous Research · 11B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	26.7 tok/s	Runs welltight fit
SOLAR 10.7B InstructUpstage · 11B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slow	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	26.7 tok/s	Runs welltight fit
Fara 7BMicrosoft · 8.3B	49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.0 tok/s	Slow	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	27.2 tok/s	Runs welltight fit
InternLM2.5 7B ChatShanghai AI Lab · 7.7B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	4.7 tok/s	Slow	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	37.0 tok/s	Runs great
Phi-3.5 Mini InstructMicrosoft · 3.8B	33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	12.3 tok/s	Runs well	33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	59.0 tok/s	Runs welltight fit
Gemma 3n E4BGoogle · 7.9B	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	11.8 tok/s	Runs well	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	43.6 tok/s	Runs welltight fit
Command R7B ArabicCohere · 8.0B	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Qwen3 4BAlibaba · 4.0B	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	11.7 tok/s	Runs well	47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	43.4 tok/s	Runs great
Unlimited OCR (3B-A0.6B)Baidu · 3.3B (0.6B active)	38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	42.6 tok/s	Runs great	40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	94.7 tok/s	Runs great
Granite 3.0 8B InstructIBM · 8.2B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	35.1 tok/s	Runs great
Granite 4.2 3BIBM · 3.7B	43Fair quality (C) · 43 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	12.9 tok/s	Runs well	50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	39.4 tok/s	Runs great
Dolphin 2.9 Llama 3 8BCognitive Computations · 8.0B	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Granite 4.1 3BIBM · 3.4B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	10.7 tok/s	Runs well	44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	42.4 tok/s	Runs great
Raon-1 mini Preview (30B-A4B)KRAFTON · 29B (4.3B active)	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	11.3 tok/s	Runs wellover memory limit	83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	20.9 tok/s	Slowheavy RAM offload
OLMo 2 7B InstructAi2 · 7.3B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	39.2 tok/s	Runs welltight fit
North Micro VisionCohere · 2.5B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	14.7 tok/s	Runs well	43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	48.8 tok/s	Runs great
Phi-3.5 Vision InstructMicrosoft · 4.2B	31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	11.3 tok/s	Runs well	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	69.0 tok/s	Runs welltight fit
Mistral 7B Instruct v0.3Mistral AI · 7.3B	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	39.5 tok/s	Runs great
Gemma 3n E2BGoogle · 5.4B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	18.2 tok/s	Runs well	34Basic quality (D) · 34 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	72.0 tok/s	Runs great
Mage-VL 4.7BMicrosoft · 4.7B	39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	12.6 tok/s	Runs well	53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	36.8 tok/s	Runs great
OLMo 2 13B InstructAi2 · 14B	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slowtight fit	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.	8.9 tok/s	Slowheavy RAM offload
Phi-3 Mini 128K InstructMicrosoft · 3.8B	30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	12.3 tok/s	Runs well	30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	59.0 tok/s	Runs welltight fit
Zamba2 7B InstructZyphra · 7.4B	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	4.9 tok/s	Slow	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	38.9 tok/s	Runs great
EXAONE 3.0 7.8B InstructLG AI Research · 7.8B	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	4.7 tok/s	Slow	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	36.6 tok/s	Runs great
Yi 1.5 9B Chat01.AI · 8.8B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	4.1 tok/s	Slow	25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	32.4 tok/s	Runs great
Hermes 2 Pro Mistral 7BNous Research · 7.2B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Starling LM 7B BetaNexusflow · 7.2B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Aya 23 8BCohere · 8.0B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Runs welltight fit
Nemotron 3 Nano 4BNVIDIA · 4.0B	42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	11.9 tok/s	Runs well	47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	43.9 tok/s	Runs great
CodeGeeX4 All 9BZhipu AI · 9.4B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	24.0 tok/s	Runs welltight fit
LFM2.5 2.6BLiquid AI · 2.7B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	13.5 tok/s	Runs well	43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	44.8 tok/s	Runs great
Phi-3 Vision 128K InstructMicrosoft · 4.2B	28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	11.3 tok/s	Runs well	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	69.0 tok/s	Runs welltight fit
SmolLM3 3BHugging Face · 3.1B	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	11.8 tok/s	Runs well	41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	39.3 tok/s	Runs great
OLMoE 1B-7B InstructAi2 · 6.9B (1.3B active)	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	32.4 tok/s	Runs great	24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	92.4 tok/s	Runs welltight fit
InternLM2.5 20B ChatShanghai AI Lab · 20B	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	3.0 tok/s	Slowtight fit	47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	4.9 tok/s	Slowheavy RAM offload
OpenChat 3.5 (0106)OpenChat · 7.2B	33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	24Basic quality (D) · 24 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
LFM2.5-VL 3BLiquid AI · 3.1B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	11.7 tok/s	Runs well	43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	38.8 tok/s	Runs great
MiniCPM5 1BOpenBMB · 1.1B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	33.7 tok/s	Runs great	36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	82.2 tok/s	Runs great
DeepSeek MoE 16B ChatDeepSeek · 16B (2.8B active)	18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	20.0 tok/s	Runs well	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	44.6 tok/s	Slowheavy RAM offload
Reka Flash 3.1 (21B)Reka AI · 21B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	2.9 tok/s	Very slowtight fit	64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slowheavy RAM offload
Qwen1.5 MoE A2.7B ChatAlibaba · 14B (2.7B active)	20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	16.5 tok/s	Runs well	22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	34.1 tok/s	Slowheavy RAM offload
RecurrentGemma 9B ITGoogle · 9.6B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slow	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	29.7 tok/s	Runs well
Phi-4 Mini (3.8B)Microsoft · 3.8B	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	12.3 tok/s	Runs well	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	45.4 tok/s	Runs great
Llama 3.1 Nemotron Nano 4B v1.1NVIDIA · 4.5B	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	10.4 tok/s	Runs well	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	38.7 tok/s	Runs great
MiniCPM-V 4.7 1BOpenBMB · 1.6B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	23.5 tok/s	Runs well	40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	57.3 tok/s	Runs great
Jamba Reasoning 3BAI21 Labs · 3.2B	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	11.4 tok/s	Runs well	42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	45.0 tok/s	Runs great
Mistral 7B Instruct v0.2Mistral AI · 7.2B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Nous Hermes 2 Mistral 7B DPONous Research · 7.2B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
OpenChat 3.5 (1210)OpenChat · 7.2B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Qwen3 4B Thinking 2507Alibaba · 4.0B	47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	11.7 tok/s	Runs well	53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	43.4 tok/s	Runs great
Ministral 3 3BMistral AI · 3.9B	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	12.2 tok/s	Runs well	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	45.3 tok/s	Runs great
Falcon3 Mamba 7B InstructTII · 7.3B	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	31.0 tok/s	Runs great
Granite 4.0 Micro (3B)IBM · 3.4B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	10.7 tok/s	Runs well	38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	42.4 tok/s	Runs great
Hunyuan 4B InstructTencent · 4.2B	35Basic quality (D) · 35 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	11.2 tok/s	Runs well	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	41.3 tok/s	Runs great
TMax 2BAi2 · 1.9B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	19.4 tok/s	Runs well	40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	47.2 tok/s	Runs great
DiffusionGemma 26B-A4B ITGoogle · 26B (4.0B active)	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	12.1 tok/s	Runs wellover memory limit	74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	22.5 tok/s	Slowheavy RAM offload
Jamba2 3BAI21 Labs · 3.0B	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	12.0 tok/s	Runs well	41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	39.9 tok/s	Runs great
DeepSeek R1 Distill Qwen 7BDeepSeek · 7.6B	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	4.8 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	29.6 tok/s	Runs welltight fit
gpt-oss 20BOpenAI · 21B (3.6B active)	70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	8.9 tok/s	Slowfar over memory limit	70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	28.1 tok/s	Slowheavy RAM offload
OpenHermes 2.5 Mistral 7BTeknium · 7.2B	30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Starling LM 7B AlphaBerkeley · 7.2B	30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
DeepSeek Prover V2 7BDeepSeek · 6.9B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	5.3 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.	13.7 tok/s	Slowheavy RAM offload
Qwen3-VL 2B InstructAlibaba · 2.1B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	17.1 tok/s	Runs well	36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	41.7 tok/s	Runs great
Qwen3.5 2BAlibaba · 2.3B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	16.0 tok/s	Runs well	40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	39.1 tok/s	Runs great
Gemma 3 4B ITGoogle · 4.3B	37Basic quality (D) · 37 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	10.9 tok/s	Runs well	42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	40.6 tok/s	Runs great
MedGemma 4B ITGoogle · 4.3B	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	10.9 tok/s	Runs well	41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	40.6 tok/s	Runs great
PaddleOCR-VL 1.6 0.9BBaidu · 1.0B	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	37.9 tok/s	Runs great	42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	42.6 tok/s	Runs great
Yi Coder 9B Chat01.AI · 8.8B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	4.1 tok/s	Slow	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	32.4 tok/s	Runs welltight fit
Mistral 7B Instruct v0.1Mistral AI · 7.2B	28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Phi-3 Mini 4K InstructMicrosoft · 3.8B	30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	12.3 tok/s	Runs well	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	45.7 tok/s	Runs great
TMax 27BAi2 · 27B	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	1.9 tok/s	Very slowover memory limit	85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slowheavy RAM offload
Llama 3.2 3B InstructMeta · 3.2B	33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	11.3 tok/s	Runs well	33Basic quality (D) · 33 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	44.9 tok/s	Runs great
LFM2.5 1.2B InstructLiquid AI · 1.2B	26Basic quality (D) · 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	40.2 tok/s	Runs great	32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	75.9 tok/s	Runs great
Granite 3.1 3B-A800M InstructIBM · 3.3B (0.8B active)	19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	39.5 tok/s	Runs great	20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	78.0 tok/s	Runs great
InternLM2 Chat 20BShanghai AI Lab · 20B	29Basic quality (D) · 29 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	3.0 tok/s	Slowtight fit	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.	4.9 tok/s	Slowheavy RAM offload
Mage-VL-Cowork 4.7BMicrosoft · 4.7B	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	12.6 tok/s	Runs well	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	36.8 tok/s	Runs great
AFM 4.5BArcee AI · 4.6B	33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	10.2 tok/s	Runs well	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	37.8 tok/s	Runs great
GLM-4.7 Flash (30B-A3B)Zhipu AI · 31B (3.0B active)	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	15.9 tok/s	Slowfar over memory limit	73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	37.0 tok/s	Slowheavy RAM offload
Gemma 1.1 7B ITGoogle · 8.5B	23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	5.5 tok/s	Slow	26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.	11.1 tok/s	Slowheavy RAM offload
Granite 4.0 H MicroIBM · 3.2B	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	11.4 tok/s	Runs well	36Basic quality (D) · 36 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	45.2 tok/s	Runs great
LLaVA 1.6 Mistral 7BLLaVA · 7.6B	26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	4.8 tok/s	Slow	19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	37.8 tok/s	Runs great
Phi-4 Multimodal InstructMicrosoft · 5.6B	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	10.7 tok/s	Runs well	36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	40.5 tok/s	Runs great
LLM4Decompile 9B V2LLM4Binary · 8.8B	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	4.1 tok/s	Slow	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	32.4 tok/s	Runs welltight fit
Fara 1.5 27BMicrosoft · 27B	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	1.9 tok/s	Very slowover memory limit	83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slowheavy RAM offload
EXAONE 4.0 1.2BLG AI Research · 1.3B	25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	36.8 tok/s	Runs great	30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	69.4 tok/s	Runs great
Neural Chat 7B v3.3Intel · 7.2B	26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Qwen1.5 7B ChatAlibaba · 7.7B	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	6.1 tok/s	Slow	18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking.	20.2 tok/s	Slowheavy RAM offload
Falcon-H1 3B InstructTII · 3.2B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	11.6 tok/s	Runs well	35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	38.4 tok/s	Runs great
TeleOCRXingChen-AGI · 1.4B	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	25.6 tok/s	Runs well	40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	62.5 tok/s	Runs great
Voxtral Mini 3BMistral AI · 4.7B	30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	10.1 tok/s	Runs well	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	37.3 tok/s	Runs great
Gemma 7B ITGoogle · 8.5B	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	5.5 tok/s	Slow	24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 43/100 (quality, fit, speed and memory use) sets the ranking.	11.1 tok/s	Slowheavy RAM offload
Qwen3.5 0.8BAlibaba · 0.9B	24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	41.8 tok/s	Runs great	26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	47.0 tok/s	Runs great
UI-Mate 27BTencent · 27B	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	1.9 tok/s	Very slowover memory limit	81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slowheavy RAM offload
Trinity Mini (26B-A3B)Arcee AI · 26B (3.0B active)	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	15.9 tok/s	Runs wellover memory limit	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	29.4 tok/s	Slowheavy RAM offload
StableLM 2 12B ChatStability AI · 12B	23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slow	19Basic quality (D) · 19 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	20.7 tok/s	Runs welloffloads to RAM
Hermes 3 Llama 3.2 3BNous Research · 3.2B	30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	11.3 tok/s	Runs well	31Basic quality (D) · 31 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	44.9 tok/s	Runs great
MiniCPM-OCR 1.0 1.3BOpenBMB · 1.3B	39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	28.0 tok/s	Runs well	42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	68.3 tok/s	Runs great
Raon 1 nanoKRAFTON · 3.5B	33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	10.3 tok/s	Runs well	33Basic quality (D) · 33 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	40.8 tok/s	Runs great
Granite 3.0 3B-A800M InstructIBM · 3.4B (0.8B active)	17Basic quality (D) · 17 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	39.5 tok/s	Runs great	18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	78.0 tok/s	Runs great
Qwen3 1.7BAlibaba · 2.0B	28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	17.9 tok/s	Runs well	30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	43.7 tok/s	Runs great
DeepSeek Coder 6.7B InstructDeepSeek · 6.7B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	5.4 tok/s	Slow	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking.	23.7 tok/s	Slowheavy RAM offload
Zephyr 7B BetaHugging Face · 7.2B	24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Orca 2 7BMicrosoft · 6.7B	20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	5.4 tok/s	Slow	15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	42.5 tok/s	Runs welltight fit
Qwen2.5 3B InstructAlibaba · 3.1B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	11.8 tok/s	Runs well	33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	39.2 tok/s	Runs great
MiniCPM3 4BOpenBMB · 4.1B	31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	11.6 tok/s	Runs well	36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	42.9 tok/s	Runs great
LFM2 2.6BLiquid AI · 2.6B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	14.2 tok/s	Runs well	33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	47.1 tok/s	Runs great
Falcon Mamba 7B InstructTII · 7.3B	28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	31.0 tok/s	Runs great
S5 nanoMaritaca AI · 4.7B	31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	10.1 tok/s	Runs well	35Basic quality (D) · 35 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	37.4 tok/s	Runs great
Phi-4 Mini Reasoning (3.8B)Microsoft · 3.8B	38Basic quality (D) · 38 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	12.3 tok/s	Runs well	43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	45.4 tok/s	Runs great
LLM4Decompile 6.7B V2LLM4Binary · 6.7B	30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	5.4 tok/s	Slow	22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 40/100 (quality, fit, speed and memory use) sets the ranking.	23.7 tok/s	Slowheavy RAM offload
LFM2 1.2BLiquid AI · 1.2B	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	40.2 tok/s	Runs great	26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	75.9 tok/s	Runs great
Orca 2 13BMicrosoft · 13B	23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	3.6 tok/s	Slowtight fit	23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking.	9.5 tok/s	Slowheavy RAM offload
Granite 3.2 2B InstructIBM · 2.5B	28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	14.4 tok/s	Runs well	29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	47.8 tok/s	Runs great
Granite 3.3 2BIBM · 2.5B	28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	14.4 tok/s	Runs well	29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	47.8 tok/s	Runs great
EXAONE 3.5 2.4BLG AI Research · 2.4B	28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	15.1 tok/s	Runs well	29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	50.2 tok/s	Runs great
Llama 2 7B ChatMeta · 6.7B	19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	5.4 tok/s	Slow	14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	42.5 tok/s	Runs welltight fit
Falcon 3 3B InstructTII · 3.2B	28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	11.3 tok/s	Runs well	29Basic quality (D) · 29 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	44.6 tok/s	Runs great
Baichuan 2 7B ChatBaichuan · 7.5B	19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	4.8 tok/s	Slow	14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	37.4 tok/s	Runs welloffloads to RAM
LightOnOCR-3 4BLightOn · 4.5B	40Fair quality (C) · 40 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	10.4 tok/s	Runs well	45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	38.4 tok/s	Runs great
Granite 4.0 H 1BIBM · 1.5B	23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	32.2 tok/s	Runs great	28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	60.8 tok/s	Runs great
Granite Code 8B Instruct (128K)IBM · 8.1B	30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slow	22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	35.6 tok/s	Runs welltight fit
CodeGemma 7BGoogle · 8.5B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	4.3 tok/s	Slowtight fit	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking.	11.1 tok/s	Slowheavy RAM offload
Mathstral 7BMistral AI · 7.3B	32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	39.5 tok/s	Runs great
Yi 1.5 6B Chat01.AI · 6.1B	26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	6.0 tok/s	Slow	23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	37.2 tok/s	Runs great
WeVisOne VL 2BTencent · 2.4B	25Basic quality (D) · 25 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	14.9 tok/s	Runs well	26Basic quality (D) · 26 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	49.6 tok/s	Runs great
Qwen2.5-VL 3B InstructAlibaba · 3.8B	28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	12.5 tok/s	Runs well	32Basic quality (D) · 32 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	38.4 tok/s	Runs great
Llama 2 13B ChatMeta · 13B	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	3.6 tok/s	Slowtight fit	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking.	9.5 tok/s	Slowheavy RAM offload
SQLCoder 7B v2Defog · 6.7B	28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	5.4 tok/s	Slow	21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking.	23.7 tok/s	Slowheavy RAM offload
Granite 3.1 2B InstructIBM · 2.5B	26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	14.4 tok/s	Runs well	27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	47.8 tok/s	Runs great
Baichuan 2 13B ChatBaichuan · 14B	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slowtight fit	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 40/100 (quality, fit, speed and memory use) sets the ranking.	8.8 tok/s	Slowheavy RAM offload
Qianfan-OCR 4BBaidu · 4.7B	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	12.6 tok/s	Runs well	47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	36.8 tok/s	Runs great
SmolVLM2 2.2B InstructHugging Face · 2.3B	20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	20.9 tok/s	Runs well	24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	39.5 tok/s	Runs great
DeepSeek LLM 7B ChatDeepSeek · 6.9B	17Basic quality (D) · 17 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	5.3 tok/s	Slow	12Basic quality (D) · 12 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	41.5 tok/s	Runs welltight fit
Nemotron Cascade 2 30B-A3BNVIDIA · 32B (3.5B active)	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	13.8 tok/s	Slowfar over memory limit	76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	24.9 tok/s	Slowheavy RAM offload
InternLM2 Math Plus 7BShanghai AI Lab · 7.7B	30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	4.7 tok/s	Slow	22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	37.0 tok/s	Runs great
Qwen2.5-Math 7B InstructAlibaba · 7.6B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	4.8 tok/s	Slow	30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	29.6 tok/s	Runs well
Gemma 2 2BGoogle · 2.6B	26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	13.9 tok/s	Runs well	27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	46.4 tok/s	Runs great
SmolLM2 1.7B InstructHugging Face · 1.7B	14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	35.0 tok/s	Runs great	20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	51.9 tok/s	Runs great
Llama 3.2 1B InstructMeta · 1.2B	18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	37.9 tok/s	Runs great	22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	71.6 tok/s	Runs great
Falcon-H1 1.5B InstructTII · 1.6B	18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	38.6 tok/s	Runs great	26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	57.3 tok/s	Runs great
Devstral Small 2 (24B)Mistral AI · 24B	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	2.1 tok/s	Very slowover memory limit	72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slowheavy RAM offload
SmolVLM 2.2B InstructHugging Face · 2.3B	18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	20.9 tok/s	Runs well	21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	53.8 tok/s	Runs great
Qwen3 30B-A3B Instruct 2507Alibaba · 31B (3.3B active)	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	14.6 tok/s	Slowfar over memory limit	64Good quality (B) · 64 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	33.7 tok/s	Slowheavy RAM offload
Codestral Mamba 7BMistral AI · 7.3B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	30.9 tok/s	Runs great
Qwen3.8 27BAlibaba · 28B	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	1.8 tok/s	Very slowfar over memory limit	90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slowheavy RAM offload
LLaVA 1.5 7BLLaVA · 7.1B	15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	5.2 tok/s	Slow	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	40.6 tok/s	Runs welltight fit
Granite Vision 3.2 2BIBM · 3.0B	24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	12.2 tok/s	Runs well	25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	40.6 tok/s	Runs great
StarCoder2 15B InstructBigCode · 16B	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	3.7 tok/s	Slow	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking.	6.8 tok/s	Slowheavy RAM offload
Gemma 3 1BGoogle · 1.0B	19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	36.4 tok/s	Runs great	20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	40.9 tok/s	Runs great
LFM2 700MLiquid AI · 0.7B	17Basic quality (D) · 17 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	40.6 tok/s	Runs great	18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	55.3 tok/s	Runs great
Nemotron 3 Nano 30B-A3BNVIDIA · 32B (3.5B active)	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	13.8 tok/s	Slowfar over memory limit	73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	24.9 tok/s	Slowheavy RAM offload
WizardLM 13B v1.2WizardLM · 13B	15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	4.6 tok/s	Slow	18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking.	9.5 tok/s	Slowheavy RAM offload
Qwen2.5 1.5B InstructAlibaba · 1.5B	16Basic quality (D) · 16 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	38.8 tok/s	Runs great	24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	57.7 tok/s	Runs great
Qwen3 0.6BAlibaba · 0.8B	13Basic quality (D) · 13 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	40.1 tok/s	Runs great	14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	54.6 tok/s	Runs great
Qwen3.6 27BAlibaba · 28B	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	1.8 tok/s	Very slowfar over memory limit	89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slowheavy RAM offload
LLaVA 1.5 13BLLaVA · 13B	18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slowtight fit	18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking.	9.2 tok/s	Slowheavy RAM offload
Granite 3.0 2B InstructIBM · 2.6B	24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	13.8 tok/s	Runs well	25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	46.0 tok/s	Runs great
Sarvam 30BSarvam AI · 32B (2.4B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	19.6 tok/s	Slowfar over memory limit	66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	34.8 tok/s	Slowheavy RAM offload
LFM2-VL 1.6BLiquid AI · 1.6B	16Basic quality (D) · 16 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	37.8 tok/s	Runs great	24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	56.2 tok/s	Runs great
WeVisOne-VL 4BTencent · 4.8B	31Basic quality (D) · 31 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	12.4 tok/s	Runs well	42Fair quality (C) · 42 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	36.1 tok/s	Runs great
Qwen3.5 27BAlibaba · 28B	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	1.8 tok/s	Very slowfar over memory limit	88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slowheavy RAM offload
Vicuna 13B v1.5LMSYS · 13B	14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	4.6 tok/s	Slow	16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 38/100 (quality, fit, speed and memory use) sets the ranking.	9.5 tok/s	Slowheavy RAM offload
Qwen3 30B-A3BAlibaba · 31B (3.3B active)	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	14.6 tok/s	Slowfar over memory limit	62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	33.7 tok/s	Slowheavy RAM offload
Granite Code 20B InstructIBM · 20B	30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	2.3 tok/s	Very slowtight fit	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking.	5.2 tok/s	Slowheavy RAM offload
Nemotron 3.5 Lightning 30B-A3BNVIDIA · 32B (3.0B active)	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	15.9 tok/s	Slowfar over memory limit	68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	28.6 tok/s	Slowheavy RAM offload
Muse Glimmer 30BMeta · 30B	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	1.7 tok/s	Very slowfar over memory limit	87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	3.2 tok/s	Slowheavy RAM offload
Hunyuan 1.8B InstructTencent · 1.8B	15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	33.4 tok/s	Runs great	22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	49.6 tok/s	Runs great
Code Llama 7B InstructMeta · 6.7B	22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	5.4 tok/s	Slow	16Basic quality (D) · 16 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 38/100 (quality, fit, speed and memory use) sets the ranking.	23.7 tok/s	Slowheavy RAM offload
Qwen2.5 Coder 3B InstructAlibaba · 3.1B	34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	11.8 tok/s	Runs well	35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	39.2 tok/s	Runs great
InternLM2 Math Plus 20BShanghai AI Lab · 20B	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	3.0 tok/s	Slowtight fit	38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking.	4.9 tok/s	Slowheavy RAM offload
Chandra OCR 2Datalab · 5.3B	33Basic quality (D) · 33 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	11.3 tok/s	Runs well	45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	32.9 tok/s	Runs great
Qwen2-VL 2B InstructAlibaba · 2.2B	20Basic quality (D) · 20 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	21.3 tok/s	Runs well	24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	40.2 tok/s	Runs great
Devstral Small 1.1 (24B)Mistral AI · 24B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	2.2 tok/s	Very slowover memory limit	66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	4.0 tok/s	Slowheavy RAM offload
Magistral Small 2507Mistral AI · 24B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	2.2 tok/s	Very slowover memory limit	66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	4.0 tok/s	Slowheavy RAM offload
Falcon-H1 0.5B InstructTII · 0.5B	13Basic quality (D) · 13 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	48.6 tok/s	Runs great	14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	78.7 tok/s	Runs great
MedGemma 27B Text ITGoogle · 27B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	1.9 tok/s	Very slowover memory limit	66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slowheavy RAM offload
Hunyuan Hy-Memory 30B-A3BTencent · 30B (3.0B active)	59Good quality (B) · 59 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	15.9 tok/s	Slowfar over memory limit	71Great quality (A) · 71 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	36.8 tok/s	Slowheavy RAM offload
LFM2.5 350MLiquid AI · 0.4B	14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	52.9 tok/s	Runs great	14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	117 tok/s	Runs great
Qwen2.5-Omni 3BAlibaba · 5.5B	21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	10.8 tok/s	Runs well	25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	40.7 tok/s	Runs great
Vicuna 7B v1.5LMSYS · 6.7B	11Basic quality (D) · 11 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	7.0 tok/s	Slow	9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	42.5 tok/s	Runs welltight fit
InternLM2.5 1.8B ChatShanghai AI Lab · 1.9B	12Basic quality (D) · 12 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	31.6 tok/s	Runs great	18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	47.0 tok/s	Runs great
WizardMath 7B v1.1WizardLM · 7.2B	24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slow	18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	39.6 tok/s	Runs great
Yi 6B Chat01.AI · 6.1B	19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	6.0 tok/s	Slow	16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	37.2 tok/s	Runs great
Magistral Small 2506Mistral AI · 24B	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	2.2 tok/s	Very slowover memory limit	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	4.0 tok/s	Slowheavy RAM offload
Mistral Small 3.2 24BMistral AI · 24B	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	2.1 tok/s	Very slowover memory limit	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slowheavy RAM offload
Falcon 3 1B InstructTII · 1.7B	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	35.8 tok/s	Runs great	16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	53.2 tok/s	Runs great
StarCoder2 15B (base)BigCode · 16B	22Basic quality (D) · 22 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	3.7 tok/s	Slow	30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 38/100 (quality, fit, speed and memory use) sets the ranking.	6.8 tok/s	Slowheavy RAM offload
OLMo 2 1B InstructAi2 · 1.5B	9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	40.4 tok/s	Runs great	14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	60.0 tok/s	Runs great
Zamba2 2.7B InstructZyphra · 2.7B	18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	17.7 tok/s	Runs well	21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	45.5 tok/s	Runs great
LFM2-VL 450MLiquid AI · 0.5B	12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	41.2 tok/s	Runs great	12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	90.9 tok/s	Runs great
Qwen2 1.5B InstructAlibaba · 1.5B	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	38.8 tok/s	Runs great	16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	57.7 tok/s	Runs great
Hemmingway 1.5Hemmingway · 27B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	1.9 tok/s	Very slowover memory limit	63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	3.6 tok/s	Slowheavy RAM offload
LFM2 350MLiquid AI · 0.4B	12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	52.9 tok/s	Runs great	12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	117 tok/s	Runs great
Nemotron Mini 4B InstructNVIDIA · 4.2B	18Basic quality (D) · 18 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	11.2 tok/s	Runs well	20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	41.6 tok/s	Runs great
Mistral Small 3.1 24BMistral AI · 24B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	2.1 tok/s	Very slowover memory limit	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slowheavy RAM offload
Granite 4.0 H 350MIBM · 0.3B	12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	54.5 tok/s	Runs great	12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	120 tok/s	Runs great
GLiDE 0.8BFastino · 0.8B	10Basic quality (D) · 10 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	40.1 tok/s	Runs great	11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	54.6 tok/s	Runs great
Hunyuan 0.5B InstructTencent · 0.5B	7Basic quality (D) · 7 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	46.8 tok/s	Runs great	8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	75.8 tok/s	Runs great
SmolVLM 500M InstructHugging Face · 0.5B	9Basic quality (D) · 9 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	49.5 tok/s	Runs great	10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	80.2 tok/s	Runs great
StableLM Zephyr 3BStability AI · 2.8B	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	21.4 tok/s	Runs well	15Basic quality (D) · 15 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	43.2 tok/s	Runs great
North Mini Code 1.0 (30B-A3B)Cohere · 30B (3.0B active)	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	15.9 tok/s	Slowfar over memory limit	76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	28.6 tok/s	Slowheavy RAM offload
Mistral Small 3 24B (2501)Mistral AI · 24B	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	2.2 tok/s	Very slowover memory limit	60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	4.0 tok/s	Slowheavy RAM offload
Spark X2.6 1.7BSparkLLM · 1.7B	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	35.0 tok/s	Runs great	16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	51.9 tok/s	Runs great
SmolLM 1.7B InstructHugging Face · 1.7B	9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	35.0 tok/s	Runs great	14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	51.9 tok/s	Runs great
Qwen2.5 0.5B InstructAlibaba · 0.5B	9Basic quality (D) · 9 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	51.5 tok/s	Runs great	10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	83.5 tok/s	Runs great
GLiDE 2BFastino · 1.9B	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	31.8 tok/s	Runs great	17Basic quality (D) · 17 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	47.2 tok/s	Runs great
StableLM 2 1.6B ChatStability AI · 1.6B	7Basic quality (D) · 7 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	36.5 tok/s	Runs great	10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	54.1 tok/s	Runs great
StableLM 2 Zephyr 1.6BStability AI · 1.6B	7Basic quality (D) · 7 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	36.5 tok/s	Runs great	10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	54.1 tok/s	Runs great
LFM2.5 230MLiquid AI · 0.2B	10Basic quality (D) · 10 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	80.6 tok/s	Runs great	10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	178 tok/s	Runs great
Stable Code Instruct 3BStability AI · 2.8B	16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	16.8 tok/s	Runs well	19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	43.2 tok/s	Runs welltight fit
OLMo 7B InstructAi2 · 6.9B	11Basic quality (D) · 11 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	6.8 tok/s	Slow	9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	41.6 tok/s	Runs great
StarCoder2 7BBigCode · 7.2B	22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	5.1 tok/s	Slow	16Basic quality (D) · 16 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	39.9 tok/s	Runs great
Dolphin Mistral 24B Venice EditionCognitive Computations · 24B	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	2.1 tok/s	Very slowover memory limit	58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slowheavy RAM offload
SmolLM2 360M InstructHugging Face · 0.4B	8Basic quality (D) · 8 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	51.5 tok/s	Runs great	8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	114 tok/s	Runs great
Voxtral Small 24BMistral AI · 24B	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	2.1 tok/s	Very slowover memory limit	58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slowheavy RAM offload
StarCoder 15BBigCode · 16B	18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slow	18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking.	14.5 tok/s	Runs welloffloads to RAM
Granite Code 3B Instruct (128K)IBM · 3.5B	16Basic quality (D) · 16 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	13.5 tok/s	Runs well	19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	50.1 tok/s	Runs welltight fit
Qwen3 30B-A3B Thinking 2507Alibaba · 31B (3.3B active)	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	14.6 tok/s	Slowfar over memory limit	65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	33.7 tok/s	Slowheavy RAM offload
Qwen3 Coder 30B-A3BAlibaba · 31B (3.3B active)	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	14.6 tok/s	Slowfar over memory limit	65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	33.7 tok/s	Slowheavy RAM offload
Qwen2.5 Coder 1.5B InstructAlibaba · 1.5B	18Basic quality (D) · 18 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	38.8 tok/s	Runs great	26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	57.7 tok/s	Runs great
Phi-2 2.7BMicrosoft · 2.8B	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	21.5 tok/s	Runs well	15Basic quality (D) · 15 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	43.5 tok/s	Runs great
Dolphin 3.0 R1 Mistral 24BCognitive Computations · 24B	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	2.2 tok/s	Very slowover memory limit	57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	4.0 tok/s	Slowheavy RAM offload
Yi Coder 1.5B Chat01.AI · 1.5B	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	40.4 tok/s	Runs great	16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	60.0 tok/s	Runs great
DeepScaleR 1.5B PreviewAgentica · 1.8B	21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	26.4 tok/s	Runs well	26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	49.9 tok/s	Runs great
ERNIE 4.5 0.3BBaidu · 0.4B	6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	51.5 tok/s	Runs great	6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	114 tok/s	Runs great
Qwen2 0.5B InstructAlibaba · 0.5B	5Basic quality (D) · 5 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	51.5 tok/s	Runs great	6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	83.5 tok/s	Runs great
SmolVLM 256M InstructHugging Face · 0.3B	6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	71.3 tok/s	Runs great	6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	157 tok/s	Runs great
TinyLlama 1.1B ChatTinyLlama · 1.1B	5Basic quality (D) · 5 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	42.8 tok/s	Runs great	6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	80.7 tok/s	Runs great
Gemma 3 270MGoogle · 0.3B	6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	68.6 tok/s	Runs great	6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	152 tok/s	Runs great
InternLM 7BShanghai AI Lab · 7.3B	8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	8.2 tok/s	Slow	8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	39.1 tok/s	Runs great
SmolLM 360M InstructHugging Face · 0.4B	5Basic quality (D) · 5 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	51.5 tok/s	Runs great	5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	114 tok/s	Runs great
Magistral Small 1.2 (24B)Mistral AI · 24B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	2.1 tok/s	Very slowover memory limit	68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slowheavy RAM offload
DeepSeek Coder 1.3B InstructDeepSeek · 1.4B	8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	44.3 tok/s	Runs great	12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	65.8 tok/s	Runs great
LLM4Decompile 1.3B V2LLM4Binary · 1.4B	8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	44.3 tok/s	Runs great	12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	65.8 tok/s	Runs great
surya ocr 2.5Datalab · 0.7B	14Basic quality (D) · 14 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	43.6 tok/s	Runs great	15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	59.3 tok/s	Runs great
SmolLM2 135M InstructHugging Face · 0.1B	4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	65.7 tok/s	Runs great	4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	315 tok/s	Runs great
Mistral Small 22B (2409)Mistral AI · 22B	37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	2.3 tok/s	Very slowover memory limit	45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	5.6 tok/s	Slowheavy RAM offload
Gemma 2B ITGoogle · 2.5B	8Basic quality (D) · 8 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	23.8 tok/s	Runs well	11Basic quality (D) · 11 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	48.2 tok/s	Runs great
DeepSeek R1 Distill Qwen 1.5BDeepSeek · 1.8B	15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	33.6 tok/s	Runs great	22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	49.9 tok/s	Runs great
Qwen2.5-Math 1.5B InstructAlibaba · 1.5B	14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	38.8 tok/s	Runs great	20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	57.7 tok/s	Runs great
Gemma 3 27B ITGoogle · 27B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	1.9 tok/s	Very slowfar over memory limit	68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slowheavy RAM offload
Falcon-H1 Tiny 90M InstructTII · 0.1B	3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	94.9 tok/s	Runs great	3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	455 tok/s	Runs great
d1 2BLiquid AI · 1.9B	14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	31.1 tok/s	Runs great	21Basic quality (D) · 21 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	46.2 tok/s	Runs great
SmolLM 135M InstructHugging Face · 0.1B	2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	65.7 tok/s	Runs great	2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	315 tok/s	Runs great
Devstral Small 2505Mistral AI · 24B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	2.2 tok/s	Very slowover memory limit	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	4.0 tok/s	Slowheavy RAM offload
Granite 4.0 H Small (32B-A9B)IBM · 32B (9.0B active)	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	5.5 tok/s	Slowfar over memory limit	58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	10.1 tok/s	Slowheavy RAM offload
SenseNova 6.8 Code2VideoSenseTime · 27B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	1.9 tok/s	Very slowover memory limit	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	3.5 tok/s	Slowheavy RAM offload
Terminal RSI 27BTeriminal-RSI · 28B	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	1.8 tok/s	Very slowfar over memory limit	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	3.4 tok/s	Slowheavy RAM offload
Codestral 22BMistral AI · 22B	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	2.3 tok/s	Very slowover memory limit	52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	5.6 tok/s	Slowheavy RAM offload
Qwen2.5 Coder 0.5B InstructAlibaba · 0.5B	7Basic quality (D) · 7 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.	51.5 tok/s	Runs great	8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	83.5 tok/s	Runs great
Falcon 7B InstructTII · 7.2B	4Basic quality (D) · 4 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.	8.3 tok/s	Slow	4Basic quality (D) · 4 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	39.7 tok/s	Runs great
StarCoder2 3BBigCode · 3.0B	12Basic quality (D) · 12 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.	19.7 tok/s	Runs well	17Basic quality (D) · 17 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	39.9 tok/s	Runs great
LLM4Decompile 22B V2LLM4Binary · 22B	40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.	2.3 tok/s	Very slowover memory limit	48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.	5.6 tok/s	Slowheavy RAM offload
Gemma 2 27B ITGoogle · 27B	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.	1.9 tok/s	Very slowfar over memory limit	52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	4.5 tok/s	Slowheavy RAM offload
CodeGemma 2BGoogle · 2.5B	9Basic quality (D) · 9 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.	23.8 tok/s	Runs well	13Basic quality (D) · 13 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	48.2 tok/s	Runs great
CogVLM2 Llama 3 Chat 19BZhipu AI · 20B	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking.	2.6 tok/s	Very slowfar over memory limit	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen1.5 14B ChatAlibaba · 14B	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 39/100 (quality, fit, speed and memory use) sets the ranking.	3.6 tok/s	Slowfar over memory limit	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Code Llama 13B InstructMeta · 13B	25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 34/100 (quality, fit, speed and memory use) sets the ranking.	3.1 tok/s	Slowfar over memory limit	21Basic quality (D) · 21 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 2/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
gpt-oss 120BOpenAI · 117B (5.1B active)	84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kimi K3.1 Pro (2.8T)Moonshot AI · 2,780B (104.0B active)	69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V4.1 Flash 1017 (552B-A16B)DeepSeek · 763B (16.0B active)	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-5.3 (753B-A40B)Zhipu AI · 753B (40.0B active)	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3.8 Max (2.4T-A95B)Alibaba · 2,446B (95.0B active)	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kimi K3 (2.8T)Moonshot AI · 2,780B (104.0B active)	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
LongCat 2.5 (1.6T-A48B)Meituan · 1,780B (48.0B active)	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiMo V2.6 Pro (1T-A42B)Xiaomi · 1,024B (42.0B active)	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ling 3.1 Flash 1019 (560B-A25B)Ant Group (inclusionAI) · 572B (25.0B active)	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax M3.1 Flash (428B-A23B)MiniMax · 429B (23.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Nex N2.6 (321B-A18B)Nex-AGI · 321B (18.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Step 5 Preview (600B-A27B)StepFun · 604B (27.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-5.3 Flash (320B-A18B)Zhipu AI · 321B (18.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiMo V2.6 Flash (310B-A15B)Xiaomi · 311B (15.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V4.1 Flash (552B-A16B)DeepSeek · 763B (16.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mistral Large 4 (1T-A52B)Mistral AI · 1,053B (52.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3.8 Flash Next (125B-A6B)Alibaba · 180B (6.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-5.2 (753B-A40B)Zhipu AI · 753B (40.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Hunyuan Hy4 Preview (780B)Tencent · 780B (55.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Muse Spark Open (816B-A42B)Meta · 822B (42.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V4 Pro (1.6T-A49B)DeepSeek · 1,651B (49.0B active)	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kimi K2.6 (1T)Moonshot AI · 1,027B (32.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Nanbeige4.3 33BBOSS Zhipin (Nanbeige) · 33B	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	4.7 tok/s	Slowheavy RAM offload
Ling 3.1 Flash (560B-A25B)Ant Group (inclusionAI) · 572B (25.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiMo V2.5 Pro (1T-A42B)Xiaomi · 1,023B (42.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Intern-S2 (397B-A17B)Shanghai AI Lab · 403B (17.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-5.1 (744B-A40B)Zhipu AI · 754B (40.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Nex N2 Pro (397B-A17B)Nex-AGI · 397B (17.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V4 Pro (April 2026)DeepSeek · 1,599B (49.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Naive N0.5 Flash (309B-A15.5B)Naive AI · 309B (15.5B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3.5 397B-A17BAlibaba · 403B (17.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax M3 (427B)MiniMax · 427B (25.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-5 (744B-A40B)Zhipu AI · 754B (40.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
K2-Horizon (375B-A23B)Institute of Foundation Models · 379B (23.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Hunyuan Hy3 (295B-A21B)Tencent · 299B (21.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Beam 1.0 (501B-A23B)Reflection AI · 501B (23.4B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kimi K2.5 (1T)Moonshot AI · 1,027B (32.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kimi K2.7 Code (1T)Moonshot AI · 1,027B (32.0B active)	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V4 Flash (284B-A13B)DeepSeek · 304B (13.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiMo V2.5 (310B-A15B)Xiaomi · 311B (15.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ornith 1.5 397B-A17BDeepReinforce · 403B (17.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Nemotron 3.5 Super VL 120B-A12BNVIDIA · 124B (12.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V3.2 (671B)DeepSeek · 685B (37.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Inkling Small (276B-A12B)Thinking Machines Lab · 266B (12.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-4.7 (355B-A32B)Zhipu AI · 358B (32.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Inkling (975B-A41B)Thinking Machines Lab · 952B (41.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax M2.7 (230B-A10B)MiniMax · 229B (10.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3.6 35B-A3BAlibaba · 36B (3.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	36.5 tok/s	Slowheavy RAM offload
Step 3.7 Flash (198B-A11B)StepFun · 201B (11.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniCPM-V 4.7 35B-A3BOpenBMB · 35B (3.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	36.6 tok/s	Slowheavy RAM offload
Nemotron 3 Ultra 550B-A55BNVIDIA · 561B (55.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek R1 0528 (671B)DeepSeek · 685B (37.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kimi K2 Thinking (1T)Moonshot AI · 1,026B (32.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3.5 35B-A3BAlibaba · 36B (3.0B active)	65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	78Great quality (A) · 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	36.5 tok/s	Slowheavy RAM offload
Ornith 1.5 122B-A10BDeepReinforce · 125B (10.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V3.2 ExpDeepSeek · 685B (37.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ling 2.6 1TAnt Group (inclusionAI) · 1,025B (63.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax M2.5 (230B-A10B)MiniMax · 229B (10.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Nemotron 3.5 Bolt 74B-A9BNVIDIA · 75B (9.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek R1 (671B)DeepSeek · 671B (37.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V4 Flash (April 2026)DeepSeek · 291B (13.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Nex N2 Mini (35B-A3B)Nex-AGI · 35B (3.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	36.7 tok/s	Slowheavy RAM offload
Qwen3.5 122B-A10BAlibaba · 125B (10.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Step 3.5 Flash (196B-A11B)StepFun · 199B (11.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V3.1 TerminusDeepSeek · 685B (37.0B active)	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V3.1 (671B)DeepSeek · 685B (37.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Gemma 4 31BGoogle · 31B	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	5.0 tok/s	Slowheavy RAM offload
KAT-Coder V2.5 Dev (35B-A3B)Kuaishou · 35B (3.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	36.7 tok/s	Slowheavy RAM offload
Kimi K2 Instruct 0905 (1T)Moonshot AI · 1,026B (32.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kolibri 1 (78B-A3.5B)Aleph Alpha · 78B (3.5B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax M2.1 (230B-A10B)MiniMax · 229B (10.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3 235B-A22B Thinking 2507Alibaba · 235B (22.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3 Coder 480B-A35BAlibaba · 480B (35.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Sage-A1.5 35B-A3BInternScience · 36B (3.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	36.5 tok/s	Slowheavy RAM offload
MiMo V2 Flash (309B-A15B)Xiaomi · 310B (15.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Flint (40B-A2.5B)Recursive · 40B (2.5B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	53.5 tok/s	Slowheavy RAM offload
Ling 1TAnt Group (inclusionAI) · 1,000B (50.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mistral Large 3 (675B)Mistral AI · 675B (41.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ornith 1.5 35B-A3BDeepReinforce · 36B (3.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	36.5 tok/s	Slowheavy RAM offload
Qwen3 235B-A22B Instruct 2507Alibaba · 235B (22.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Solar Open 2 250BUpstage · 250B (15.0B active)	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Apodex 1.1 Mini (35B-A3B)Apodex · 36B (3.0B active)	62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	74Great quality (A) · 74 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	36.5 tok/s	Slowheavy RAM offload
Kimi K2 Instruct (July 2025)Moonshot AI · 1,026B (32.0B active)	62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax M2 (230B-A10B)MiniMax · 229B (10.0B active)	62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ornith 1.0 35B-A3BDeepReinforce · 36B (3.0B active)	62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	74Great quality (A) · 74 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	36.5 tok/s	Slowheavy RAM offload
Qwen3-VL 235B-A22B InstructAlibaba · 236B (22.0B active)	62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Command A+ (218B-A25B)Cohere · 219B (25.0B active)	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
K-EXAONE 236B-A23BLG AI Research · 237B (23.0B active)	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
LongCat Flash Chat (560B)Meituan · 562B (27.0B active)	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Nemotron 3 Super 120B-A12BNVIDIA · 124B (12.0B active)	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3 235B-A22BAlibaba · 235B (22.0B active)	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Agents-A1 35B-A3BInternScience · 35B (3.0B active)	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	36.7 tok/s	Slowheavy RAM offload
Qwen AgentWorld 35B-A3BAlibaba · 35B (3.0B active)	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	73Great quality (A) · 73 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	36.7 tok/s	Slowheavy RAM offload
Devstral 2 (123B)Mistral AI · 125B	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
InternVL3.5 241B-A28BShanghai AI Lab · 241B (28.0B active)	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Laguna S 2.1 (118B-A8B)Poolside · 118B (8.0B active)	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Llama 3.1 Nemotron Ultra 253B v1NVIDIA · 253B	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3 Coder Next (80B-A3B)Alibaba · 80B (3.0B active)	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3-Next 80B-A3B ThinkingAlibaba · 81B (3.0B active)	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Baichuan M3 235B-A22BBaichuan · 235B (22.0B active)	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-4.6 (355B-A32B)Zhipu AI · 357B (32.0B active)	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3 Next 80B-A3B InstructAlibaba · 81B (3.0B active)	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V3 0324 (671B)DeepSeek · 685B (37.0B active)	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-4.6VZhipu AI · 108B (12.0B active)	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax-M1 80KMiniMax · 456B (45.9B active)	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Command A Reasoning (111B)Cohere · 111B	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
EXAONE 4.5 33BLG AI Research · 34B	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	3.6 tok/s	Slowheavy RAM offload
GLM-4.5 (355B-A32B)Zhipu AI · 358B (32.0B active)	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Laguna XS 2.1 (33B-A3B)Poolside · 33B (3.0B active)	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	45.2 tok/s	Slowheavy RAM offload
Ling 3.0 Flash (127B-A5B)Ant Group (inclusionAI) · 127B (5.1B active)	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax-M1 40KMiniMax · 456B (45.9B active)	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mistral Small 4 (119B-A6B)Mistral AI · 119B (6.0B active)	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mistral Medium 3.5 (128B)Mistral AI · 128B	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Trinity Large Thinking (400B-A13B)Arcee AI · 399B (13.0B active)	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Command A (111B)Cohere · 111B	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
ERNIE 4.5 300B-A47BBaidu · 300B (47.0B active)	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-4.5VZhipu AI · 108B (12.0B active)	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Hermes 4 405BNous Research · 406B	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kimi Dev 72BMoonshot AI · 73B	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ling 2.6 Flash (104B-A7.4B)Ant Group (inclusionAI) · 107B (7.4B active)	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Llama 3.3 Nemotron Super 49B v1.5NVIDIA · 50B	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mistral Large 2.1 (2411)Mistral AI · 123B	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Seed-OSS 36B InstructByteDance · 36B	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Solar Open 100BUpstage · 103B (12.0B active)	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Hermes 4.3 36BNous Research · 36B	55Good quality (B) · 55 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	55Good quality (B) · 55 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-4.5 Air (106B-A12B)Zhipu AI · 110B (12.0B active)	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mistral Large 2 (123B)Mistral AI · 123B	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
NorthCUA Probeipo-ai · 397B (17.0B active)	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Pixtral Large 124B (2411)Mistral AI · 124B	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3-VL 32B InstructAlibaba · 33B	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
QwQ 32BAlibaba · 33B	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ring Flash 2.0 (100B-A6B)Ant Group (inclusionAI) · 103B (6.1B active)	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
dots.llm1 Instructrednote · 143B (14.0B active)	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Granite 4.2 30BIBM · 29B	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	5.3 tok/s	Slowheavy RAM offload
InternVL3.5 38BShanghai AI Lab · 38B	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ling Flash 2.0 (100B-A6B)Ant Group (inclusionAI) · 103B (6.1B active)	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax-Text-01MiniMax · 456B (45.9B active)	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Olmo 3.1 32B ThinkAi2 · 32B	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slowheavy RAM offload
OpenReasoning Nemotron 32BNVIDIA · 33B	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3 32BAlibaba · 33B	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3-VL 30B-A3B ThinkingAlibaba · 31B (3.3B active)	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	33.6 tok/s	Slowheavy RAM offload
Skywork OR1 32BSkywork · 33B	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Tülu 3 405BAi2 · 406B	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V3 (671B)DeepSeek · 671B (37.0B active)	52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Sarvam 105BSarvam AI · 106B (10.0B active)	52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek Coder V2 (236B)DeepSeek · 236B (21.0B active)	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
EXAONE Deep 32BLG AI Research · 32B	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
GLM-4 32B 0414Zhipu AI · 33B	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slowheavy RAM offload
GLM-Z1 32B (0414)Zhipu AI · 33B	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slowheavy RAM offload
Hermes 4 70BNous Research · 71B	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Hunyuan A13B (80B-A13B)Tencent · 80B (13.0B active)	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
InternVL3 78BShanghai AI Lab · 78B	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kimi Linear 48B-A3BMoonshot AI · 49B (3.0B active)	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	44.6 tok/s	Slowheavy RAM offload
Leanstral 119B-A6BMistral AI · 119B (6.5B active)	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Llama 3.3 70B InstructMeta · 71B	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3-VL 30B-A3B InstructAlibaba · 31B (3.3B active)	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	33.6 tok/s	Slowheavy RAM offload
Qwen2.5 72B InstructAlibaba · 73B	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen2.5-VL 72B InstructAlibaba · 73B	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek Coder V2 Instruct (0614)DeepSeek · 236B (21.0B active)	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek R1 Distill Llama 70BDeepSeek · 71B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek R1 Distill Qwen 32BDeepSeek · 33B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V2.5 1210DeepSeek · 236B (21.0B active)	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
EXAONE 4.0 32BLG AI Research · 32B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	3.9 tok/s	Slowheavy RAM offload
Granite 4.1 30BIBM · 29B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	5.4 tok/s	Slowheavy RAM offload
Llama 3.1 405B InstructMeta · 406B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Llama 4 Maverick (17B x 128E)Meta · 402B (17.0B active)	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Llama 4 Scout 17B-16EMeta · 109B (17.0B active)	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Olmo 3.1 32B InstructAi2 · 32B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slowheavy RAM offload
Tülu 3 70BAi2 · 71B	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Baichuan M2 32BBaichuan · 33B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Command R+ (08-2024) 104BCohere · 104B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek V2.5DeepSeek · 236B (21.0B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Groot 65B A6B 261023GrootTeam · 118B (6.0B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
InternVL3 38BShanghai AI Lab · 38B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Jamba 1.5 LargeAI21 Labs · 399B (94.0B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Llama 3.1 70B InstructMeta · 71B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
OpenHands LM 32BAll Hands AI · 33B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen2.5 32B InstructAlibaba · 33B	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen3-Omni 30B-A3B InstructAlibaba · 35B (3.3B active)	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	42.2 tok/s	Slowheavy RAM offload
Hermes 3 (Llama 3.1 70B)Nous Research · 71B	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ling 3.0 flash SanteAnt Group (inclusionAI) · 127B (5.1B active)	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
QVQ 72B PreviewAlibaba · 73B	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen2.5-VL 32B InstructAlibaba · 33B	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen2.5 Coder 32B InstructAlibaba · 33B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Apertus v1.5 70BSwiss AI · 72B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Llama 3.1 Nemotron 70B InstructNVIDIA · 71B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Llama 3.2 90B Vision InstructMeta · 89B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
QwQ 32B PreviewAlibaba · 33B	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Command R+ 104B (04-2024)Cohere · 104B	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Falcon-H1 34B InstructTII · 34B	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	4.7 tok/s	Slowheavy RAM offload
Qwen2 72B InstructAlibaba · 73B	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen2-VL 72B InstructAlibaba · 73B	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Axion 1.0 35BAxion · 35B (3.0B active)	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	36.7 tok/s	Slowheavy RAM offload
Command R (08-2024) 32BCohere · 32B	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	4.9 tok/s	Slowheavy RAM offload
DeepSeek V2 ChatDeepSeek · 236B (21.0B active)	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Apertus 70B InstructSwiss AI · 71B	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Aya Expanse 32BCohere · 32B	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	4.9 tok/s	Slowheavy RAM offload
Aya Vision 32BCohere · 33B	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	4.8 tok/s	Slowheavy RAM offload
OLMo 2 32B InstructAi2 · 32B	42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slowheavy RAM offload
NorthCUA Minibeipo-ai · 35B (3.0B active)	41Fair quality (C) · 41 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	49Fair quality (C) · 49 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	36.7 tok/s	Slowheavy RAM offload
Llama 3 70B InstructMeta · 71B	40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen2.5-Math 72B InstructAlibaba · 73B	40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Phi-3.5 MoE (42B-A6.6B)Microsoft · 42B (6.6B active)	39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	39Basic quality (D) · 39 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mixtral 8x22B InstructMistral AI · 141B (39.0B active)	38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Command R 35B (v01)Cohere · 35B	37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Nous Hermes 2 Mixtral 8x7B DPONous Research · 47B (12.9B active)	36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen1.5 110B ChatAlibaba · 111B	36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Aya 23 35BCohere · 35B	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Grok-1 (314B)xAI · 314B (86.0B active)	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Jamba 1.5 MiniAI21 Labs · 52B (12.0B active)	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mixtral 8x7B InstructMistral AI · 47B (12.9B active)	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen2 57B-A14B InstructAlibaba · 57B (14.0B active)	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Snowflake Arctic InstructSnowflake · 479B (17.0B active)	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Yi 1.5 34B Chat01.AI · 34B	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.	4.6 tok/s	Slowheavy RAM offload
Code Llama 70B InstructMeta · 69B	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek Coder 33B InstructDeepSeek · 33B	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
LLaVA 1.6 34BLLaVA · 35B	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 42/100 (quality, fit, speed and memory use) sets the ranking.	4.6 tok/s	Slowheavy RAM offload
Qwen1.5 32B ChatAlibaba · 33B	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
DeepSeek LLM 67B ChatDeepSeek · 67B	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Granite Code 34B InstructIBM · 34B	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 38/100 (quality, fit, speed and memory use) sets the ranking.	3.7 tok/s	Slowheavy RAM offload
Yi 34B Chat01.AI · 34B	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 41/100 (quality, fit, speed and memory use) sets the ranking.	4.6 tok/s	Slowheavy RAM offload
Code Llama 34B InstructMeta · 34B	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	26Basic quality (D) · 26 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 36/100 (quality, fit, speed and memory use) sets the ranking.	4.6 tok/s	Slowheavy RAM offload
Llama 2 70B ChatMeta · 69B	25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Falcon 180B ChatTII · 180B	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Falcon 40B InstructTII · 42B	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 1/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 33/100 (quality, fit, speed and memory use) sets the ranking.	3.8 tok/s	Slowheavy RAM offload

Image models

54 of 61 image models run on the Intel Xeon E-2100 / E-2200 NAS (16 GB). 12.0 GB usable for models.

43 of 61 image models run on the Intel Arc A380. 6.0 GB usable for models.

Model	QualityXeon E-2100 / E-2200 NAS	SpeedXeon E-2100 / E-2200 NAS	RunsXeon E-2100 / E-2200 NAS	QualityArc A380	SpeedArc A380	RunsArc A380
	Intel Xeon E-2100 / E-2200 NAS (16 GB)			Intel Arc A380
FLUX 3 Image KleinBlack Forest Labs · 5.6B	84Great quality (A) · 84 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	13.9 min	Very slow	84Great quality (A) · 84 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	73 s	Slowtight fit · encoder in RAM
FLUX.2 [klein] 4BBlack Forest Labs · 3.9B	76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	1.8 min	Slow	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	13 s	Runs greatencoder in RAM
Z-Image TurboAlibaba (Tongyi) · 6.2B	78Great quality (A) · 78 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	6.1 min	Very slow	82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	30 s	Runs wellCPU offload
ERNIE-Image TurboBaidu · 8.0B	79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	7.8 min	Very slowtight fit	83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	38 s	Runs welltight fit · CPU offload
Z-ImageAlibaba (Tongyi) · 6.2B	76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	41.4 min	Very slow	80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	3.4 min	Very slowCPU offload
ERNIE-ImageBaidu · 8.0B	87Excellent quality (A+) · 87 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	53.9 min	Very slowtight fit · encoder unloaded after use	87Excellent quality (A+) · 87 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	5.6 min	Very slowtight fit · CPU offload
FLUX.2 [klein] 9BBlack Forest Labs · 9.1B	78Great quality (A) · 78 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	4.6 min	Very slowencoder unloaded after use	82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	21 s	Runs welltight fit · CPU offload
MOSS-Image SFTOpenMOSS · 5.1B	81Great quality (A) · 81 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	34.3 min	Very slowencoder unloaded after use	76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.0 min	Very slowtight fit · encoder in RAM
FLUX.1 Krea [dev]Black Forest Labs · 11.9B	76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	30.7 min	Very slowencoder unloaded after use	76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	3.2 min	Very slowtight fit · CPU offload
Juggernaut XL v9RunDiffusion · 2.6B	62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	9.1 min	Very slow	60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	41 s	Slowtight fit
RealVisXL V5.0SG161222 · 2.6B	62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	9.1 min	Very slow	60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	41 s	Slowtight fit
DreamShaper XL v2 TurboLykon · 2.6B	58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	2.6 min	Very slow	56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	12 s	Runs welltight fit
Fibo 2.0 DevBria AI · 8.1B	93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	59.3 min	Very slowCPU offload	93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
FLUX.1 [dev]Black Forest Labs · 11.9B	74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	28.1 min	Very slowencoder unloaded after use	74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	2.9 min	Very slowCPU offload
SDXL-Lightning (4-step)ByteDance · 2.6B	54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	85 s	Slow	52Fair quality (C) · 52 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	6.4 s	Runs welltight fit
Animagine XL 4.0Cagliostro Lab · 2.6B	60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	8.5 min	Very slow	58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	38 s	Runs welltight fit
Illustrious XL v0.1OnomaAI · 2.6B	60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	8.5 min	Very slow	58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	38 s	Runs welltight fit
Playground v2.5Playground AI · 2.6B	60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	15.2 min	Very slow	58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	69 s	Slowtight fit
Lumina-Image 2.0Shanghai AI Lab · 2.6B	60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	12.8 min	Very slow	60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	67 s	Slowencoder in RAM
Iris-3BSperidlabs · 3.0B	72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	34.1 min	Very slowencoder unloaded after use	72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	3.5 min	Very slowCPU offload
Krea 2 TurboKrea · 12.8B	90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	10.2 min	Very slowCPU offload	90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ideogram 4Ideogram · 18.6B	90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	59.3 min	Very slowCPU offload	85Excellent quality (A+) · 85 of 100: how capable the model is at NF4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Chroma1-HDLodestone Rock · 8.9B	72Great quality (A) · 72 of 100: how capable the model is at NF4, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	46.8 min	Very slowencoder unloaded after use	76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	4.0 min	Very slowtight fit · CPU offload
Pony Diffusion V6 XLPurpleSmartAI · 2.6B	58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	7.7 min	Very slow	56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	35 s	Runs welltight fit
OmniGen2BAAI (VectorSpaceLab) · 4.0B	64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	29.2 min	Very slowtight fit	64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	3.2 min	Very slowCPU offload
FLUX.1 Kontext [dev]Black Forest Labs · 11.9B	76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	56.2 min	Very slowtight fit · encoder unloaded after use	76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	5.8 min	Very slowtight fit · CPU offload
Animagine XL 3.1Cagliostro Lab · 2.6B	56Good quality (B) · 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	8.5 min	Very slow	54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	38 s	Runs welltight fit
FLUX.1 [schnell]Black Forest Labs · 11.9B	65Good quality (B) · 65 of 100: how capable the model is at NF4, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	4.7 min	Very slowencoder unloaded after use	65Good quality (B) · 65 of 100: how capable the model is at NF4, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	29 s	Runs wellCPU offload
CogView4 6BZhipu AI · 6.4B	68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	34.5 min	Very slowencoder unloaded after use	68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	3.6 min	Very slowCPU offload
Stable Diffusion XL 1.0Stability AI · 2.6B	55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	9.1 min	Very slow	53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	41 s	Slowtight fit
KolorsKuaishou · 2.6B	58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	7.3 min	Very slowtight fit	58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	38 s	Runs wellencoder in RAM
FLUX.2 [dev]Black Forest Labs · 32.2B	85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	1.9 h	Very slowCPU offload	85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Stable Diffusion 3 MediumStability AI · 2.0B	50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	5.7 min	Very slow	50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	30 s	Runs wellencoder in RAM
Stable Diffusion 3.5 MediumStability AI · 2.5B	57Good quality (B) · 57 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	14.0 min	Very slowtight fit	56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	58 s	Slowencoder in RAM
Hunyuan-DiT v1.2Tencent · 1.5B	52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	12.2 min	Very slow	52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	64 s	Slowtight fit · encoder in RAM
AuraFlow v0.3fal · 6.9B	53Fair quality (C) · 53 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	34.1 min	Very slow	56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	2.8 min	Very slowCPU offload
Stable CascadeStability AI · 5.2B	50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	9.1 min	Very slow	50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	60 s	SlowCPU offload
HiDream-I1 FullHiDream.ai · 17.1B	83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	2.3 h	Very slowCPU offload	78Great quality (A) · 78 of 100: how capable the model is at NF4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
FireRed-Image-Edit 1.1rednote (FireRed Team) · 20.4B	86Excellent quality (A+) · 86 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	4.0 h	Very slowtight fit · CPU offload	80Great quality (A) · 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen-Image-Edit 2511Alibaba · 20.4B	85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	4.0 h	Very slowtight fit · CPU offload	79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
HiDream-I1 DevHiDream.ai · 17.1B	79Great quality (A) · 79 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	51.5 min	Very slowCPU offload	74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Stable Diffusion 3.5 Large TurboStability AI · 8.1B	57Good quality (B) · 57 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	4.1 min	Very slowencoder unloaded after use	60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	19 s	Runs welltight fit · CPU offload
Stable Diffusion 3.5 LargeStability AI · 8.1B	61Good quality (B) · 61 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	28.1 min	Very slowencoder unloaded after use	64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	2.2 min	Slowtight fit · CPU offload
Qwen-Image 2512Alibaba · 20.4B	83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	4.7 h	Very slowtight fit · CPU offload	83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
SDXL TurboStability AI · 2.6B	39Basic quality (D) · 39 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	16 s	Runs well	39Basic quality (D) · 39 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	1.4 s	Runs great
HiDream-I1 FastHiDream.ai · 17.1B	75Great quality (A) · 75 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	30.3 min	Very slowCPU offload	70Great quality (A) · 70 of 100: how capable the model is at NF4, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
FLUX.1 Fill [dev]Black Forest Labs · 11.9B	74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	29.7 min	Very slowCPU offload	74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
SD TurboStability AI · 0.9B	28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	9.7 s	Runs great	28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	0.8 s	Runs great
Kandinsky 3Sber AI · 3.0B	48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	19.2 min	Very slowencoder unloaded after use	48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	2.0 min	SlowCPU offload
Realistic Vision V5.1SG161222 · 0.9B	42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	2.4 min	Very slow	42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	12 s	Runs great
LCM DreamShaper v7Tsinghua (LCM) · 0.9B	30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	30 s	Runs well	30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	2.5 s	Runs great
DreamShaper 8Lykon · 0.9B	40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	2.4 min	Very slow	40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	12 s	Runs great
PixArt-Σ XL 1024PixArt · 0.6B	50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.	3.2 min	Very slowencoder unloaded after use	50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	16 s	Runs wellencoder in RAM
Stable Diffusion 1.5Stability AI / Runway · 0.9B	35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	2.4 min	Very slow	35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	12 s	Runs great
MiniMax H3 ImageMiniMax · 33.1B	91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Cosmos3-Super Text2Image (64B)NVIDIA · 64.0B	90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
HunyuanImage 3.0 (80B-A13B)Tencent · 83.0B	88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen-Image 2.1Alibaba · 7.1B	86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen-ImageAlibaba · 20.4B	86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Ming-Image 0.1 DesignAnt Group (inclusionAI) · 6.2B	84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Qwen-Image 2.1 TurboAlibaba · 7.1B	82Great quality (A) · 82 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	82Great quality (A) · 82 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run

Video models

14 of 43 video models run on the Intel Xeon E-2100 / E-2200 NAS (16 GB). 12.0 GB usable for models.

3 of 43 video models run on the Intel Arc A380. 6.0 GB usable for models.

Model	QualityXeon E-2100 / E-2200 NAS	SpeedXeon E-2100 / E-2200 NAS	RunsXeon E-2100 / E-2200 NAS	QualityArc A380	SpeedArc A380	RunsArc A380
	Intel Xeon E-2100 / E-2200 NAS (16 GB)			Intel Arc A380
LTX-Video 2BLightricks · 1.9B	49Fair quality (C) · 49 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	45.6 min	Very slowtight fit	49Fair quality (C) · 49 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	5.0 min	Runs wellCPU offload
Bernini-R A14B (video generation + editing)ByteDance · 14.3B	83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	28.5 h	Very slowtight fit · CPU offload	83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
HY-OmniWeaving (8.3B, unified video)Tencent · 8.3B	74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.	21.0 h	Very slowCPU offload	74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.1 T2V 1.3BAlibaba · 1.3B	55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	8.5 h	Very slowencoder unloaded after use	53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.1 VACE 1.3BAlibaba · 2.2B	55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	9.2 h	Very slowencoder unloaded after use	53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
CogVideoX 5BZhipu AI · 5.0B	53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	5.7 h	Very slowencoder unloaded after use	53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
CogVideoX 5B I2VZhipu AI · 5.6B	53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	5.7 h	Very slowencoder unloaded after use	53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
CogVideoX 2BZhipu AI · 1.7B	44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	1.8 h	Very slowtight fit	44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.	11.9 min	Runs wellCPU offload
Wan 2.2 S2V 14B (speech to video)Alibaba · 16.3B	70Great quality (A) · 70 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.	52.7 h	Very slowCPU offload	70Great quality (A) · 70 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kandinsky 6.0 Video Lite (3B, video + audio)Sber AI · 3.2B	68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	13.6 h	Very slowCPU offload	68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
FramePack I2V (HunyuanVideo 13B)lllyasviel · 12.9B	72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	15.8 h	Very slowtight fit · CPU offload	72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.1 I2V 14B 480PAlibaba · 14.0B	73Great quality (A) · 73 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.	45.5 h	Very slowtight fit · CPU offload	68Good quality (B) · 68 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
SkyReels V2 Diffusion Forcing 1.3BSkywork · 1.4B	50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.	9.6 h	Very slowtight fit · encoder unloaded after use	50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
AnimateDiff v3 (SD 1.5)Shanghai AI Lab · 1.3B	35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.	40.6 min	Very slow	35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	3.4 min	Runs great
Cosmos3-Super Image2Video (64B)NVIDIA · 64.0B	88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Kandinsky 6.0 Pro (29B, video + audio)Sber AI · 30.1B	88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
MiniMax H3 (33B, video + audio)MiniMax · 20.1B	83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
HunyuanVideo 1.5 (8.3B)Tencent · 8.3B	82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
LTX-2.5 (22B, video + audio)Lightricks · 21.0B	81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
LTX-2.3Lightricks · 18.9B	79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.2 I2V A14BAlibaba · 14.0B	79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.2 T2V A14BAlibaba · 14.0B	79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
SCAIL-2 14B (character animation)Zhipu AI · 16.4B	78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
SkyReels V2 T2V 14B 720PSkywork · 14.0B	76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
LTX-2 (19B, video + audio)Lightricks · 18.9B	76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Step-Video T2V (30B)StepFun · 30.0B	74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan-Dancer 14B (music to dance)Alibaba · 17.3B	74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.1 I2V 14B 720PAlibaba · 14.0B	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.1 T2V 14BAlibaba · 14.0B	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.1 VACE 14BAlibaba · 17.3B	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.2 Animate 14BAlibaba · 17.3B	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
SkyReels V2 T2V 14B 540PSkywork · 14.0B	72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Open-Sora 2.0 (11B)HPC-AI Tech · 11.0B	70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Cosmos-Predict2 14B Video2WorldNVIDIA · 14.0B	68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
HunyuanVideo 13BTencent · 13.0B	66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
HunyuanVideo I2V 13BTencent · 13.0B	66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
LTX-Video 13B 0.9.8 (distilled)Lightricks · 13.0B	66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Wan 2.2 TI2V 5BAlibaba · 5.0B	65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Mochi 1 Preview (10B)Genmo · 10.0B	60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Cosmos 1.0 Diffusion 7B Text2WorldNVIDIA · 7.0B	60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
CogVideoX 1.5 5BZhipu AI · 5.6B	58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Cosmos-Predict2 2B Video2WorldNVIDIA · 2.0B	56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Allegro (2.8B)Rhymes AI · 2.8B	52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run

Decision models

65 of 70 decision models run on the Intel Xeon E-2100 / E-2200 NAS (16 GB). 12.0 GB usable for models.

63 of 70 decision models run on the Intel Arc A380. 6.0 GB usable for models.

Model	QualityXeon E-2100 / E-2200 NAS	SpeedXeon E-2100 / E-2200 NAS	RunsXeon E-2100 / E-2200 NAS	QualityArc A380	SpeedArc A380	RunsArc A380
	Intel Xeon E-2100 / E-2200 NAS (16 GB)			Intel Arc A380
OpenAI Privacy FilterOpenAI · 1.4B (50M active)	78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	97 ms	Runs great	78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	7.5 ms	Runs great
Jina Reranker v3.5Jina AI · 600M	86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	2.1 s	Very slow	86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.	87 ms	Runs great
Qwen3 Reranker 4BAlibaba · 4.0B	90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	13.9 s	Very slow	89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	494 ms	Slow
Jina Reranker v3Jina AI · 600M	84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.	2.1 s	Very slow	84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.	87 ms	Runs great
Qwen3 Reranker 8BAlibaba · 8.2B	89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	28.3 s	Very slow	89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	1.3 s	Slowoffloads to RAM
GLiNER2.5 DecideFastino · 490M	80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	913 ms	Slow	80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.	39 ms	Runs great
zerank-2ZeroEntropy · 4.0B	88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	13.9 s	Very slow	87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.	494 ms	Slow
Skywork Reward V2 Llama 3.1 8BSkywork · 7.5B	90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	55.7 s	Very slowtight fit	90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Shieldstral 1.0 3BMistral AI · 3.9B	86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	13.4 s	Very slow	85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.	473 ms	Slow
FRIDA-DecisionsSber AI · 820M	80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	1.5 s	Very slow	80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.	62 ms	Runs great
Llama Prompt Guard 2 22MMeta · 71M	68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	267 ms	Runs well	68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.	14 ms	Runs great
Intern-Decision 4BShanghai AI Lab · 4.5B	84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	8.0 s	Very slow	83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	299 ms	Runs well
Qwen3 Reranker 0.6BAlibaba · 600M	80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	2.1 s	Very slow	80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.	87 ms	Runs great
mxbai-rerank-large v2Mixedbread · 1.5B	82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	5.3 s	Very slow	82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.	196 ms	Runs well
d1 3BLiquid AI · 3.1B	82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	5.5 s	Very slow	82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.	216 ms	Runs well
Llama Prompt Guard 2 86MMeta · 280M	76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	1.0 s	Slow	76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.	44 ms	Runs great
GLiClass ModernBERT Large v3Knowledgator · 400M	74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	746 ms	Slow	74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	32 ms	Runs great
ms-marco MiniLM L6 v2Sentence Transformers · 23M	52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	89 ms	Runs great	52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.	7.3 ms	Runs great
zerank-1 smallZeroEntropy · 1.7B	80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	6.0 s	Very slow	80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	219 ms	Runs well
GTE Reranker ModernBERT BaseAlibaba · 150M	70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	561 ms	Slow	70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	25 ms	Runs great
mxbai-rerank-base v2Mixedbread · 490M	76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	1.7 s	Very slow	76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.	72 ms	Runs great
GLiClass Base v3Knowledgator · 190M	66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	356 ms	Runs well	66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	17 ms	Runs great
DeBERTa v3 Large Zero-shot v2.0Moritz Laurer · 440M	72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	820 ms	Slow	72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.	35 ms	Runs great
LemonDecider 4BAnt Group (inclusionAI) · 4.5B	80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	8.4 s	Very slow	80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Skywork Reward V2 Qwen3 8BSkywork · 7.6B	84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	56.2 s	Very slowtight fit	84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
ModernBERT Large Zero-shot v2.0Moritz Laurer · 400M	70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	746 ms	Slow	70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	32 ms	Runs great
DeBERTa v3 Base Zero-shot v2.0Moritz Laurer · 180M	64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	338 ms	Runs well	64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	17 ms	Runs great
Granite Embedding Reranker English R2IBM · 150M	68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	561 ms	Slow	68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.	25 ms	Runs great
BGE Reranker v2 GemmaBAAI · 2.5B	78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	8.7 s	Very slow	78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	317 ms	Runs well
Qwen3Guard Gen 8BAlibaba · 8.2B	85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	29.9 s	Very slow	81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	1.3 s	Slowoffloads to RAM
GLiNER2 Base v1Fastino · 210M	68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	394 ms	Runs well	68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	19 ms	Runs great
Jina Reranker v2 Base MultilingualJina AI · 280M	70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	1.0 s	Slow	70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	44 ms	Runs great
Qwen3Guard Gen 4BAlibaba · 4.4B	84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	16.5 s	Very slow	83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.	723 ms	Slow
GLiGuard 300MFastino · 210M	68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	783 ms	Slow	68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.	34 ms	Runs great
LayaConvai Innovations · 420M	68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	783 ms	Slow	68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.	34 ms	Runs great
DuoGuard 0.5BDuoGuard · 490M	72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	1.8 s	Very slow	72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.	74 ms	Runs great
BGE Reranker v2 M3BAAI · 570M	72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	2.1 s	Very slow	72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.	85 ms	Runs great
ProtectAI DeBERTa v3 Prompt Injection v2Protect AI · 180M	66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	672 ms	Slow	66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	30 ms	Runs great
Llama 3.1 Nemotron Safety Guard 8B v3NVIDIA · 8.0B	83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	29.3 s	Very slow	79Great quality (A) · 79 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	1.2 s	Slowoffloads to RAM
Granite Guardian 4.1 8BIBM · 8.4B	83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	30.6 s	Very slow	79Great quality (A) · 79 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.	1.3 s	Slowoffloads to RAM
Skywork Reward V2 Qwen3 0.6BSkywork · 600M	72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	4.5 s	Very slow	72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.	152 ms	Runs well
WildGuard 7BAi2 · 7.3B	80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	27.1 s	Very slow	76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	1.1 s	Slowtight fit
NuExtract 2.0 4BNuMind · 3.8B	76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	11.3 s	Very slow	75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.	961 ms	Slow
Granite Guardian 3.3 8BIBM · 8.2B	79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	29.8 s	Very slow	76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.	1.3 s	Slowoffloads to RAM
HHEM 2.1 OpenVectara · 110M	62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	820 ms	Slow	62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	35 ms	Runs great
Laya MultilingualConvai Innovations · 320M	60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	598 ms	Slow	60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	27 ms	Runs great
Qwen3Guard Gen 0.6BAlibaba · 750M	74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	2.8 s	Very slow	73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.	126 ms	Runs great
GLiNER Multi v2.1Urchade Zaratiana · 290M	62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	542 ms	Slow	62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	25 ms	Runs great
NuExtract 2.0 2BNuMind · 2.2B	70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	6.6 s	Very slow	69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	568 ms	Slow
Nemotron Content Safety Reasoning 4BNVIDIA · 4.3B	78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	51.8 s	Very slow	77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	6.2 s	Very slow
DynaGuard 8BUniversity of Maryland · 8.2B	78Great quality (A) · 78 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	79.8 s	Very slow	78Great quality (A) · 78 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	15.5 s	Very slowoffloads to RAM
BART Large MNLIMeta · 410M	56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	765 ms	Slow	56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.	33 ms	Runs great
LettuceDetect LargeKR Labs · 400M	66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	3.0 s	Very slow	66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	103 ms	Runs great
Selene 1 Mini 8BAtla · 8.0B	79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.	102.5 s	Very slow	79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	15.2 s	Very slowoffloads to RAM
CompassJudger-2 7BOpenCompass · 7.6B	78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	113.7 s	Very slow	74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	8.3 s	Very slowtight fit
ShieldGemma 9BGoogle · 9.2B	63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.	31.9 s	Very slow	60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.	1.3 s	Slowoffloads to RAM
Llama Guard 3 8BMeta · 8.0B	67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	29.3 s	Very slow	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.	1.2 s	Slowoffloads to RAM
ShieldGemma 2BGoogle · 2.6B	58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	9.1 s	Very slow	58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	330 ms	Runs well
DynaGuard 1.7BUniversity of Maryland · 2.0B	70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.	24.5 s	Very slow	69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.	2.9 s	Very slow
Llama Guard 4 12BMeta · 12B	65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.	43.8 s	Very slowtight fit	66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.	3.8 s	Very slowheavy RAM offload
Prometheus 2 7BPrometheus Eval · 7.2B	68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	108.0 s	Very slow	64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	7.9 s	Very slowtight fit
GLIDERPatronus AI · 3.8B	66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	57.0 s	Very slow	65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.	5.6 s	Very slow
Llama Guard 3 1BMeta · 1.5B	56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	5.6 s	Very slow	55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.	249 ms	Runs well
Flow Judge v0.1Flow AI · 3.8B	64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.	57.0 s	Very slow	63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.	5.6 s	Very slow
gpt-oss-safeguard 20BOpenAI · 22B (3.6B active)	76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	30.3 s	Very slowfar over memory limit	76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.	6.4 s	Very slowheavy RAM offload
GLiDE 27BFastino · 26B	86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
CompassJudger-2 32BOpenCompass · 33B	81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
gpt-oss-safeguard 120BOpenAI · 117B (5.1B active)	80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
Prometheus 2 8x7BPrometheus Eval · 47B (13B active)	70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run
ShieldGemma 27BGoogle · 27B	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.	—	Won't run	62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.	6.7 s	Very slowheavy RAM offload

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.