M4 Pro (48 GB) vs M4 Max (64 GB) vs M4 Max (128 GB) for local AI
Pick two to four systems or devices to see which runs more models, faster, for the kind of work you do.
Winner: Apple M4 Max (128 GB)
For general chat: It runs 27 more LLMs (548 vs 521) (against the Apple M4 Max (64 GB), the next best). On the other hand, the Apple M4 Max (64 GB) is about 1.4× faster on its own top models.
Side by side
All models on the M4 Pro (48 GB) All models on the M4 Max (64 GB) All models on the M4 Max (128 GB)
 Apple M4 Pro (48 GB)
Remove
|
Apple M4 Max (64 GB)
Remove
|
Apple M4 Max (128 GB)
Remove
|
|
|---|---|---|---|
| Memory | 48.0 GB unified | 64.0 GB unified | 128 GB unified |
| Usable for models | 36.0 GB | 48.0 GB | 96.0 GB |
| Memory bandwidth | 273 GB/s | 546 GB/s | 546 GB/s |
| FP16 compute | 9.2 TFLOPS | 18.4 TFLOPS | 18.4 TFLOPS |
| Launch price | — | — | — |
| Released | 2024 | 2024 | 2024 |
| LLMs that run | 492 of 622 | 521 of 622 | 548 of 622 |
| LLMs that run well | 469 | 504 | 533 |
| Image models that run | 61 of 61 | 61 of 61 | 61 of 61 |
| Video models that run | 41 of 43 | 42 of 43 | 43 of 43 |
| Decision models that run | 69 of 70 | 69 of 70 | 70 of 70 |
| Top-10 LLM average score | 100 | 100 | 99 |
| Top-10 LLM average speed | 55.5 tok/s | 89.4 tok/s | 65.5 tok/s |
| Best LLM | MiniCPM-V 4.7 35B-A3B | Qwen3.5 35B-A3B | Nemotron 3.5 Bolt 74B-A9B |
Highlighted values are the best of the 3. Model results assume 8K of context and the default macOS GPU memory limit on Macs.
LLMs
492 of 622 LLMs run on the Apple M4 Pro (48 GB). 36.0 GB usable for models.
521 of 622 LLMs run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
548 of 622 LLMs run on the Apple M4 Max (128 GB). 96.0 GB usable for models.
| Model | M4 Pro (48 GB)fit · speed · quality | M4 Max (64 GB)fit · speed · quality | M4 Max (128 GB)fit · speed · quality |
|---|---|---|---|
| MiniCPM-V 4.7 35B-A3B | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Qwen3.5 35B-A3B | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Qwen3.6 35B-A3B | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Nex N2 Mini (35B-A3B) | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Sage-A1.5 35B-A3B | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Gemma 4 26B-A4B | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.51.4 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.76.3 tok/s | Runs great92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.59.7 tok/s |
| Ornith 1.5 35B-A3B | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| GLM-4.7 Flash (30B-A3B) | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Raon-1 mini Preview (30B-A4B) | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.54.2 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.54.2 tok/s |
| Apodex 1.1 Mini (35B-A3B) | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Ornith 1.0 35B-A3B | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Flint (40B-A2.5B) | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.83.1 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.81.2 tok/s |
| Agents-A1 35B-A3B | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Qwen AgentWorld 35B-A3B | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Laguna XS 2.1 (33B-A3B) | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Nemotron Cascade 2 30B-A3B | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| DiffusionGemma 26B-A4B IT | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.57.4 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.57.4 tok/s |
| KAT-Coder V2.5 Dev (35B-A3B) | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Nemotron 3 Nano 30B-A3B | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Qwen3 30B-A3B Instruct 2507 | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s |
| Qwen3-VL 30B-A3B Thinking | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s |
| LFM2.5 24B-A2B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.48.8 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.82.4 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s |
| Qwen3 30B-A3B | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s |
| Qwen3-VL 30B-A3B Instruct | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s |
| Hunyuan Hy-Memory 30B-A3B | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Nemotron 3.5 Lightning 30B-A3B | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Qwen3.8 27B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s |
| Qwen3.6 27B | Runs well89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs well89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s |
| Sarvam 30B | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.1 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.83.5 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s |
| Qwen3.5 27B | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s |
| North Mini Code 1.0 (30B-A3B) | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Kimi Linear 48B-A3B | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Qwen3-Omni 30B-A3B Instruct | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.57.7 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s |
| Qwen3 30B-A3B Thinking 2507 | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s |
| Qwen3 Coder 30B-A3B | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s |
| TMax 27B | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s |
| Trinity Mini (26B-A3B) | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| ERNIE 4.5 21B-A3B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Fara 1.5 27B | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s |
| Nemotron 3.5 Bolt 74B-A9B | Runs welltight fit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s |
| Nanbeige4.3 33B | Runs well80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11.3 tok/s | Runs well94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s | Runs well94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s |
| UI-Mate 27B | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s |
| Kolibri 1 (78B-A3.5B) | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.95.9 tok/s | Runs welltight fit76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.127 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.81.6 tok/s |
| Axion 1.0 35B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| gpt-oss 20B | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.66.6 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| ERNIE 4.5 21B-A3B Thinking | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Maple Preview (20B-A1B) | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.71.7 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.113 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s |
| NorthCUA Mini | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Gemma 4 31B | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.11.9 tok/s | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18.5 tok/s | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.18.5 tok/s |
| Groot 19B A3B 261023 | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Kimi-VL A3B Thinking 2506 | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| LFM2 24B-A2B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.51.4 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.46.8 tok/s |
| Muse Glimmer 30B | Runs well77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.12.5 tok/s | Runs well87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.19.4 tok/s | Runs well87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.19.4 tok/s |
| Ling 3.0 Tiny (8B-A1.3B) | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s |
| Gemma 4 12B | Runs well78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.24.1 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s |
| Granite 4.0 H Small (32B-A9B) | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.28.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.38.3 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.38.3 tok/s |
| K2 Horizon 7B | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.32.1 tok/s | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s |
| Phi-3.5 MoE (42B-A6.6B) | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.38.1 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s |
| Devstral Small 2 (24B) | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s |
| Seed-OSS 36B Instruct | Runs well67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s |
| Nemotron 3.5 Bolt E4B | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s |
| SenseNova Loop 9B Preview | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s |
| Hermes 4.3 36B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s |
| EXAONE 4.5 33B | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.9 tok/s | Runs well77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs well77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s |
| Ornith 1.5 9B | Runs well73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s |
| Mellum2.1 Thinking (12B-A2.5B) | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.48.0 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s |
| Gemma 3 27B IT | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s |
| Ling Mini 2.0 (16B-A1.4B) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.75.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| Gemma 4 E4B | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s |
| Qwen3.5 9B | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s |
| Qwen3-VL 32B Instruct | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.11.2 tok/s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s |
| Ornith 1.0 9B | Runs well71Great quality (A) · 71 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s |
| Qwen3 Coder Next (80B-A3B) | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s |
| Kimi-VL A3B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| TMax 9B | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.32.2 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s |
| MedGemma 27B Text IT | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.21.4 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.21.4 tok/s |
| Llama 3.3 Nemotron Super 49B v1.5 | Slowtight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.5 tok/s | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s |
| Nanbeige4.2 3B | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.48.0 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s |
| Qwen3 32B | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Devstral Small 1.1 (24B) | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s |
| Magistral Small 2507 | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s |
| Apriel 1.5 15B Thinker | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.19.4 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s |
| Terminal RSI 27B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s |
| Granite 4.2 30B | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.12.7 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s |
| Nemotron Labs Dory 1.0 4B | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.50.3 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s |
| Mistral Small 3.2 24B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s |
| Magistral Small 2506 | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s |
| Phi-4 Reasoning Vision 15B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.19.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s |
| Hemmingway 1.5 | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.21.5 tok/s | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.21.5 tok/s |
| Granite 4.2 8B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.32.8 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s |
| Spark-X2.6 4B | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71.5 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.71.5 tok/s |
| Moonlight 16B-A3B Instruct | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s |
| Mistral Small 3.1 24B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s |
| GLM-4 32B 0414 | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s |
| Gemma 4 E2B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.58.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.58.9 tok/s |
| Qwen3 14B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.39.0 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.39.0 tok/s |
| Granite 4.1 30B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.12.9 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.20.0 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.20.0 tok/s |
| Qwen2.5 32B Instruct | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Baichuan M2 32B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Gemma 2 27B IT | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.21.2 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.21.2 tok/s |
| Olmo 3.1 32B Instruct | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s |
| EXAONE 4.0 32B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.7 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s |
| Mistral Small 3 24B (2501) | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s |
| Qwen2.5-VL 32B Instruct | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.11.2 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s |
| Hermes 4 14B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| MiMo V2.6 Distill Qwen 9B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| Qwen3.5 4B | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| Voxtral Small 24B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.11.9 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.23.8 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.23.8 tok/s |
| Dolphin Mistral 24B Venice Edition | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s |
| Nous Hermes 2 Mixtral 8x7B DPO | Runs welltight fit49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s |
| Phi-4 (14B) | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.19.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s |
| InternVL3 14B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.19.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s |
| TMax 4B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.47.6 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.69.8 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.69.8 tok/s |
| Ministral 3 14B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.20.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| InternVL3.5 8B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.33.8 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s |
| Jamba 1.5 Mini | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.22.2 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| Qwen3-VL 8B Instruct | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.32.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s |
| Agents-A1 4B | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.44.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s |
| Magistral Small 1.2 (24B) | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s |
| DeepSeek Coder V2 Lite (16B) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s |
| LFM2.5 8B-A1B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s |
| QwQ 32B | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Dolphin 3.0 R1 Mistral 24B | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s |
| Mixtral 8x7B Instruct | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s |
| MiniCPM5 2B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.58.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.53.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.53.7 tok/s |
| DeepSeek V2 Lite Chat | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s |
| Qwen2.5 14B Instruct | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.19.6 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s |
| Granite 4.0 H Tiny (7B-A1B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.52.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.85.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.85.0 tok/s |
| Solar Pro Preview 22B | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.13.0 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.26.1 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.26.1 tok/s |
| Qwen3 8B | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.35.2 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s |
| Granite 4.1 8B | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.32.8 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s |
| InternVL3.5 38B | Runs well53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.12.3 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s |
| MiniCPM-V 4.5 8B | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.33.2 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.46.0 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.46.0 tok/s |
| Falcon-H1 34B Instruct | Runs well53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.2 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.17.2 tok/s |
| OpenReasoning Nemotron 32B | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Skywork OR1 32B | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Fara 1.5 4B | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.44.1 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s |
| Mage-VL 4.7B | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s |
| Phi-4 Reasoning Plus (14B) | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s |
| Qwen2.5 14B Instruct 1M | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| MiMo-VL 7B RL | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.34.7 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s |
| Nemotron Nano 9B v2 | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.32.5 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s |
| Command R (08-2024) 32B | Runs well52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s |
| EXAONE Deep 32B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.7 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s |
| GLM-4.1V 9B Thinking | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.28.0 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s |
| Qwen2 57B-A14B Instruct | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s |
| Olmo 3.1 32B Think | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s |
| Gemma 3 12B IT | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.23.7 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| Phi-4 Reasoning (14B) | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s |
| MiMo 7B RL | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.36.9 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.51.1 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.51.1 tok/s |
| Qwen3-VL 4B Instruct | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.45.1 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s |
| SenseNova 6.8 Code2Video | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s |
| Qwen3 4B Instruct 2507 | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s |
| UI-Mate 9B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| UI-Venus 2 9B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| Aya Vision 32B | Runs well50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.3 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.4 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.4 tok/s |
| DeepSeek R1 Distill Qwen 32B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Aya Expanse 32B | Runs well50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s |
| Ministral 3 8B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.32.4 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s |
| InternVL3 8B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.36.3 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s |
| InternVL3 38B | Runs well49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.12.3 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s |
| Devstral Small 2505 | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s |
| OLMo 2 32B Instruct | Runs well50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s |
| GLM-Z1 32B (0414) | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s |
| Mistral Small 22B (2409) | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.13.0 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s |
| LFM3 1.7B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.84.9 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s |
| Granite 4.2 3B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s |
| Hermes 4 70B | Slowtight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| Llama 3.3 70B Instruct | Slowtight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| Llama 3.1 Nemotron Nano 8B v1 | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| OpenHands LM 32B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Reka Flash 3.1 (21B) | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.13.8 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.27.6 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.27.6 tok/s |
| Qwen2.5 72B Instruct | Slowtight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s |
| Qwen2.5-VL 72B Instruct | Slowtight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs well61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s |
| DeepSeek R1 Distill Llama 70B | Slowtight fit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| Tülu 3 70B | Slowtight fit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| Hunyuan 7B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.38.5 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.2 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.2 tok/s |
| Apriel Nemotron 15B Thinker | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.19.3 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.38.6 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.38.6 tok/s |
| MiniCPM4 8B | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.35.2 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s |
| AceReason Nemotron 14B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| OpenReasoning Nemotron 14B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| Qwen2.5 Coder 14B Instruct | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| Ministral 3 14B Reasoning | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.20.7 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| Qwen3 4B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s |
| Qwen2.5 7B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s |
| Olmo 3 7B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s |
| Qwen2.5 Coder 32B Instruct | Runs well56Good quality (B) · 56 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| Mage-VL-Cowork 4.7B | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s |
| Llama 3.1 70B Instruct | Slowtight fit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| Codestral 22B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.13.0 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s |
| Command R 35B (v01) | Runs welltight fit45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| Qwen3-Next 80B-A3B Thinking | Runs wellover GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s |
| Qwen2.5-VL 7B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.34.8 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s |
| Gemma 3n E4B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.50.1 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s |
| Falcon 3 10B Instruct | Runs well49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.28.0 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s |
| DeepCoder 14B Preview | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| QwQ 32B Preview | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s |
| MiniCPM-o 2.6 8B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.33.3 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.46.2 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.46.2 tok/s |
| DeepSeek R1 0528 Qwen3 8B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.35.2 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s |
| Nemotron 3 Nano 4B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.74.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.74.0 tok/s |
| Foundation-Sec 1.1 8B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| InternVL2.5 8B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.35.7 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s |
| Mistral Nemo 12B Instruct | Runs well49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.23.7 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| Hermes 3 (Llama 3.1 70B) | Slowtight fit47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| Gemma 2 9B IT | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.31.2 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.43.3 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.43.3 tok/s |
| GLM-4 9B 0414 | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| Qwen2.5 7B Instruct 1M | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s |
| Qwen3 Next 80B-A3B Instruct | Runs wellover GPU memory limit58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s |
| NorthCUA Nano | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| QVQ 72B Preview | Slowtight fit47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s |
| InternLM2.5 20B Chat | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.14.5 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s |
| Falcon-H1 7B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.38.0 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.52.8 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.8 tok/s |
| GLiDE 9B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.36.3 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s |
| LFM3-VL 2B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s |
| DeepSeek R1 Distill Qwen 14B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| Llama 3.1 Nemotron 70B Instruct | Slowtight fit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| InternLM3 8B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.32.8 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.45.5 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.45.5 tok/s |
| Apertus v1.5 70B | Slowtight fit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Runs welltight fit55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s |
| JanusCoder 14B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s |
| Hermes 3 (Llama 3.1 8B) | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Llama 3.1 8B Instruct | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Pixtral 12B (2409) | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.23.3 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.38.5 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.38.5 tok/s |
| Granite 4.1 3B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s |
| DeepSeek MoE 16B Chat | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.43.7 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.75.0 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.75.0 tok/s |
| Aya 23 35B | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| LLM4Decompile 22B V2 | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.13.0 tok/s | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s |
| Yi 1.5 34B Chat | Runs well42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s |
| MiniCPM-V 2.6 8B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.6 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.4 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.4 tok/s |
| Kimi Dev 72B | Slowtight fit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s |
| Qwen2 72B Instruct | Slowtight fit44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s |
| Qwen2-VL 72B Instruct | Slowtight fit44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs well53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s |
| Falcon 3 7B Instruct | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s |
| Qwen3 4B Thinking 2507 | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s |
| LFM2 8B-A1B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s |
| EXAONE 3.5 7.8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.36.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s |
| Granite 3.2 8B Instruct | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s |
| Granite 3.3 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s |
| Ministral 8B (2410) | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.36.0 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| GLM-4 9B Chat 1M | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.30.4 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s |
| Seed-Coder 8B Reasoning | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.0 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s |
| DeepHermes 3 Llama 3 8B Preview | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Dolphin 3.0 Llama 3.1 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Tülu 3 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Llama 3.1 Nemotron Nano 4B v1.1 | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.65.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.65.1 tok/s |
| GLM-4 9B Chat (2024) | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| Gemma 3 4B IT | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s |
| Phi-4 Multimodal Instruct | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s |
| Apertus v1.5 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.32.4 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s |
| Phi-4 Mini (3.8B) | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s |
| Phi-3 Small 8K Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.39.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s |
| OLMo 2 13B Instruct | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.21.0 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s |
| Ministral 3 3B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.76.3 tok/s |
| EXAONE Deep 7.8B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.36.9 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s |
| North Micro Vision | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.59.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.54.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.54.6 tok/s |
| MedGemma 4B IT | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s |
| LFM2.5-VL 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.47.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s |
| Phi-3 Medium 128K Instruct | Runs well43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.20.7 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s |
| Jamba Reasoning 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.45.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.42.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.42.3 tok/s |
| Granite 3.1 8B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s |
| Phi-3.5 Mini Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s |
| Seed-Coder 8B Instruct | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.0 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s |
| Dolphin 2.9.4 Llama 3.1 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Hermes 2 Pro Llama 3 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| OpenChat 3.6 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| InternVL2 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.7 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s |
| GLM-Z1 9B (0414) | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| Apertus 70B Instruct | Slowtight fit42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| Olmo 3 7B Think | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s |
| Falcon-H1R 7B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.38.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s |
| LFM2.5 2.6B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.54.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s |
| Hunyuan 4B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.69.6 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.69.6 tok/s |
| Qwen2 7B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s |
| Qwen1.5 MoE A2.7B Chat | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.45.1 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s |
| Qwen2.5-Omni 7B | Runs well43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.26.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s |
| Phi-3 Medium (14B) | Runs well43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.20.7 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s |
| Command R7B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Qwen2.5 Coder 7B Instruct | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s |
| SmolLM3 3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.47.7 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s |
| Qwen2-VL 7B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.34.8 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s |
| Qwen1.5 32B Chat | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.17.8 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.17.8 tok/s |
| Llama 3 70B Instruct | Slowtight fit40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s |
| Phi-3.5 Vision Instruct | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s |
| Aya Vision 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.33.4 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.46.4 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.46.4 tok/s |
| Llama 3 8B Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Mistral-NeMo-Minitron 8B Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.34.3 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.47.6 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.47.6 tok/s |
| Jamba2 3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s |
| LLaVA 1.6 34B | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.16.6 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.16.6 tok/s |
| OpenReasoning Nemotron 7B | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s |
| OLMoE 1B-7B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s |
| InternLM2 Chat 20B | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.14.5 tok/s | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s |
| CogVLM2 Llama 3 Chat 19B | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.14.8 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.6 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.29.6 tok/s |
| Granite 4.0 Micro (3B) | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s |
| AFM 4.5B | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.43.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Command R7B Arabic | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Qianfan-OCR 4B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s |
| Llama 3.2 11B Vision Instruct | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.27.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s |
| InternLM2.5 7B Chat | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.37.3 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s |
| Foundation-Sec 8B Reasoning | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Aya Expanse 8B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Apertus 8B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.8 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s |
| Phi-3 Mini 128K Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s |
| Qwen1.5 14B Chat | Runs well32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.20.4 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s |
| MiniCPM-V 4.7 1B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.43.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s |
| Gemma 3n E2B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Granite 3.0 8B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s |
| Mistral 7B Instruct v0.3 | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s |
| Qwen3.5 2B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.59.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.59.7 tok/s |
| OLMo 2 7B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s |
| Zamba2 7B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s |
| DeepSeek R1 Distill Llama 8B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| Unlimited OCR 2 (3B-A0.6B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.75.5 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Phi-3 Vision 128K Instruct | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s |
| Yi 34B Chat | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s |
| TMax 2B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.78.1 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s |
| Granite 4.0 H Micro | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.46.1 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s |
| Dolphin 2.9 Llama 3 8B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| MiniCPM3 4B | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s |
| Voxtral Mini 3B | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.62.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.62.8 tok/s |
| Nous Hermes 2 SOLAR 10.7B | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s |
| SOLAR 10.7B Instruct | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s |
| S5 nano | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| Phi-3 Mini 4K Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s |
| EXAONE 3.0 7.8B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.36.9 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s |
| Falcon3 Mamba 7B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.7 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s |
| LightOnOCR-3 4B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.44.1 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s |
| Chandra OCR 2 | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.37.8 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.55.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.55.4 tok/s |
| Fara 7B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.34.8 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s |
| Hermes 2 Pro Mistral 7B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| Starling LM 7B Beta | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| Hunyuan A13B (80B-A13B) | Runs wellover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.27.8 tok/s | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.38.6 tok/s |
| Llama 3.2 3B Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.45.8 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s |
| Falcon-H1 3B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s |
| OpenChat 3.5 (0106) | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| WeVisOne-VL 4B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.60.8 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.60.8 tok/s |
| Raon 1 nano | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.83.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.83.2 tok/s |
| Qwen3-VL 2B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.69.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Phi-4 Mini Reasoning (3.8B) | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.52.1 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s |
| Mistral 7B Instruct v0.2 | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| Nous Hermes 2 Mistral 7B DPO | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| OpenChat 3.5 (1210) | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| MiniCPM5 1B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.62.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.125 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.125 tok/s |
| Yi 1.5 9B Chat | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.32.7 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s |
| Unlimited OCR (3B-A0.6B) | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.75.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Qwen2.5 3B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.47.5 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s |
| Aya 23 8B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s |
| DeepSeek R1 Distill Qwen 7B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s |
| DeepSeek Prover V2 7B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.41.8 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s |
| Qwen2.5-VL 3B Instruct | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.53.4 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s |
| Hermes 3 Llama 3.2 3B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.45.8 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s |
| OpenHermes 2.5 Mistral 7B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| Starling LM 7B Alpha | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| LFM2 2.6B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.57.2 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s |
| CodeGeeX4 All 9B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| Gemma 1.1 7B IT | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.33.8 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s |
| Qwen1.5 7B Chat | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.37.4 tok/s | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s |
| DeepSeek Coder 33B Instruct | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.11.2 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s |
| Mistral 7B Instruct v0.1 | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| Falcon 3 3B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.45.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s |
| Qwen2.5-Math 72B Instruct | Slowtight fit40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s |
| LFM2.5 1.2B Instruct | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.57.9 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s |
| Qwen2.5-Omni 3B | Runs great29Basic quality (D) · 29 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.0 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.53.0 tok/s |
| RecurrentGemma 9B IT | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s |
| DeepSeek LLM 67B Chat | Slowtight fit28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.7.0 tok/s | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s |
| Gemma 7B IT | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.33.8 tok/s | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s |
| Granite 3.2 2B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Granite 3.3 2B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Falcon Mamba 7B Instruct | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.7 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s |
| MiniCPM-OCR 1.0 1.3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.52.1 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.104 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.104 tok/s |
| EXAONE 3.5 2.4B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.61.0 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.56.2 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.56.2 tok/s |
| Qwen3 1.7B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.72.4 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s |
| EXAONE 4.0 1.2B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.52.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s |
| Neural Chat 7B v3.3 | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| TeleOCR | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.47.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.95.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.95.4 tok/s |
| LLaVA 1.6 Mistral 7B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.38.1 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.52.9 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.52.9 tok/s |
| PaddleOCR-VL 1.6 0.9B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s |
| Yi 1.5 6B Chat | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s |
| InternLM2 Math Plus 20B | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.14.5 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s |
| DeepSeek Coder 6.7B Instruct | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s |
| Granite 3.1 2B Instruct | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Gemma 2 2B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s |
| Yi Coder 9B Chat | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.32.7 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s |
| Zephyr 7B Beta | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| Orca 2 13B | Runs well23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.28.7 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s |
| Granite 4.0 H 1B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.46.4 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.92.7 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.92.7 tok/s |
| WeVisOne VL 2B | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.55.5 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.55.5 tok/s |
| Llama 2 70B Chat | Slowtight fit25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.6.9 tok/s | Runs well30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs well30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s |
| Granite Vision 3.2 2B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.49.3 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s |
| CodeGemma 7B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.33.8 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s |
| LLM4Decompile 6.7B V2 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s |
| Code Llama 34B Instruct | Runs well31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s |
| StarCoder2 15B Instruct | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s |
| Granite Code 34B Instruct | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s |
| Falcon-H1 1.5B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.7 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s |
| LLM4Decompile 9B V2 | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.32.7 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s |
| Llama 2 13B Chat | Runs well21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.28.7 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s |
| Granite 3.0 2B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.55.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.51.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.51.5 tok/s |
| Baichuan 2 13B Chat | Runs well21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.26.8 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s |
| Granite 3.1 3B-A800M Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.61.8 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s |
| Qwen2.5 Coder 3B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.47.5 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s |
| Orca 2 7B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s |
| SmolVLM2 2.2B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s |
| LFM2 1.2B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.57.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s |
| SQLCoder 7B v2 | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s |
| Codestral Mamba 7B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s |
| Qwen2.5-Math 7B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s |
| StableLM 2 12B Chat | Runs great23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s |
| Mathstral 7B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s |
| Qwen2.5 1.5B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s |
| LFM2-VL 1.6B | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.42.9 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.85.7 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.85.7 tok/s |
| Qwen3.5 0.8B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.77.8 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s |
| Granite 3.0 3B-A800M Instruct | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.61.8 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s |
| Baichuan 2 7B Chat | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.38.4 tok/s | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.53.3 tok/s | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.53.3 tok/s |
| Llama 2 7B Chat | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s |
| SmolVLM 2.2B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s |
| Qwen2-VL 2B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.61.3 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.61.3 tok/s |
| Code Llama 13B Instruct | Runs well25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.28.7 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s |
| Granite Code 20B Instruct | Runs well34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.14.4 tok/s | Runs well34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.28.8 tok/s | Runs well34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.28.8 tok/s |
| Nemotron Mini 4B Instruct | Runs great21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.47.8 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.70.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.70.1 tok/s |
| Zamba2 2.7B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.55.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.50.9 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.50.9 tok/s |
| InternLM2 Math Plus 7B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.37.3 tok/s | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s |
| Code Llama 70B Instruct | Slowtight fit30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.6.9 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s |
| Granite Code 8B Instruct (128K) | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.35.8 tok/s | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s |
| WizardLM 13B v1.2 | Runs great15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.36.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s |
| DeepSeek LLM 7B Chat | Runs great17Basic quality (D) · 17 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.41.8 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s |
| Llama 3.2 1B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.54.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| LLaVA 1.5 13B | Runs great15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.35.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s |
| Yi 6B Chat | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s |
| Hunyuan 1.8B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.82.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s |
| InternVL3 78B | Slowover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.5.1 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s |
| SmolLM2 1.7B Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.85.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| Vicuna 13B v1.5 | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.36.4 tok/s | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s |
| LLaVA 1.5 7B | Runs great15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.40.9 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s |
| Code Llama 7B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s |
| StarCoder2 15B (base) | Runs well30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s |
| Gemma 3 1B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Vicuna 7B v1.5 | Runs great13Basic quality (D) · 13 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s |
| StableLM Zephyr 3B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.52.5 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s |
| InternLM2.5 1.8B Chat | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.77.7 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s |
| Falcon 3 1B Instruct | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.81.1 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.81.1 tok/s |
| WizardMath 7B v1.1 | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s |
| Phi-2 2.7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.52.8 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s |
| OLMo 7B Instruct | Runs great13Basic quality (D) · 13 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s |
| Qwen2 1.5B Instruct | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s |
| LFM2 700M | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.183 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.183 tok/s |
| GLiDE 2B | Runs great17Basic quality (D) · 17 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.78.1 tok/s | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s |
| Falcon 40B Instruct | Runs well11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.11.3 tok/s | Runs well11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.22.7 tok/s | Runs well11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.22.7 tok/s |
| Qwen2.5 Coder 1.5B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s |
| InternLM 7B | Runs great11Basic quality (D) · 11 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s |
| OLMo 2 1B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s |
| Spark X2.6 1.7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.85.9 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| StarCoder2 7B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.41.0 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.41.0 tok/s |
| Granite Code 3B Instruct (128K) | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s |
| DeepScaleR 1.5B Preview | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.82.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s |
| Qwen3 0.6B | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.90.3 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s |
| SmolLM 1.7B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.85.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| Stable Code Instruct 3B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.52.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s |
| Gemma 2B IT | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.58.5 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s |
| StableLM 2 1.6B Chat | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s |
| StableLM 2 Zephyr 1.6B | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s |
| Falcon-H1 0.5B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.260 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.260 tok/s |
| LFM2.5 350M | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.193 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s |
| StarCoder 15B | Runs well21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.23.6 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.36.5 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.36.5 tok/s |
| DeepSeek R1 Distill Qwen 1.5B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.82.5 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s |
| Qwen2.5-Math 1.5B Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s |
| LFM2-VL 450M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.150 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.301 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.301 tok/s |
| GLiDE 0.8B | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.90.3 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s |
| LFM2 350M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.193 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s |
| Granite 4.0 H 350M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.199 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.398 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.398 tok/s |
| d1 2B | Runs great21Basic quality (D) · 21 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s |
| Falcon 7B Instruct | Runs great5Basic quality (D) · 5 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.40.0 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s |
| SmolVLM 500M Instruct | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.133 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.266 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.266 tok/s |
| StarCoder2 3B | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s |
| Qwen2.5 0.5B Instruct | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s |
| Yi Coder 1.5B Chat | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s |
| LFM2.5 230M | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.294 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.589 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.589 tok/s |
| Hunyuan 0.5B Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.125 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.251 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.251 tok/s |
| SmolLM2 360M Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s |
| TinyLlama 1.1B Chat | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.61.5 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s |
| Qwen2 0.5B Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s |
| ERNIE 4.5 0.3B | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s |
| DeepSeek Coder 1.3B Instruct | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s |
| LLM4Decompile 1.3B V2 | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s |
| SmolVLM 256M Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.260 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s |
| CodeGemma 2B | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.58.5 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s |
| Gemma 3 270M | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.251 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.502 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.502 tok/s |
| SmolLM 360M Instruct | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s |
| surya ocr 2.5 | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.98.1 tok/s | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s |
| Llama 3.2 90B Vision Instruct | Slowfar over GPU memory limit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.4.5 tok/s | Runs welltight fit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s |
| SmolLM2 135M Instruct | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s |
| Falcon-H1 Tiny 90M Instruct | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.752 tok/s | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,505 tok/s | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,505 tok/s |
| SmolLM 135M Instruct | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s |
| Qwen2.5 Coder 0.5B Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s |
| gpt-oss 120B | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Runs great84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Kimi K3.1 Pro (2.8T) | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4.1 Flash 1017 (552B-A16B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.3 (753B-A40B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.8 Max (2.4T-A95B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K3 (2.8T) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| LongCat 2.5 (1.6T-A48B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.6 Pro (1T-A42B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.1 Flash 1019 (560B-A25B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M3.1 Flash (428B-A23B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nex N2.6 (321B-A18B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Step 5 Preview (600B-A27B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.3 Flash (320B-A18B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.6 Flash (310B-A15B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4.1 Flash (552B-A16B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 4 (1T-A52B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.8 Flash Next (125B-A6B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s |
| GLM-5.2 (753B-A40B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hunyuan Hy4 Preview (780B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Muse Spark Open (816B-A42B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Pro (1.6T-A49B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.6 (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.1 Flash (560B-A25B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.5 Pro (1T-A42B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Intern-S2 (397B-A17B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.1 (744B-A40B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nex N2 Pro (397B-A17B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Pro (April 2026) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Naive N0.5 Flash (309B-A15.5B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.5 397B-A17B | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M3 (427B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5 (744B-A40B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| K2-Horizon (375B-A23B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hunyuan Hy3 (295B-A21B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Beam 1.0 (501B-A23B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.5 (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.7 Code (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Flash (284B-A13B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.5 (310B-A15B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ornith 1.5 397B-A17B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3.5 Super VL 120B-A12B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs welltight fit89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| DeepSeek V3.2 (671B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Inkling Small (276B-A12B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s |
| GLM-4.7 (355B-A32B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Inkling (975B-A41B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.7 (230B-A10B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s |
| Step 3.7 Flash (198B-A11B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.67.9 tok/s |
| Nemotron 3 Ultra 550B-A55B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek R1 0528 (671B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Thinking (1T) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ornith 1.5 122B-A10B | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s |
| DeepSeek V3.2 Exp | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 2.6 1T | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.5 (230B-A10B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s |
| DeepSeek R1 (671B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Flash (April 2026) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.5 122B-A10B | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs welltight fit87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s |
| Step 3.5 Flash (196B-A11B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.67.9 tok/s |
| DeepSeek V3.1 Terminus | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.1 (671B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Instruct 0905 (1T) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.1 (230B-A10B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s |
| Qwen3 235B-A22B Thinking 2507 | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s |
| Qwen3 Coder 480B-A35B | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2 Flash (309B-A15B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 1T | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 3 (675B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 235B-A22B Instruct 2507 | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s |
| Solar Open 2 250B | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s |
| Kimi K2 Instruct (July 2025) | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2 (230B-A10B) | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s |
| Qwen3-VL 235B-A22B Instruct | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s |
| Command A+ (218B-A25B) | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.28.8 tok/s |
| K-EXAONE 236B-A23B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.31.0 tok/s |
| LongCat Flash Chat (560B) | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3 Super 120B-A12B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs welltight fit83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| Qwen3 235B-A22B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s |
| Devstral 2 (123B) | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.6.4 tok/s | Slowtight fit81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.4.6 tok/s |
| InternVL3.5 241B-A28B | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.26.0 tok/s |
| Laguna S 2.1 (118B-A8B) | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Runs welltight fit83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.49.3 tok/s |
| Llama 3.1 Nemotron Ultra 253B v1 | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s |
| Baichuan M3 235B-A22B | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s |
| GLM-4.6 (355B-A32B) | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3 0324 (671B) | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.6V | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| MiniMax-M1 80K | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A Reasoning (111B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.7.3 tok/s | Slow77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.2 tok/s |
| GLM-4.5 (355B-A32B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.0 Flash (127B-A5B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.99.9 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.82.0 tok/s |
| MiniMax-M1 40K | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Small 4 (119B-A6B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s |
| Mistral Medium 3.5 (128B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.6.3 tok/s | Slowtight fit76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.5 tok/s |
| Trinity Large Thinking (400B-A13B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A (111B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.7.3 tok/s | Slow76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.5.2 tok/s |
| ERNIE 4.5 300B-A47B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.5V | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| Hermes 4 405B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 2.6 Flash (104B-A7.4B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.61.8 tok/s |
| Mistral Large 2.1 (2411) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Slowtight fit76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.7 tok/s |
| Solar Open 100B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.63.4 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| GLM-4.5 Air (106B-A12B) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s |
| Mistral Large 2 (123B) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.4.7 tok/s |
| NorthCUA Pro | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Pixtral Large 124B (2411) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.4.7 tok/s |
| Ring Flash 2.0 (100B-A6B) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.53.1 tok/s |
| dots.llm1 Instruct | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s |
| Ling Flash 2.0 (100B-A6B) | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.53.1 tok/s |
| MiniMax-Text-01 | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Tülu 3 405B | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3 (671B) | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Sarvam 105B | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s |
| DeepSeek Coder V2 (236B) | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s |
| Leanstral 119B-A6B | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.85.1 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.68.4 tok/s |
| DeepSeek Coder V2 Instruct (0614) | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s |
| DeepSeek V2.5 1210 | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s |
| Llama 3.1 405B Instruct | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 4 Maverick (17B x 128E) | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 4 Scout 17B-16E | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.30.5 tok/s |
| Command R+ (08-2024) 104B | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.7.8 tok/s | Slow66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.6 tok/s |
| DeepSeek V2.5 | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s |
| Groot 65B A6B 261023 | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s |
| Jamba 1.5 Large | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.0 flash Sante | Won't run47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.99.9 tok/s | Runs welltight fit64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.82.0 tok/s |
| Command R+ 104B (04-2024) | Won't run44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.7.8 tok/s | Slow60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.5.6 tok/s |
| DeepSeek V2 Chat | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s |
| Mixtral 8x22B Instruct | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit52Fair quality (C) · 52 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.14.1 tok/s |
| Qwen1.5 110B Chat | Won't run36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.7.2 tok/s | Slow49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.5.2 tok/s |
| Grok-1 (314B) | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— |
| Snowflake Arctic Instruct | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— |
| Falcon 180B Chat | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— | Slow23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.5.3 tok/s |
Image models
61 of 61 image models run on the Apple M4 Pro (48 GB). 36.0 GB usable for models.
61 of 61 image models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
61 of 61 image models run on the Apple M4 Max (128 GB). 96.0 GB usable for models.
| Model | M4 Pro (48 GB)fit · speed · quality | M4 Max (64 GB)fit · speed · quality | M4 Max (128 GB)fit · speed · quality |
|---|---|---|---|
| Z-Image Turbo | Runs well84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs well84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15 s | Runs well84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.15 s |
| ERNIE-Image Turbo | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.32 s | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.19 s | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.19 s |
| FLUX.2 [klein] 4B | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.9 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.5.9 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.5.9 s |
| FLUX.2 [klein] 9B | Runs well82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs great84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.11 s |
| FLUX 3 Image Klein | Slow91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.56 s | Runs well91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.33 s | Runs well91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.33 s |
| Krea 2 Turbo | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.38 s | Runs well92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs well92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.25 s |
| Fibo 2.0 Dev | Very slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.3.7 min | Slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.2.2 min |
| Ideogram 4 | Very slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.3.7 min | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.2.2 min |
| Qwen-Image 2.1 Turbo | Slowtight fit87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Slow87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.61 s | Slow89Excellent quality (A+) · 89 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.68 s |
| Z-Image | Very slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.2.8 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.7 min |
| FLUX.1 [schnell] | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.18 s | Runs great68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11 s |
| ERNIE-Image | Very slow87Excellent quality (A+) · 87 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.4.2 min | Slow88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slow88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.2.1 min |
| Qwen-Image 2512 | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.7 min | Very slow89Excellent quality (A+) · 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.1 min | Very slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.8.8 min |
| Chroma1-HD | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.3.3 min | Slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.2.0 min | Slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.2.0 min |
| FireRed-Image-Edit 1.1 | Very slowtight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.15.1 min | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.9.0 min | Very slow87Excellent quality (A+) · 87 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.7.8 min |
| FLUX.1 Krea [dev] | Slow80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.70 s | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.70 s |
| Qwen-Image-Edit 2511 | Very slowtight fit85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.15.1 min | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.9.0 min | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.8 min |
| FLUX.1 Kontext [dev] | Very slow80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.3.3 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.1 min |
| FLUX.1 [dev] | Slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.70 s | Slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.70 s |
| HiDream-I1 Full | Very slowtight fit82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.6.7 min | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.1 min | Very slow84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.4.4 min |
| Ming-Image 0.1 Design | Very slowencoder unloaded after use86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.2.8 min | Slow84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.86 s | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.1.6 min |
| HiDream-I1 Dev | Very slowtight fit78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.2.5 min | Slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.90 s | Slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.1.7 min |
| HiDream-I1 Fast | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.89 s | Slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.53 s | Slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.59 s |
| Qwen-Image 2.1 | Very slowtight fit · encoder unloaded after use92Excellent quality (A+) · 92 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.11.9 min | Very slowtight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.6.7 min | Very slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.5.9 min |
| FLUX.1 Fill [dev] | Slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.73 s | Slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.73 s |
| MOSS-Image SFT | Slow80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.82 s | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.82 s |
| Qwen-Image | Very slowencoder unloaded after use86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.14.0 min | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.9 min | Very slow88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.8.8 min |
| Stable Diffusion 3.5 Large Turbo | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.14 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.9.4 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.9.4 s |
| FLUX.2 [dev] | Very slowencoder unloaded after use85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.7.5 min | Very slowencoder unloaded after use91Excellent quality (A+) · 91 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.4.2 min | Very slow91Excellent quality (A+) · 91 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.0 min |
| Stable Diffusion 3.5 Large | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.64 s | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.64 s |
| MiniMax H3 Image | SlowCPU offload93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slowencoder unloaded after use91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.55 s | Slow91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.53 s |
| Iris-3B | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.6 min | Slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.6 min |
| Kolors | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.23 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.23 s |
| CogView4 6B | Very slow68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.2.7 min | Slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.1.8 min |
| OmniGen2 | Very slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.2.6 min | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.1.6 min |
| SDXL-Lightning (4-step) | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.9 s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.4.1 s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.4.1 s |
| Cosmos3-Super Text2Image (64B) | Very slowCPU offload90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.7.7 min | Very slowCPU offload90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.4.6 min | Very slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.3.5 min |
| Stable Diffusion 3.5 Medium | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.59 s | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35 s | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.35 s |
| Stable Diffusion 3 Medium | Runs well52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.31 s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.16 s |
| DreamShaper XL v2 Turbo | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.7.6 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.7.6 s |
| PixArt-Σ XL 1024 | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.15 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.8.8 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.8.8 s |
| Lumina-Image 2.0 | Slow62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.69 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.37 s |
| Kandinsky 3 | Slow48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.89 s | Slow48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.53 s | Slow48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.53 s |
| HunyuanImage 3.0 (80B-A13B) | Very slowtight fit · CPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.11.5 min | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.6.9 min | Very slowtight fit88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.3 min |
| Pony Diffusion V6 XL | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs well56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.20 s |
| Stable Cascade | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.49 s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.26 s |
| Juggernaut XL v9 | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.26 s |
| RealVisXL V5.0 | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.26 s |
| Playground v2.5 | Slow60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.74 s | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.40 s |
| SDXL Turbo | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.0.9 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.0.9 s |
| Animagine XL 4.0 | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25 s |
| Illustrious XL v0.1 | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25 s |
| AuraFlow v0.3 | Slow56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.82 s | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.82 s |
| Animagine XL 3.1 | Runs well54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.22 s | Runs well54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.22 s |
| Hunyuan-DiT v1.2 | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.59 s | Runs well52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.35 s | Runs well52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.35 s |
| Stable Diffusion XL 1.0 | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.24 s | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.24 s |
| Realistic Vision V5.1 | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.7.0 s |
| DreamShaper 8 | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.7.0 s |
| Stable Diffusion 1.5 | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.7.0 s |
| LCM DreamShaper v7 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.2.5 s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.1.5 s |
| SD Turbo | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.0.8 s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.0.5 s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.0.5 s |
Video models
41 of 43 video models run on the Apple M4 Pro (48 GB). 36.0 GB usable for models.
42 of 43 video models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
43 of 43 video models run on the Apple M4 Max (128 GB). 96.0 GB usable for models.
| Model | M4 Pro (48 GB)fit · speed · quality | M4 Max (64 GB)fit · speed · quality | M4 Max (128 GB)fit · speed · quality |
|---|---|---|---|
| Bernini-R A14B (video generation + editing) | Very slow83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.1.8 h | Very slow83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.3 min | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.70.3 min |
| LTX-Video 13B 0.9.8 (distilled) | Runs well66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.8.9 min | Runs well66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.3 min | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.5.9 min |
| LTX-2.5 (22B, video + audio) | Slowencoder unloaded after use81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.18.8 min | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.10.7 min | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.8.0 min |
| SCAIL-2 14B (character animation) | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.3.0 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.8 h | Very slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.2.0 h |
| HY-OmniWeaving (8.3B, unified video) | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.78.3 min | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.46.6 min | Very slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.51.8 min |
| FramePack I2V (HunyuanVideo 13B) | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.59.1 min | Slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Slow74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| Wan 2.2 S2V 14B (speech to video) | Very slow75Great quality (A) · 75 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.3.1 h | Very slow75Great quality (A) · 75 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.1.9 h | Very slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.6 h |
| Wan 2.2 I2V A14B | Very slowtight fit79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.5.3 h | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.0 h | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.6 h |
| Wan 2.2 T2V A14B | Very slowtight fit79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.5.3 h | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.0 h | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.6 h |
| Wan 2.1 I2V 14B 480P | Very slow73Great quality (A) · 73 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.8 h | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.79.1 min | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.87.9 min |
| SkyReels V2 T2V 14B 540P | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.4 h | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.1 h | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.3 h |
| LTX-2.3 | Very slowtight fit · encoder unloaded after use85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.79.3 min | Slowtight fit84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| HunyuanVideo 1.5 (8.3B) | Very slowencoder unloaded after use82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.7 h | Very slow82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.6 h | Very slow84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.2.9 h |
| LTX-2 (19B, video + audio) | Very slowtight fit · encoder unloaded after use80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.62.1 min | Slowtight fit80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| MiniMax H3 (33B, video + audio) | Very slowtight fit · encoder unloaded after use83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.3.6 h | Very slowtight fit · encoder unloaded after use89Excellent quality (A+) · 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Very slow89Excellent quality (A+) · 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.1.9 h |
| Kandinsky 6.0 Video Lite (3B, video + audio) | Very slowtight fit68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.50.9 min | Slow68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.30.3 min | Slow68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.30.3 min |
| Wan 2.2 Animate 14B | Very slowtight fit72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.2 h | Very slowtight fit77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.6 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.3.1 h |
| Wan 2.1 I2V 14B 720P | Very slowtight fit72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.6 h | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.7 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.3 h |
| Wan 2.1 T2V 14B | Very slowtight fit72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.6 h | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.7 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.3.3 h |
| Wan 2.1 VACE 14B | Very slowtight fit72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.7.2 h | Very slowtight fit77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.4.1 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.6 h |
| Wan 2.2 TI2V 5B | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.1.7 h | Very slow68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.45.7 min | Very slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.50.8 min |
| Cosmos-Predict2 2B Video2World | Slow56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.37.0 min | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.24.4 min | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.24.4 min |
| CogVideoX 5B | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.6 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.17.6 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.17.6 min |
| CogVideoX 5B I2V | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.6 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.17.6 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.17.6 min |
| Mochi 1 Preview (10B) | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.2.7 h | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.1.6 h | Very slow62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.1.8 h |
| Kandinsky 6.0 Pro (29B, video + audio) | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.10.5 h | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.6.2 h | Very slowtight fit88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.4.8 h |
| LTX-Video 2B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.3.7 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.2.4 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.2.4 min |
| SkyReels V2 Diffusion Forcing 1.3B | Very slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.3 min | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.29.3 min | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.29.3 min |
| Wan 2.1 T2V 1.3B | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.4 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.23.4 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.23.4 min |
| HunyuanVideo 13B | Very slowencoder unloaded after use66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.8.3 h | Very slow66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.7 h | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.9 h |
| HunyuanVideo I2V 13B | Very slowencoder unloaded after use66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.8.3 h | Very slow66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.7 h | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.9 h |
| Open-Sora 2.0 (11B) | Very slowtight fit · encoder unloaded after use70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.8.6 h | Very slowtight fit70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.9 h | Very slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.4.9 h |
| Cosmos-Predict2 14B Video2World | Very slowtight fit · encoder unloaded after use68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.4.1 h | Very slow68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.2.3 h | Very slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.6 h |
| SkyReels V2 T2V 14B 720P | Very slowCPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.7.7 h | Very slowtight fit76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.5 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.9 h |
| Wan 2.1 VACE 1.3B | Slow53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.38.4 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.4 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.25.4 min |
| Step-Video T2V (30B) | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.9.9 h | Very slowtight fit · encoder unloaded after use74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.4.8 h | Very slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.5.0 h |
| CogVideoX 2B | Runs well44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.8.9 min | Runs well44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.5.3 min | Runs well44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.5.3 min |
| Allegro (2.8B) | Very slowencoder unloaded after use52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.3.4 h | Very slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Very slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.2.0 h |
| AnimateDiff v3 (SD 1.5) | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.3.3 min | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.2.0 min | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.2.0 min |
| Cosmos 1.0 Diffusion 7B Text2World | Very slowCPU offload60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Very slowencoder unloaded after use60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.56.4 min | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.53.7 min |
| CogVideoX 1.5 5B | Very slowtight fit · CPU offload60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.2 h | Very slow58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.79.1 min | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.87.9 min |
| Cosmos3-Super Image2Video (64B) | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.4.8 h |
| Wan-Dancer 14B (music to dance) | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.8.4 h | Very slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.7.2 h |
Decision models
69 of 70 decision models run on the Apple M4 Pro (48 GB). 36.0 GB usable for models.
69 of 70 decision models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
70 of 70 decision models run on the Apple M4 Max (128 GB). 96.0 GB usable for models.
| Model | M4 Pro (48 GB)fit · speed · quality | M4 Max (64 GB)fit · speed · quality | M4 Max (128 GB)fit · speed · quality |
|---|---|---|---|
| Jina Reranker v3.5 | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.67 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.35 ms |
| Jina Reranker v3 | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.67 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.35 ms |
| FRIDA-Decisions | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.50 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.27 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.27 ms |
| GLiNER2.5 Decide | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.31 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18 ms |
| Qwen3 Reranker 0.6B | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.67 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.35 ms |
| Llama Prompt Guard 2 86M | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.20 ms |
| mxbai-rerank-base v2 | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.55 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.30 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.30 ms |
| mxbai-rerank-large v2 | Runs well82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.153 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.84 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.84 ms |
| d1 3B | Runs well82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.165 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.97 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.97 ms |
| GLiClass ModernBERT Large v3 | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.26 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.15 ms |
| OpenAI Privacy Filter | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.6.8 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.5.4 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.5.4 ms |
| Qwen3 Reranker 4B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.392 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.198 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.198 ms |
| Intern-Decision 4B | Runs well84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.239 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.121 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.121 ms |
| zerank-1 small | Runs well80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.170 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.94 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.94 ms |
| BGE Reranker v2 M3 | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.68 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.36 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.36 ms |
| DeBERTa v3 Large Zero-shot v2.0 | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.29 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16 ms |
| DuoGuard 0.5B | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.59 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.31 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.31 ms |
| zerank-2 | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.392 ms | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.198 ms | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.198 ms |
| GTE Reranker ModernBERT Base | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.21 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.12 ms |
| Jina Reranker v2 Base Multilingual | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20 ms |
| ModernBERT Large Zero-shot v2.0 | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.26 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15 ms |
| Shieldstral 1.0 3B | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.376 ms | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.190 ms | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.190 ms |
| Skywork Reward V2 Qwen3 0.6B | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.121 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71 ms |
| LemonDecider 4B | Runs well80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.225 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.115 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.115 ms |
| Qwen3 Reranker 8B | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.795 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.399 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.399 ms |
| GLiGuard 300M | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.28 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms |
| Granite Embedding Reranker English R2 | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.21 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.12 ms |
| Laya | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.28 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms |
| Llama Prompt Guard 2 22M | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.8.0 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.8.0 ms |
| Qwen3Guard Gen 0.6B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.102 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.71 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.71 ms |
| GLiClass Base v3 | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.9.3 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.9.3 ms |
| ProtectAI DeBERTa v3 Prompt Injection v2 | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.24 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.14 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.14 ms |
| BGE Reranker v2 Gemma | Runs well78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.246 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.125 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.125 ms |
| Skywork Reward V2 Llama 3.1 8B | Very slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.736 ms | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.736 ms |
| DeBERTa v3 Base Zero-shot v2.0 | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.14 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.0 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.0 ms |
| GLiNER2 Base v1 | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.9 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.9 ms |
| HHEM 2.1 Open | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.29 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.16 ms |
| LettuceDetect Large | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.94 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.49 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.49 ms |
| Laya Multilingual | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.22 ms | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 ms | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 ms |
| GLiDE 27B | Slowtight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.640 ms | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.640 ms |
| Skywork Reward V2 Qwen3 8B | Very slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.743 ms | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.743 ms |
| Qwen3Guard Gen 4B | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.580 ms | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.268 ms | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.268 ms |
| GLiNER Multi v2.1 | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12 ms |
| Qwen3Guard Gen 8B | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.539 ms | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.539 ms |
| BART Large MNLI | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.27 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.16 ms |
| Llama 3.1 Nemotron Safety Guard 8B v3 | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.528 ms | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.528 ms |
| Granite Guardian 4.1 8B | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.551 ms | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.551 ms |
| ms-marco MiniLM L6 v2 | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.6.6 ms | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms |
| NuExtract 2.0 4B | Slow75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.622 ms | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.313 ms | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.313 ms |
| WildGuard 7B | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.951 ms | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.477 ms | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.477 ms |
| Granite Guardian 3.3 8B | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.538 ms | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.538 ms |
| NuExtract 2.0 2B | Runs well69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.368 ms | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.145 ms | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.145 ms |
| ShieldGemma 2B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.256 ms | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.130 ms | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.130 ms |
| ShieldGemma 9B | Slow64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.896 ms | Slow64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.450 ms | Slow64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.450 ms |
| Llama Guard 3 1B | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.184 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.102 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.102 ms |
| CompassJudger-2 32B | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.26.5 s | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.19.7 s | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.19.7 s |
| ShieldGemma 27B | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.6 s | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.3 s |
| Llama Guard 3 8B | Slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.528 ms | Slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.528 ms |
| Nemotron Content Safety Reasoning 4B | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.5.1 s | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.9 s | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.9 s |
| DynaGuard 8B | Very slow79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.9.7 s | Very slow79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s | Very slow79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s |
| gpt-oss-safeguard 20B | Very slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.7 s | Very slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.6 s | Very slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.6 s |
| Selene 1 Mini 8B | Very slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.9.6 s | Very slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s | Very slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s |
| CompassJudger-2 7B | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.9.2 s | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.6 s | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.6 s |
| Llama Guard 4 12B | Slow62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.788 ms | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.788 ms |
| DynaGuard 1.7B | Slow66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.906 ms | Slow69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.906 ms |
| Prometheus 2 7B | Very slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.8.7 s | Very slow67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.3.4 s | Very slow67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.3.4 s |
| Prometheus 2 8x7B | Very slowtight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.9.5 s | Very slow71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.5.7 s | Very slow74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.8.3 s |
| GLIDER | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.3.6 s | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s |
| Flow Judge v0.1 | Very slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.3.6 s | Very slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Very slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s |
| gpt-oss-safeguard 120B | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Very slow80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.1 s |
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.