M4 Max (64 GB) vs M4 Max (128 GB) vs RTX 5090 for local AI
Pick two to four systems or devices to see which runs more models, faster, for the kind of work you do.
Winner: Apple M4 Max (128 GB)
For general chat: It runs 27 more LLMs (548 vs 521) (against the Apple M4 Max (64 GB), the next best). On the other hand, the Apple M4 Max (64 GB) is about 1.4× faster on its own top models.
Side by side
All models on the M4 Max (64 GB) All models on the M4 Max (128 GB) All models on the RTX 5090
 Apple M4 Max (64 GB)
Remove
|
Apple M4 Max (128 GB)
Remove
|
 NVIDIA GeForce RTX 5090
Remove
|
|
|---|---|---|---|
| Memory | 64.0 GB unified | 128 GB unified | 32.0 GB VRAM |
| Usable for models | 48.0 GB | 96.0 GB | 32.0 GB |
| Memory bandwidth | 546 GB/s | 546 GB/s | 1,792 GB/s |
| FP16 compute | 18.4 TFLOPS | 18.4 TFLOPS | 104.8 TFLOPS |
| Launch price | — | — | $1,999 |
| Released | 2024 | 2024 | 2025 |
| LLMs that run | 521 of 622 | 548 of 622 | 514 of 622 |
| LLMs that run well | 504 | 533 | 493 |
| Image models that run | 61 of 61 | 61 of 61 | 60 of 61 |
| Video models that run | 42 of 43 | 43 of 43 | 40 of 43 |
| Decision models that run | 69 of 70 | 70 of 70 | 68 of 70 |
| Top-10 LLM average score | 100 | 99 | 100 |
| Top-10 LLM average speed | 89.4 tok/s | 65.5 tok/s | 114 tok/s |
| Best LLM | Qwen3.5 35B-A3B | Nemotron 3.5 Bolt 74B-A9B | Nanbeige4.3 33B |
Highlighted values are the best of the 3. Model results assume 8K of context and the default macOS GPU memory limit on Macs.
LLMs
521 of 622 LLMs run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
548 of 622 LLMs run on the Apple M4 Max (128 GB). 96.0 GB usable for models.
514 of 622 LLMs run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | M4 Max (64 GB)fit · speed · quality | M4 Max (128 GB)fit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| Qwen3.5 35B-A3B | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Qwen3.6 35B-A3B | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Sage-A1.5 35B-A3B | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Flint (40B-A2.5B) | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.81.2 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.222 tok/s |
| Ornith 1.5 35B-A3B | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| MiniCPM-V 4.7 35B-A3B | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great93Excellent quality (A+) · 93 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Apodex 1.1 Mini (35B-A3B) | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Ornith 1.0 35B-A3B | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| GLM-4.7 Flash (30B-A3B) | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Raon-1 mini Preview (30B-A4B) | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.54.2 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.54.2 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s |
| Nex N2 Mini (35B-A3B) | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Agents-A1 35B-A3B | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Qwen AgentWorld 35B-A3B | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Gemma 4 26B-A4B | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.76.3 tok/s | Runs great92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.59.7 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.139 tok/s |
| Laguna XS 2.1 (33B-A3B) | Runs great81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Nemotron Cascade 2 30B-A3B | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| Nanbeige4.3 33B | Runs well94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s | Runs well94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s | Runs welltight fit94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.46.1 tok/s |
| KAT-Coder V2.5 Dev (35B-A3B) | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| DiffusionGemma 26B-A4B IT | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.57.4 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.57.4 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| Nemotron 3 Nano 30B-A3B | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| Qwen3 30B-A3B Instruct 2507 | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Qwen3-VL 30B-A3B Thinking | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Nemotron 3.5 Bolt 74B-A9B | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs welloffloads to RAM77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s |
| Kolibri 1 (78B-A3.5B) | Runs welltight fit76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.127 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.81.6 tok/s | Runs welloffloads to RAM76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.86.5 tok/s |
| Qwen3 30B-A3B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Qwen3-VL 30B-A3B Instruct | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Hunyuan Hy-Memory 30B-A3B | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great82Great quality (A) · 82 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| LFM2.5 24B-A2B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.82.4 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.192 tok/s |
| Nemotron 3.5 Lightning 30B-A3B | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Kimi Linear 48B-A3B | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.233 tok/s |
| Qwen3.8 27B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| Qwen3.6 27B | Runs well89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs well89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| Qwen3-Next 80B-A3B Thinking | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welloffloads to RAM72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.88.4 tok/s |
| Sarvam 30B | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.83.5 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.207 tok/s |
| Qwen3-Omni 30B-A3B Instruct | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s |
| Gemma 4 31B | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18.5 tok/s | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.18.5 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s |
| Qwen3.5 27B | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| North Mini Code 1.0 (30B-A3B) | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Qwen3 Next 80B-A3B Instruct | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welloffloads to RAM70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.88.4 tok/s |
| Muse Glimmer 30B | Runs well87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.19.4 tok/s | Runs well87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.19.4 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.51.3 tok/s |
| Qwen3 30B-A3B Thinking 2507 | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Qwen3 Coder 30B-A3B | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| TMax 27B | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Trinity Mini (26B-A3B) | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Fara 1.5 27B | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| UI-Mate 27B | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Gemma 4 12B | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great83Great quality (A) · 83 of 100: how capable the model is at F16, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s |
| Axion 1.0 35B | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Seed-OSS 36B Instruct | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.60.9 tok/s |
| ERNIE 4.5 21B-A3B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| EXAONE 4.5 33B | Runs well77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs well77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs welltight fit81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s |
| Llama 3.3 Nemotron Super 49B v1.5 | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.57.0 tok/s |
| Granite 4.0 H Small (32B-A9B) | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.38.3 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.38.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.118 tok/s |
| Hermes 4.3 36B | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.60.9 tok/s |
| Hunyuan A13B (80B-A13B) | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.38.6 tok/s | Runs welloffloads to RAM62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.25.1 tok/s |
| K2 Horizon 7B | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| Qwen3 Coder Next (80B-A3B) | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welloffloads to RAM72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| NorthCUA Mini | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Qwen3-VL 32B Instruct | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.45.8 tok/s |
| InternVL3.5 38B | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s | Runs welltight fit73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.47.3 tok/s |
| SenseNova Loop 9B Preview | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.53.4 tok/s |
| Mellum2.1 Thinking (12B-A2.5B) | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| Ornith 1.5 9B | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Qwen3 32B | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| ERNIE 4.5 21B-A3B Thinking | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Phi-3.5 MoE (42B-A6.6B) | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.157 tok/s |
| Qwen3.5 9B | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Granite 4.2 30B | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.62.1 tok/s |
| LFM2 24B-A2B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.46.8 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.195 tok/s |
| Ornith 1.0 9B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| gpt-oss 20B | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Ling 2.6 Flash (104B-A7.4B) | Runs welltight fit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.61.8 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Solar Open 100B | Runs welltight fit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.63.4 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s |
| Maple Preview (20B-A1B) | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.113 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.195 tok/s |
| Devstral Small 2 (24B) | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| TMax 9B | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.57.7 tok/s |
| GLM-4 32B 0414 | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| InternVL3 38B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.47.3 tok/s |
| Ling Mini 2.0 (16B-A1.4B) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.198 tok/s |
| Granite 4.1 30B | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.20.0 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.20.0 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.62.9 tok/s |
| Ling Flash 2.0 (100B-A6B) | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.53.1 tok/s | Runs welloffloads to RAM53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| Apriel 1.5 15B Thinker | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.75.4 tok/s |
| Nemotron 3.5 Bolt E4B | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s |
| Groot 19B A3B 261023 | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Olmo 3.1 32B Instruct | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| EXAONE 4.0 32B | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.8 tok/s |
| Qwen2.5 32B Instruct | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Baichuan M2 32B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Sarvam 105B | Runs welltight fit52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Slowheavy RAM offload52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.36.1 tok/s |
| Kimi-VL A3B Thinking 2506 | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Gemma 3 27B IT | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.55.7 tok/s |
| Nous Hermes 2 Mixtral 8x7B DPO | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs welltight fit49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| Phi-4 Reasoning Vision 15B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.74.1 tok/s |
| Qwen2.5-VL 32B Instruct | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s |
| Gemma 4 E4B | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s |
| Ling 3.0 Tiny (8B-A1.3B) | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.145 tok/s |
| Granite 4.2 8B | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s |
| Jamba 1.5 Mini | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Devstral Small 1.1 (24B) | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Magistral Small 2507 | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| MedGemma 27B Text IT | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.21.4 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.21.4 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.5 tok/s |
| Mixtral 8x7B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| QwQ 32B | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Qwen3 14B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.39.0 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.39.0 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.75.7 tok/s |
| MiMo V2.6 Distill Qwen 9B | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| Qwen2 57B-A14B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.122 tok/s |
| Nanbeige4.2 3B | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| Mistral Small 3.2 24B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Magistral Small 2506 | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Terminal RSI 27B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| OpenReasoning Nemotron 32B | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Skywork OR1 32B | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Hermes 4 14B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Gemma 4 E2B | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.58.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.58.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.224 tok/s |
| Nemotron Labs Dory 1.0 4B | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s |
| Hemmingway 1.5 | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.21.5 tok/s | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.21.5 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.56.7 tok/s |
| Falcon-H1 34B Instruct | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.2 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.17.2 tok/s | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s |
| Hermes 4 70B | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Llama 3.3 70B Instruct | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| Mistral Small 3.1 24B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Phi-4 (14B) | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.76.2 tok/s |
| Olmo 3.1 32B Think | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| InternVL3 14B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.74.1 tok/s |
| EXAONE Deep 32B | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.8 tok/s |
| Ministral 3 14B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s |
| Qwen2.5 72B Instruct | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| Qwen2.5-VL 72B Instruct | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs well61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| Spark-X2.6 4B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71.5 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.71.5 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.126 tok/s |
| DeepSeek R1 Distill Llama 70B | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Tülu 3 70B | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Qwen3-VL 8B Instruct | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.58.8 tok/s |
| Mistral Small 3 24B (2501) | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Command R (08-2024) 32B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s |
| LFM2.5 8B-A1B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.137 tok/s |
| InternVL3.5 8B | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.60.5 tok/s |
| Qwen2.5 14B Instruct | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.76.2 tok/s |
| Ring Flash 2.0 (100B-A6B) | Runs welltight fit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.53.1 tok/s | Runs welloffloads to RAM54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| GLM-Z1 32B (0414) | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| DeepSeek R1 Distill Qwen 32B | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| DeepSeek Coder V2 Lite (16B) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| Gemma 2 27B IT | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.21.2 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.21.2 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.1 tok/s |
| Llama 3.1 70B Instruct | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| Kimi-VL A3B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Aya Vision 32B | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.4 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.4 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| DeepSeek V2 Lite Chat | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| Granite 4.1 8B | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s |
| Voxtral Small 24B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.23.8 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.23.8 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.62.9 tok/s |
| Dolphin Mistral 24B Venice Edition | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Aya Expanse 32B | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s |
| Phi-4 Reasoning Plus (14B) | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s |
| GLM-4.1V 9B Thinking | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s |
| MiniCPM-V 4.5 8B | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.46.0 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.46.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.59.3 tok/s |
| OLMo 2 32B Instruct | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| Command R 35B (v01) | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s | Runs welltight fit45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.81.3 tok/s |
| Qwen2.5 14B Instruct 1M | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Qwen3 8B | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| OpenHands LM 32B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Gemma 3 12B IT | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| Qwen3.5 4B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.111 tok/s |
| Nemotron Nano 9B v2 | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s |
| Hermes 3 (Llama 3.1 70B) | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| QVQ 72B Preview | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs welloffloads to RAM47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| Magistral Small 1.2 (24B) | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| UI-Mate 9B | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| UI-Venus 2 9B | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| Dolphin 3.0 R1 Mistral 24B | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Phi-4 Reasoning (14B) | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s |
| Solar Pro Preview 22B | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.26.1 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.26.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.50.6 tok/s |
| TMax 4B | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.69.8 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.69.8 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s |
| Kimi Dev 72B | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs welloffloads to RAM56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| MiMo-VL 7B RL | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.62.1 tok/s |
| MiniCPM5 2B | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.53.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.53.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.205 tok/s |
| Llama 3.1 Nemotron 70B Instruct | Runs welltight fit55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Apertus v1.5 70B | Runs welltight fit55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs welloffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.19.9 tok/s |
| Agents-A1 4B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| InternVL3 78B | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s |
| Qwen2.5 Coder 32B Instruct | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Ministral 3 8B | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.57.9 tok/s |
| Moonlight 16B-A3B Instruct | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Aya 23 35B | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.81.3 tok/s |
| QwQ 32B Preview | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Qwen2 72B Instruct | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs welloffloads to RAM44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| Qwen2-VL 72B Instruct | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs well53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs welloffloads to RAM44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| MiMo 7B RL | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.51.1 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.51.1 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.65.9 tok/s |
| Fara 1.5 4B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| InternVL3 8B | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s |
| Mage-VL 4.7B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| AceReason Nemotron 14B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| OpenReasoning Nemotron 14B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Qwen2.5 Coder 14B Instruct | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Apriel Nemotron 15B Thinker | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.38.6 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.38.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.74.8 tok/s |
| Ministral 3 14B Reasoning | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s |
| Llama 3.1 Nemotron Nano 8B v1 | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Falcon 3 10B Instruct | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.50.1 tok/s |
| Mistral Small 22B (2409) | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| Hunyuan 7B Instruct | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.2 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.2 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.68.8 tok/s |
| Devstral Small 2505 | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| MiniCPM4 8B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| SenseNova 6.8 Code2Video | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Reka Flash 3.1 (21B) | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.27.6 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.27.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| Mistral Nemo 12B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| DeepCoder 14B Preview | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Apertus 70B Instruct | Runs welltight fit50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| Qwen3-VL 4B Instruct | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s |
| GLM-4 9B 0414 | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| NorthCUA Nano | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| Olmo 3 7B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| Qwen3 4B Instruct 2507 | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| LFM3 1.7B | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.298 tok/s |
| MiniCPM-o 2.6 8B | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.46.2 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.46.2 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.59.5 tok/s |
| Gemma 2 9B IT | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.43.3 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.43.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.55.9 tok/s |
| Yi 1.5 34B Chat | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.64.0 tok/s |
| Qwen2.5-VL 7B Instruct | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| Granite 4.0 H Tiny (7B-A1B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.85.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.85.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.158 tok/s |
| Llama 3 70B Instruct | Runs welltight fit48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| DeepSeek R1 Distill Qwen 14B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| DeepSeek R1 0528 Qwen3 8B | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| Pixtral 12B (2409) | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.38.5 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.38.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.90.3 tok/s |
| Qwen2.5 7B Instruct | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| JanusCoder 14B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Codestral 22B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| InternVL2.5 8B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s |
| Foundation-Sec 1.1 8B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| InternLM3 8B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.45.5 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.45.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.58.6 tok/s |
| InternLM2.5 20B Chat | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| Granite 4.2 3B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s |
| GLiDE 9B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s |
| Qwen2.5 7B Instruct 1M | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Falcon-H1 7B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.52.8 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.8 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| GLM-4 9B Chat 1M | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.4 tok/s |
| Llama 3.2 90B Vision Instruct | Runs welltight fit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Slowoffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.5.4 tok/s |
| GLM-4 9B Chat (2024) | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| LFM2 8B-A1B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.137 tok/s |
| Qwen3 4B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| Hermes 3 (Llama 3.1 8B) | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Llama 3.1 8B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Mage-VL-Cowork 4.7B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| Falcon 3 7B Instruct | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.69.2 tok/s |
| MiniCPM-V 2.6 8B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.4 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.7 tok/s |
| Gemma 3n E4B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.129 tok/s |
| GLM-Z1 9B (0414) | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Seed-Coder 8B Reasoning | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.62.6 tok/s |
| Nemotron 3 Nano 4B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.74.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.74.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s |
| Phi-3 Medium 128K Instruct | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.80.2 tok/s |
| Apertus v1.5 8B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.58.0 tok/s |
| OLMo 2 13B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.81.6 tok/s |
| LFM3-VL 2B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.269 tok/s |
| Qwen1.5 32B Chat | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.17.8 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.17.8 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Granite 3.2 8B Instruct | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Granite 3.3 8B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| LLM4Decompile 22B V2 | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| Ministral 8B (2410) | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.4 tok/s |
| DeepHermes 3 Llama 3 8B Preview | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Dolphin 3.0 Llama 3.1 8B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Tülu 3 8B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Phi-3 Medium (14B) | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.80.2 tok/s |
| LLaVA 1.6 34B | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.16.6 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.16.6 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.3 tok/s |
| EXAONE 3.5 7.8B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Phi-3 Small 8K Instruct | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.69.8 tok/s |
| GLM-4.6V | Runs wellover GPU memory limit58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Slowheavy RAM offload58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.27.8 tok/s |
| Falcon-H1R 7B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| Qwen2.5-Omni 7B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| Olmo 3 7B Think | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| Seed-Coder 8B Instruct | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.62.6 tok/s |
| Qwen1.5 MoE A2.7B Chat | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| EXAONE Deep 7.8B | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Granite 3.1 8B Instruct | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| InternVL2 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s |
| Dolphin 2.9.4 Llama 3.1 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Hermes 2 Pro Llama 3 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| OpenChat 3.6 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Mistral Small 4 (119B-A6B) | Runs wellover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s |
| Aya Vision 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.46.4 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.46.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.59.8 tok/s |
| Phi-4 Multimodal Instruct | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.92.7 tok/s |
| Qwen2-VL 7B Instruct | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| GLM-4.5V | Runs wellover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.27.8 tok/s |
| Command R7B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Jamba Reasoning 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.42.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.42.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| LFM2.5-VL 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.165 tok/s |
| Qwen2 7B Instruct | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Qwen3 4B Thinking 2507 | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| Granite 4.1 3B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.152 tok/s |
| Qwen2.5 Coder 7B Instruct | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Mistral-NeMo-Minitron 8B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.47.6 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.47.6 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.61.4 tok/s |
| North Micro Vision | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.54.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.54.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.208 tok/s |
| Llama 3.2 11B Vision Instruct | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s |
| Llama 3.1 Nemotron Nano 4B v1.1 | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.65.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.65.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Gemma 3 4B IT | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s |
| LFM2.5 2.6B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.191 tok/s |
| GLM-4.5 Air (106B-A12B) | Runs wellover GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Slowheavy RAM offload54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.26.7 tok/s |
| Llama 3 8B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Yi 34B Chat | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.0 tok/s |
| InternLM2 Chat 20B | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| OpenReasoning Nemotron 7B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Phi-4 Mini (3.8B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Ministral 3 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.76.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| SmolLM3 3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| MedGemma 4B IT | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s |
| Command R7B Arabic | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| CogVLM2 Llama 3 Chat 19B | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.6 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.29.6 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.57.4 tok/s |
| Qwen1.5 14B Chat | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.79.0 tok/s |
| Foundation-Sec 8B Reasoning | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Jamba2 3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s |
| Hunyuan 4B Instruct | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.69.6 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.69.6 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.122 tok/s |
| Nous Hermes 2 SOLAR 10.7B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| SOLAR 10.7B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| Apertus 8B Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.1 tok/s |
| Aya Expanse 8B | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Phi-3.5 Mini Instruct | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Gemma 3n E2B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.258 tok/s |
| Nemotron 3.5 Super VL 120B-A12B | Slowfar over GPU memory limit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs welltight fit89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Slowheavy RAM offload65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.24.4 tok/s |
| InternLM2.5 7B Chat | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s |
| Qwen2.5-Math 72B Instruct | Runs welltight fit48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs welloffloads to RAM40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.31.9 tok/s |
| Granite 3.0 8B Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Mistral 7B Instruct v0.3 | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.2 tok/s |
| Zamba2 7B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.1 tok/s |
| Leanstral 119B-A6B | Runs wellover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.85.1 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.68.4 tok/s | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.44.2 tok/s |
| Ornith 1.5 122B-A10B | Slowfar over GPU memory limit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Slowheavy RAM offload64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.28.4 tok/s |
| DeepSeek R1 Distill Llama 8B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| OLMo 2 7B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| DeepSeek LLM 67B Chat | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs welltight fit28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| Qwen3.5 122B-A10B | Slowfar over GPU memory limit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs welltight fit87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Slowheavy RAM offload64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.28.4 tok/s |
| Falcon3 Mamba 7B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.0 tok/s |
| Fara 7B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| Phi-3.5 Vision Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| AFM 4.5B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.112 tok/s |
| Dolphin 2.9 Llama 3 8B | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Llama 4 Scout 17B-16E | Runs wellover GPU memory limit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.30.5 tok/s | Slowheavy RAM offload50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.19.8 tok/s |
| DeepSeek MoE 16B Chat | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.75.0 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.75.0 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.159 tok/s |
| Qianfan-OCR 4B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| Hermes 2 Pro Mistral 7B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Starling LM 7B Beta | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| EXAONE 3.0 7.8B Instruct | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Qwen3.5 2B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.59.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.59.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.227 tok/s |
| Granite 4.0 H Micro | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.162 tok/s |
| Granite 4.0 Micro (3B) | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.152 tok/s |
| OpenChat 3.5 (0106) | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Groot 65B A6B 261023 | Runs wellover GPU memory limit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s | Slowheavy RAM offload49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.47.9 tok/s |
| TMax 2B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.275 tok/s |
| Yi 1.5 9B Chat | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| Phi-3 Mini 128K Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Nemotron 3 Super 120B-A12B | Slowfar over GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs welltight fit83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Slowheavy RAM offload61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.24.6 tok/s |
| MiniCPM-V 4.7 1B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.333 tok/s |
| Mistral 7B Instruct v0.2 | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Nous Hermes 2 Mistral 7B DPO | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| OpenChat 3.5 (1210) | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| DeepSeek Coder 33B Instruct | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Chandra OCR 2 | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.55.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.55.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.97.4 tok/s |
| CodeGeeX4 All 9B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Falcon-H1 3B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.164 tok/s |
| MiniCPM3 4B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.127 tok/s |
| Llama 3.2 3B Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| Llama 2 70B Chat | Runs well30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs well30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs welltight fit25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s |
| S5 nano | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.111 tok/s |
| Voxtral Mini 3B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.62.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.62.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.110 tok/s |
| LightOnOCR-3 4B | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| RecurrentGemma 9B IT | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| Phi-3 Vision 128K Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| Phi-3 Mini 4K Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Aya 23 8B | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| DeepSeek R1 Distill Qwen 7B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Qwen3-VL 2B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.242 tok/s |
| OLMoE 1B-7B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.145 tok/s |
| OpenHermes 2.5 Mistral 7B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Starling LM 7B Alpha | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| DeepSeek Prover V2 7B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.74.7 tok/s |
| Qwen2.5 3B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s |
| WeVisOne-VL 4B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.60.8 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.60.8 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s |
| Unlimited OCR 2 (3B-A0.6B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s |
| Phi-4 Mini Reasoning (3.8B) | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Hermes 3 Llama 3.2 3B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| LFM2 2.6B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.201 tok/s |
| Mistral 7B Instruct v0.1 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Ling 3.0 Flash (127B-A5B) | Slowfar over GPU memory limit57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.99.9 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.82.0 tok/s | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Raon 1 nano | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.83.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.83.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.146 tok/s |
| Gemma 1.1 7B IT | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| Orca 2 13B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Qwen2.5-VL 3B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s |
| MiniCPM5 1B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.125 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.125 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.478 tok/s |
| Code Llama 70B Instruct | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs welltight fit30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s |
| Falcon Mamba 7B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.71.0 tok/s |
| Granite Code 34B Instruct | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s |
| Falcon 3 3B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s |
| Neural Chat 7B v3.3 | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Baichuan 2 13B Chat | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.80.6 tok/s |
| Gemma 7B IT | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| Code Llama 34B Instruct | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.65.2 tok/s |
| Qwen1.5 7B Chat | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.66.9 tok/s |
| Yi Coder 9B Chat | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| StableLM 2 12B Chat | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.92.2 tok/s |
| Llama 2 13B Chat | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| InternLM2 Math Plus 20B | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| Qwen2.5-Omni 3B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.0 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.53.0 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.93.2 tok/s |
| Unlimited OCR (3B-A0.6B) | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s |
| StarCoder2 15B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.70.2 tok/s |
| Granite 3.2 2B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Granite 3.3 2B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| EXAONE 3.5 2.4B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.56.2 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.56.2 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.214 tok/s |
| LLaVA 1.6 Mistral 7B | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.52.9 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.52.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.68.2 tok/s |
| Zephyr 7B Beta | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Qwen3 1.7B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.254 tok/s |
| Yi 1.5 6B Chat | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.85.2 tok/s |
| DeepSeek Coder 6.7B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| LLM4Decompile 9B V2 | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| LFM2.5 1.2B Instruct | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.441 tok/s |
| Granite 3.1 2B Instruct | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Gemma 2 2B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.198 tok/s |
| CodeGemma 7B | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| MiniCPM-OCR 1.0 1.3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.104 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.104 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.397 tok/s |
| EXAONE 4.0 1.2B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.403 tok/s |
| WeVisOne VL 2B | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.55.5 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.55.5 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.212 tok/s |
| LLaVA 1.5 13B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.83.9 tok/s |
| Codestral Mamba 7B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s |
| LLM4Decompile 6.7B V2 | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| PaddleOCR-VL 1.6 0.9B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.538 tok/s |
| Granite Vision 3.2 2B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s |
| Code Llama 13B Instruct | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| WizardLM 13B v1.2 | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| TeleOCR | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.95.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.95.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.363 tok/s |
| Mathstral 7B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.71.2 tok/s |
| Command A (111B) | Slowover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.7.3 tok/s | Slow76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.5.2 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.3 tok/s |
| Granite 3.0 2B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.51.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.51.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s |
| Orca 2 7B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Qwen2.5 Coder 3B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s |
| Qwen2.5-Math 7B Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Granite Code 20B Instruct | Runs well34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.28.8 tok/s | Runs well34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.28.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Granite 4.0 H 1B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.92.7 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.92.7 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.353 tok/s |
| SQLCoder 7B v2 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Vicuna 13B v1.5 | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Laguna S 2.1 (118B-A8B) | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Runs welltight fit83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.49.3 tok/s | Slowheavy RAM offload60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s |
| SmolVLM2 2.2B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.229 tok/s |
| Llama 2 7B Chat | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Falcon-H1 1.5B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.333 tok/s |
| Ling 3.0 flash Sante | Slowfar over GPU memory limit47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.99.9 tok/s | Runs welltight fit64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.82.0 tok/s | Won't run47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen2-VL 2B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.61.3 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.61.3 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.234 tok/s |
| Granite Code 8B Instruct (128K) | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.64.1 tok/s |
| StarCoder2 15B (base) | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.70.2 tok/s |
| Baichuan 2 7B Chat | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.53.3 tok/s | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.53.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.68.7 tok/s |
| LFM2 1.2B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.441 tok/s |
| SmolVLM 2.2B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.229 tok/s |
| InternLM2 Math Plus 7B | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s |
| Zamba2 2.7B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.50.9 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.50.9 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| DeepSeek LLM 7B Chat | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.74.7 tok/s |
| Qwen3.5 0.8B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.593 tok/s |
| Granite 3.1 3B-A800M Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.169 tok/s |
| Qwen2.5 1.5B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| LFM2-VL 1.6B | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.85.7 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.85.7 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.327 tok/s |
| Nemotron Mini 4B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.70.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.70.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s |
| Yi 6B Chat | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.85.2 tok/s |
| Hunyuan 1.8B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.288 tok/s |
| LLaVA 1.5 7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s |
| Command R+ (08-2024) 104B | Slowover GPU memory limit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.7.8 tok/s | Slow66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.6 tok/s | Slowheavy RAM offload49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.3.6 tok/s |
| Granite 3.0 3B-A800M Instruct | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.169 tok/s |
| SmolLM2 1.7B Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| Code Llama 7B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Llama 3.2 1B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.416 tok/s |
| WizardMath 7B v1.1 | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Vicuna 7B v1.5 | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| StableLM Zephyr 3B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| OLMo 7B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.74.9 tok/s |
| InternLM2.5 1.8B Chat | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.273 tok/s |
| Phi-2 2.7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.186 tok/s |
| StarCoder 15B | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.36.5 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.36.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s |
| Command A Reasoning (111B) | Slowover GPU memory limit57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.7.3 tok/s | Slow77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.2 tok/s | Slowheavy RAM offload57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.3.3 tok/s |
| Gemma 3 1B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.516 tok/s |
| StarCoder2 7B | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.41.0 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.41.0 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s |
| InternLM 7B | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s |
| Command R+ 104B (04-2024) | Slowover GPU memory limit44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.7.8 tok/s | Slow60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.5.6 tok/s | Slowheavy RAM offload44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3.6 tok/s |
| GLiDE 2B | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.275 tok/s |
| Falcon 40B Instruct | Runs well11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.22.7 tok/s | Runs well11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.22.7 tok/s | Runs great13Basic quality (D) · 13 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| Falcon 3 1B Instruct | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.81.1 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.81.1 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.309 tok/s |
| Spark X2.6 1.7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| LFM2 700M | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.183 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.183 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.697 tok/s |
| Qwen2 1.5B Instruct | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| DeepScaleR 1.5B Preview | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.290 tok/s |
| Qwen2.5 Coder 1.5B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| Mistral Medium 3.5 (128B) | Slowfar over GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.6.3 tok/s | Slowtight fit76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.5 tok/s | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 2.1 (2411) | Slowfar over GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Slowtight fit76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.7 tok/s | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| SmolLM 1.7B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| Stable Code Instruct 3B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| OLMo 2 1B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.349 tok/s |
| Gemma 2B IT | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.206 tok/s |
| Mistral Large 2 (123B) | Slowfar over GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.4.7 tok/s | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Pixtral Large 124B (2411) | Slowfar over GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.4.7 tok/s | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 0.6B | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.688 tok/s |
| Granite Code 3B Instruct (128K) | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.148 tok/s |
| Falcon-H1 0.5B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.260 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.260 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.992 tok/s |
| DeepSeek R1 Distill Qwen 1.5B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.290 tok/s |
| LFM2.5 350M | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.1,475 tok/s |
| StableLM 2 1.6B Chat | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.315 tok/s |
| StableLM 2 Zephyr 1.6B | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.315 tok/s |
| Falcon 7B Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.71.5 tok/s |
| d1 2B | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.269 tok/s |
| StarCoder2 3B | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s |
| LFM2-VL 450M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.301 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.301 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,147 tok/s |
| LFM2 350M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,475 tok/s |
| Granite 4.0 H 350M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.398 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.398 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,518 tok/s |
| GLiDE 0.8B | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.688 tok/s |
| Qwen1.5 110B Chat | Slowover GPU memory limit36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.7.2 tok/s | Slow49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.5.2 tok/s | Very slowheavy RAM offload36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.2.9 tok/s |
| Qwen2.5-Math 1.5B Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| SmolVLM 500M Instruct | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.266 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.266 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,012 tok/s |
| Qwen2.5 0.5B Instruct | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| Devstral 2 (123B) | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.6.4 tok/s | Slowtight fit81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.4.6 tok/s | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| LFM2.5 230M | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.589 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.589 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.2,244 tok/s |
| Hunyuan 0.5B Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.251 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.251 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.956 tok/s |
| Yi Coder 1.5B Chat | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.349 tok/s |
| SmolLM2 360M Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| CodeGemma 2B | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.206 tok/s |
| TinyLlama 1.1B Chat | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.469 tok/s |
| Qwen2 0.5B Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| ERNIE 4.5 0.3B | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| SmolVLM 256M Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,985 tok/s |
| Gemma 3 270M | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.502 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.502 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,911 tok/s |
| SmolLM 360M Instruct | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| DeepSeek Coder 1.3B Instruct | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.382 tok/s |
| LLM4Decompile 1.3B V2 | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.382 tok/s |
| surya ocr 2.5 | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.748 tok/s |
| SmolLM2 135M Instruct | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3,970 tok/s |
| Falcon-H1 Tiny 90M Instruct | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,505 tok/s | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,505 tok/s | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.5,734 tok/s |
| SmolLM 135M Instruct | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.3,970 tok/s |
| Qwen2.5 Coder 0.5B Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| gpt-oss 120B | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Runs great84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K3.1 Pro (2.8T) | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4.1 Flash 1017 (552B-A16B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.3 (753B-A40B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.8 Max (2.4T-A95B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K3 (2.8T) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| LongCat 2.5 (1.6T-A48B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.6 Pro (1T-A42B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.1 Flash 1019 (560B-A25B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M3.1 Flash (428B-A23B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nex N2.6 (321B-A18B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Step 5 Preview (600B-A27B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.3 Flash (320B-A18B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.6 Flash (310B-A15B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4.1 Flash (552B-A16B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 4 (1T-A52B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.8 Flash Next (125B-A6B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.2 (753B-A40B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hunyuan Hy4 Preview (780B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Muse Spark Open (816B-A42B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Pro (1.6T-A49B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.6 (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.1 Flash (560B-A25B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.5 Pro (1T-A42B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Intern-S2 (397B-A17B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.1 (744B-A40B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nex N2 Pro (397B-A17B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Pro (April 2026) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Naive N0.5 Flash (309B-A15.5B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.5 397B-A17B | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M3 (427B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5 (744B-A40B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| K2-Horizon (375B-A23B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hunyuan Hy3 (295B-A21B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Beam 1.0 (501B-A23B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.5 (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.7 Code (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Flash (284B-A13B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.5 (310B-A15B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ornith 1.5 397B-A17B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.2 (671B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Inkling Small (276B-A12B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.7 (355B-A32B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Inkling (975B-A41B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.7 (230B-A10B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Step 3.7 Flash (198B-A11B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.67.9 tok/s | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3 Ultra 550B-A55B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek R1 0528 (671B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Thinking (1T) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.2 Exp | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 2.6 1T | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.5 (230B-A10B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek R1 (671B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Flash (April 2026) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Step 3.5 Flash (196B-A11B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.67.9 tok/s | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.1 Terminus | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.1 (671B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Instruct 0905 (1T) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.1 (230B-A10B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 235B-A22B Thinking 2507 | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 Coder 480B-A35B | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2 Flash (309B-A15B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 1T | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 3 (675B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 235B-A22B Instruct 2507 | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Solar Open 2 250B | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Instruct (July 2025) | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2 (230B-A10B) | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3-VL 235B-A22B Instruct | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A+ (218B-A25B) | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.28.8 tok/s | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| K-EXAONE 236B-A23B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.31.0 tok/s | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| LongCat Flash Chat (560B) | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 235B-A22B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| InternVL3.5 241B-A28B | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.26.0 tok/s | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 3.1 Nemotron Ultra 253B v1 | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Baichuan M3 235B-A22B | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.32.3 tok/s | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.6 (355B-A32B) | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3 0324 (671B) | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax-M1 80K | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.5 (355B-A32B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax-M1 40K | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Trinity Large Thinking (400B-A13B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| ERNIE 4.5 300B-A47B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hermes 4 405B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| NorthCUA Pro | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| dots.llm1 Instruct | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax-Text-01 | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Tülu 3 405B | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3 (671B) | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek Coder V2 (236B) | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek Coder V2 Instruct (0614) | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V2.5 1210 | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 3.1 405B Instruct | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 4 Maverick (17B x 128E) | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V2.5 | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Jamba 1.5 Large | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V2 Chat | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.33.6 tok/s | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mixtral 8x22B Instruct | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit52Fair quality (C) · 52 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.14.1 tok/s | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— |
| Grok-1 (314B) | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— |
| Snowflake Arctic Instruct | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— |
| Falcon 180B Chat | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— | Slow23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.5.3 tok/s | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— |
Image models
61 of 61 image models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
61 of 61 image models run on the Apple M4 Max (128 GB). 96.0 GB usable for models.
60 of 61 image models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | M4 Max (64 GB)fit · speed · quality | M4 Max (128 GB)fit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| FLUX.2 [klein] 9B | Runs great84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.1.7 s |
| Krea 2 Turbo | Runs well92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs well92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| ERNIE-Image Turbo | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.19 s | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.19 s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.2.7 s |
| Z-Image Turbo | Runs well84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15 s | Runs well84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.15 s | Runs great84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.2.1 s |
| Qwen-Image 2.1 Turbo | Slow87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.61 s | Slow89Excellent quality (A+) · 89 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.68 s | Runs welltight fit87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.8.7 s |
| FLUX 3 Image Klein | Runs well91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.33 s | Runs well91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.33 s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.4.8 s |
| Fibo 2.0 Dev | Slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Runs welltight fit93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.19 s |
| Ideogram 4 | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Runs welltight fit90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.19 s |
| FLUX.2 [klein] 4B | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.5.9 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.5.9 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.0.8 s |
| ERNIE-Image | Slow88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slow88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| Ming-Image 0.1 Design | Slow84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.86 s | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Runs wellencoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| FLUX.1 Krea [dev] | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.70 s | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.70 s | Runs great80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.9.0 s |
| MiniMax H3 Image | Slowencoder unloaded after use91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.55 s | Slow91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.53 s | Runs greatCPU offload91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.9.8 s |
| Stable Diffusion 3.5 Large Turbo | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.9.4 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.9.4 s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.1.5 s |
| Qwen-Image 2512 | Very slow89Excellent quality (A+) · 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.1 min | Very slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.8.8 min | Slow83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.1.5 min |
| Qwen-Image 2.1 | Very slowtight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.6.7 min | Very slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.5.9 min | Slowtight fit · encoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.63 s |
| FLUX.1 [dev] | Slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.70 s | Slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.70 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.9.0 s |
| MOSS-Image SFT | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.82 s | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.82 s | Runs great80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11 s |
| FLUX.1 Kontext [dev] | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| Qwen-Image | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.9 min | Very slow88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.8.8 min | Slowencoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71 s |
| FireRed-Image-Edit 1.1 | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.9.0 min | Very slow87Excellent quality (A+) · 87 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.7.8 min | Slow80Great quality (A) · 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.80 s |
| Qwen-Image-Edit 2511 | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.9.0 min | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.8 min | Slow79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.80 s |
| HiDream-I1 Dev | Slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.90 s | Slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Runs well74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 s |
| HiDream-I1 Full | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.1 min | Very slow84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.4.4 min | Runs wellencoder in RAM82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.36 s |
| HiDream-I1 Fast | Slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.53 s | Slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.59 s | Runs great70Great quality (A) · 70 of 100: how capable the model is at NF4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.2 s |
| FLUX.1 [schnell] | Runs great68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.1.9 s |
| FLUX.1 Fill [dev] | Slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.73 s | Slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.73 s | Runs great74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.9.4 s |
| FLUX.2 [dev] | Very slowencoder unloaded after use91Excellent quality (A+) · 91 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.4.2 min | Very slow91Excellent quality (A+) · 91 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.0 min | Runs wellencoder in RAM85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38 s |
| Chroma1-HD | Slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.2.0 min | Slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.2.0 min | Runs well76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.15 s |
| Z-Image | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Runs well82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| CogView4 6B | Slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| Stable Diffusion 3.5 Large | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.64 s | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.64 s | Runs great64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.8.3 s |
| Kolors | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.23 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.23 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.3.3 s |
| Iris-3B | Slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Runs well72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 s |
| DreamShaper XL v2 Turbo | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.7.6 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.7.6 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.1 s |
| HunyuanImage 3.0 (80B-A13B) | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.6.9 min | Very slowtight fit88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.3 min | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| PixArt-Σ XL 1024 | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.8.8 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.8.8 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.1.3 s |
| SDXL-Lightning (4-step) | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.4.1 s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.4.1 s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.0.6 s |
| OmniGen2 | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Runs great64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.12 s |
| Cosmos3-Super Text2Image (64B) | Very slowCPU offload90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.4.6 min | Very slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.3.5 min | Runs wellCPU offload90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39 s |
| Stable Diffusion 3.5 Medium | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35 s | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.35 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.5.0 s |
| Juggernaut XL v9 | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| RealVisXL V5.0 | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| Animagine XL 4.0 | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| Illustrious XL v0.1 | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| Kandinsky 3 | Slow48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.53 s | Slow48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.53 s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.7.5 s |
| AuraFlow v0.3 | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.82 s | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.82 s | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.11 s |
| Stable Diffusion 3 Medium | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.2.6 s |
| Pony Diffusion V6 XL | Runs well56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs well56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| Lumina-Image 2.0 | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.5.9 s |
| Animagine XL 3.1 | Runs well54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.22 s | Runs well54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.22 s | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| Playground v2.5 | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.3 s |
| SDXL Turbo | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.0.9 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.0.9 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.0.1 s |
| Stable Diffusion XL 1.0 | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.24 s | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.24 s | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| Stable Cascade | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.2 s |
| Hunyuan-DiT v1.2 | Runs well52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.35 s | Runs well52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.35 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.5.0 s |
| Realistic Vision V5.1 | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| DreamShaper 8 | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| Stable Diffusion 1.5 | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| LCM DreamShaper v7 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.0.2 s |
| SD Turbo | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.0.5 s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.0.5 s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.0.1 s |
Video models
42 of 43 video models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
43 of 43 video models run on the Apple M4 Max (128 GB). 96.0 GB usable for models.
40 of 43 video models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | M4 Max (64 GB)fit · speed · quality | M4 Max (128 GB)fit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| LTX-2.5 (22B, video + audio) | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.10.7 min | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.8.0 min | Runs greatencoder in RAM81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.1.6 min |
| LTX-Video 13B 0.9.8 (distilled) | Runs well66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.3 min | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.5.9 min | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.45 s |
| LTX-2.3 | Slowtight fit84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.39.1 min | Runs wellencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.0 min |
| Wan 2.2 I2V A14B | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.0 h | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.6 h | Slowencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.28.1 min |
| Wan 2.2 T2V A14B | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.0 h | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.6 h | Slowencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.28.1 min |
| HunyuanVideo 1.5 (8.3B) | Very slow82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.6 h | Very slow84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.2.9 h | Slowtight fit · encoder in RAM82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.23.7 min |
| LTX-2 (19B, video + audio) | Slowtight fit80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.39.1 min | Runs wellencoder in RAM76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.7.0 min |
| Bernini-R A14B (video generation + editing) | Very slow83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.3 min | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.70.3 min | Runs well83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.0 min |
| SCAIL-2 14B (character animation) | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.8 h | Very slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Slowtight fit78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.15.1 min |
| Wan 2.1 I2V 14B 720P | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.7 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.3 h | Slowencoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.35.1 min |
| Wan 2.1 T2V 14B | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.7 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.3.3 h | Slowencoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.35.1 min |
| FramePack I2V (HunyuanVideo 13B) | Slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Slow74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.1 min | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.0 min |
| MiniMax H3 (33B, video + audio) | Very slowtight fit · encoder unloaded after use89Excellent quality (A+) · 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Very slow89Excellent quality (A+) · 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.1.9 h | SlowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.0 min |
| Wan 2.2 S2V 14B (speech to video) | Very slow75Great quality (A) · 75 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.1.9 h | Very slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.6 h | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.12.5 min |
| Kandinsky 6.0 Video Lite (3B, video + audio) | Slow68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.30.3 min | Slow68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.30.3 min | Runs wellencoder in RAM68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.5 min |
| Wan 2.2 TI2V 5B | Very slow68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.45.7 min | Very slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.50.8 min | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.8.7 min |
| Wan 2.2 Animate 14B | Very slowtight fit77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.6 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.3.1 h | Slowtight fit · encoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.33.4 min |
| Wan 2.1 VACE 14B | Very slowtight fit77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.4.1 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.6 h | Slowtight fit · encoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.38.6 min |
| SkyReels V2 T2V 14B 720P | Very slowtight fit76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.5 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.9 h | SlowCPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| HY-OmniWeaving (8.3B, unified video) | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.46.6 min | Very slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.51.8 min | Runs well74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.6.7 min |
| SkyReels V2 T2V 14B 540P | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.1 h | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.3 h | Slowtight fit72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.17.6 min |
| Wan 2.1 I2V 14B 480P | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.79.1 min | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.87.9 min | Runs well72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.11.3 min |
| Cosmos-Predict2 14B Video2World | Very slow68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.2.3 h | Very slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.6 h | SlowCPU offload68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.26.1 min |
| HunyuanVideo 13B | Very slow66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.7 h | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.9 h | SlowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| HunyuanVideo I2V 13B | Very slow66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.7 h | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.9 h | SlowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| Cosmos-Predict2 2B Video2World | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.24.4 min | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.24.4 min | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.1 min |
| Open-Sora 2.0 (11B) | Very slowtight fit70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.9 h | Very slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.4.9 h | Very slowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.54.4 min |
| LTX-Video 2B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.2.4 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.2.4 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.21 s |
| CogVideoX 1.5 5B | Very slow58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.79.1 min | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.87.9 min | Won't run58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| CogVideoX 5B I2V | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.17.6 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.17.6 min | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.5 min |
| Step-Video T2V (30B) | Very slowtight fit · encoder unloaded after use74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.4.8 h | Very slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.5.0 h | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.50.6 min |
| Mochi 1 Preview (10B) | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.1.6 h | Very slow62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.1.8 h | Slow60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.13.8 min |
| CogVideoX 5B | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.17.6 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.17.6 min | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.5 min |
| Kandinsky 6.0 Pro (29B, video + audio) | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.6.2 h | Very slowtight fit88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.4.8 h | Very slowtight fit · CPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.53.3 min |
| Allegro (2.8B) | Very slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Very slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Slowtight fit · encoder in RAM52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.17.6 min |
| Cosmos 1.0 Diffusion 7B Text2World | Very slowencoder unloaded after use60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.56.4 min | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.53.7 min | Runs welltight fit · CPU offload60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.10.0 min |
| Wan 2.1 VACE 1.3B | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.4 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.25.4 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.3 min |
| Wan 2.1 T2V 1.3B | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.23.4 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.23.4 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.0 min |
| SkyReels V2 Diffusion Forcing 1.3B | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.29.3 min | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.29.3 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.8 min |
| Wan-Dancer 14B (music to dance) | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.8.4 h | Very slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.7.2 h | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— |
| CogVideoX 2B | Runs well44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.5.3 min | Runs well44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.5.3 min | Runs great45Fair quality (C) · 45 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.50 s |
| AnimateDiff v3 (SD 1.5) | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.2.0 min | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.2.0 min | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| Cosmos3-Super Image2Video (64B) | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.4.8 h | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
Decision models
69 of 70 decision models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
70 of 70 decision models run on the Apple M4 Max (128 GB). 96.0 GB usable for models.
68 of 70 decision models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | M4 Max (64 GB)fit · speed · quality | M4 Max (128 GB)fit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| Jina Reranker v3.5 | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| Jina Reranker v3 | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| d1 3B | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.97 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.97 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.21 ms |
| mxbai-rerank-large v2 | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.84 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.84 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.17 ms |
| Intern-Decision 4B | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.121 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.121 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.28 ms |
| Qwen3 Reranker 4B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.198 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.198 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.39 ms |
| FRIDA-Decisions | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.27 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.27 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.7.5 ms |
| GLiNER2.5 Decide | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.6.1 ms |
| Qwen3 Reranker 0.6B | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| zerank-1 small | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.94 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.94 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.19 ms |
| zerank-2 | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.198 ms | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.198 ms | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.39 ms |
| LemonDecider 4B | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.115 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.115 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.23 ms |
| Shieldstral 1.0 3B | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.190 ms | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.190 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.38 ms |
| Llama Prompt Guard 2 86M | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms |
| mxbai-rerank-base v2 | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.30 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.30 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.8.3 ms |
| BGE Reranker v2 Gemma | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.125 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.125 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.26 ms |
| GLiClass ModernBERT Large v3 | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| OpenAI Privacy Filter | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.5.4 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.5.4 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.4.2 ms |
| Qwen3 Reranker 8B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.399 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.399 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.75 ms |
| BGE Reranker v2 M3 | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.36 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.36 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.8.8 ms |
| DeBERTa v3 Large Zero-shot v2.0 | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.5.9 ms |
| DuoGuard 0.5B | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.31 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.31 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.8.2 ms |
| Skywork Reward V2 Qwen3 0.6B | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.14 ms |
| GTE Reranker ModernBERT Base | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms |
| Jina Reranker v2 Base Multilingual | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms |
| ModernBERT Large Zero-shot v2.0 | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| Skywork Reward V2 Llama 3.1 8B | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.736 ms | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.736 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.115 ms |
| GLiGuard 300M | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.8 ms |
| Granite Embedding Reranker English R2 | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms |
| Laya | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.8 ms |
| Llama Prompt Guard 2 22M | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.8.0 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.8.0 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.4.6 ms |
| Qwen3Guard Gen 0.6B | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.71 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.71 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.18 ms |
| GLiClass Base v3 | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.9.3 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.9.3 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.4.8 ms |
| ProtectAI DeBERTa v3 Prompt Injection v2 | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.14 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.14 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.5.5 ms |
| GLiDE 27B | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.640 ms | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.640 ms | Won't run86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeBERTa v3 Base Zero-shot v2.0 | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.0 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.0 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.8 ms |
| GLiNER2 Base v1 | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.9 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.9 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.9 ms |
| Qwen3Guard Gen 4B | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.268 ms | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.268 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.85 ms |
| Skywork Reward V2 Qwen3 8B | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.743 ms | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.743 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.116 ms |
| HHEM 2.1 Open | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.9 ms |
| LettuceDetect Large | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.49 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.49 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.11 ms |
| Qwen3Guard Gen 8B | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.539 ms | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.539 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.103 ms |
| Laya Multilingual | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 ms | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 ms | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.4 ms |
| Llama 3.1 Nemotron Safety Guard 8B v3 | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.528 ms | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.528 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.102 ms |
| Granite Guardian 4.1 8B | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.551 ms | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.551 ms | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.94 ms |
| GLiNER Multi v2.1 | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.2 ms |
| NuExtract 2.0 2B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.145 ms | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.145 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.59 ms |
| NuExtract 2.0 4B | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.313 ms | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.313 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.97 ms |
| BART Large MNLI | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| WildGuard 7B | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.477 ms | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.477 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.92 ms |
| Granite Guardian 3.3 8B | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.538 ms | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.538 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.103 ms |
| ShieldGemma 2B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.130 ms | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.130 ms | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.27 ms |
| ms-marco MiniLM L6 v2 | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.4.2 ms |
| Llama Guard 3 1B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.102 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.102 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.32 ms |
| ShieldGemma 9B | Slow64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.450 ms | Slow64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.450 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.85 ms |
| Llama Guard 3 8B | Slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.528 ms | Slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.528 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.102 ms |
| ShieldGemma 27B | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.206 ms |
| CompassJudger-2 32B | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.19.7 s | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.19.7 s | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| Nemotron Content Safety Reasoning 4B | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.9 s | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.9 s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.346 ms |
| gpt-oss-safeguard 20B | Very slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.6 s | Very slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.6 s | Slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.812 ms |
| DynaGuard 8B | Very slow79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s | Very slow79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s | Slow78Great quality (A) · 78 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.918 ms |
| Selene 1 Mini 8B | Very slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s | Very slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s | Slow79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.902 ms |
| Llama Guard 4 12B | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.788 ms | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.788 ms | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.133 ms |
| CompassJudger-2 7B | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.6 s | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.6 s | Slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.856 ms |
| DynaGuard 1.7B | Slow69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.906 ms | Slow69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.906 ms | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.165 ms |
| Prometheus 2 8x7B | Very slow71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.5.7 s | Very slow74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.8.3 s | Very slowtight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.1.6 s |
| GLIDER | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.316 ms |
| Prometheus 2 7B | Very slow67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.3.4 s | Very slow67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.3.4 s | Slow67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.814 ms |
| Flow Judge v0.1 | Very slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Very slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.316 ms |
| gpt-oss-safeguard 120B | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Very slow80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.1 s | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— |
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.