M4 Pro (48 GB) vs M4 Max (64 GB) vs RTX 5090 for local AI
Pick two to four systems or devices to see which runs more models, faster, for the kind of work you do.
Winner: Apple M4 Max (64 GB)
For general chat: It runs 7 more LLMs (521 vs 514) (against the NVIDIA GeForce RTX 5090, the next best). On the other hand, the NVIDIA GeForce RTX 5090 is about 1.3× faster on its own top models.
 Apple M4 Pro (48 GB)
Remove
|
Apple M4 Max (64 GB)
Remove
|
 NVIDIA GeForce RTX 5090
Remove
|
|
|---|---|---|---|
| Memory | 48.0 GB unified | 64.0 GB unified | 32.0 GB VRAM |
| Usable for models | 36.0 GB | 48.0 GB | 32.0 GB |
| Memory bandwidth | 273 GB/s | 546 GB/s | 1,792 GB/s |
| FP16 compute | 9.2 TFLOPS | 18.4 TFLOPS | 104.8 TFLOPS |
| Launch price | — | — | $1,999 |
| Released | 2024 | 2024 | 2025 |
| LLMs that run | 492 of 622 | 521 of 622 | 514 of 622 |
| LLMs that run well | 469 | 504 | 493 |
| Image models that run | 61 of 61 | 61 of 61 | 60 of 61 |
| Video models that run | 41 of 43 | 42 of 43 | 40 of 43 |
| Decision models that run | 69 of 70 | 69 of 70 | 68 of 70 |
| Top-10 LLM average score | 100 | 100 | 100 |
| Top-10 LLM average speed | 55.5 tok/s | 89.4 tok/s | 114 tok/s |
| Best LLM | MiniCPM-V 4.7 35B-A3B | Qwen3.5 35B-A3B | Nanbeige4.3 33B |
Highlighted values are the best of the 3. Model results assume 8K of context and the default macOS GPU memory limit on Macs.
LLMs
492 of 622 LLMs run on the Apple M4 Pro (48 GB). 36.0 GB usable for models.
521 of 622 LLMs run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
514 of 622 LLMs run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | M4 Profit · speed · quality | M4 Maxfit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| MiniCPM-V 4.7 35B-A3B | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Qwen3.5 35B-A3B | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Qwen3.6 35B-A3B | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Nex N2 Mini (35B-A3B) | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs welltight fit91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Sage-A1.5 35B-A3B | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welltight fit90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Gemma 4 26B-A4B | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.51.4 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.76.3 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.139 tok/s |
| Ornith 1.5 35B-A3B | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welltight fit89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| GLM-4.7 Flash (30B-A3B) | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Raon-1 mini Preview (30B-A4B) | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.54.2 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s |
| Apodex 1.1 Mini (35B-A3B) | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Ornith 1.0 35B-A3B | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Flint (40B-A2.5B) | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.83.1 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.222 tok/s |
| Agents-A1 35B-A3B | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Qwen AgentWorld 35B-A3B | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Laguna XS 2.1 (33B-A3B) | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welltight fit81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Nemotron Cascade 2 30B-A3B | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| DiffusionGemma 26B-A4B IT | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.57.4 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| KAT-Coder V2.5 Dev (35B-A3B) | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Nemotron 3 Nano 30B-A3B | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| Qwen3 30B-A3B Instruct 2507 | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Qwen3-VL 30B-A3B Thinking | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| LFM2.5 24B-A2B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.48.8 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.82.4 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.192 tok/s |
| Qwen3 30B-A3B | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Qwen3-VL 30B-A3B Instruct | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Hunyuan Hy-Memory 30B-A3B | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great82Great quality (A) · 82 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Nemotron 3.5 Lightning 30B-A3B | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Qwen3.8 27B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| Qwen3.6 27B | Runs well89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| Sarvam 30B | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.1 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.83.5 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.207 tok/s |
| Qwen3.5 27B | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| North Mini Code 1.0 (30B-A3B) | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Kimi Linear 48B-A3B | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.233 tok/s |
| Qwen3-Omni 30B-A3B Instruct | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.57.7 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s |
| Qwen3 30B-A3B Thinking 2507 | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Qwen3 Coder 30B-A3B | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| TMax 27B | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Trinity Mini (26B-A3B) | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| ERNIE 4.5 21B-A3B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Fara 1.5 27B | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Nemotron 3.5 Bolt 74B-A9B | Runs welltight fit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs welloffloads to RAM77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s |
| Nanbeige4.3 33B | Runs well80Great quality (A) · 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11.3 tok/s | Runs well94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s | Runs welltight fit94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.46.1 tok/s |
| UI-Mate 27B | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Kolibri 1 (78B-A3.5B) | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.95.9 tok/s | Runs welltight fit76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.127 tok/s | Runs welloffloads to RAM76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.86.5 tok/s |
| Axion 1.0 35B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| gpt-oss 20B | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.66.6 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| ERNIE 4.5 21B-A3B Thinking | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Maple Preview (20B-A1B) | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.71.7 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.113 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.195 tok/s |
| NorthCUA Mini | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.90.1 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Gemma 4 31B | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.11.9 tok/s | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18.5 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s |
| Groot 19B A3B 261023 | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Kimi-VL A3B Thinking 2506 | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| LFM2 24B-A2B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.51.4 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.195 tok/s |
| Muse Glimmer 30B | Runs well77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.12.5 tok/s | Runs well87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.19.4 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.51.3 tok/s |
| Ling 3.0 Tiny (8B-A1.3B) | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.145 tok/s |
| Gemma 4 12B | Runs well78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.24.1 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great83Great quality (A) · 83 of 100: how capable the model is at F16, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s |
| Granite 4.0 H Small (32B-A9B) | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.28.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.38.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.118 tok/s |
| K2 Horizon 7B | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.32.1 tok/s | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| Phi-3.5 MoE (42B-A6.6B) | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.38.1 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.157 tok/s |
| Devstral Small 2 (24B) | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Seed-OSS 36B Instruct | Runs well67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.60.9 tok/s |
| Nemotron 3.5 Bolt E4B | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s |
| SenseNova Loop 9B Preview | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.53.4 tok/s |
| Hermes 4.3 36B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.0 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.60.9 tok/s |
| EXAONE 4.5 33B | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.9 tok/s | Runs well77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs welltight fit81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s |
| Ornith 1.5 9B | Runs well73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Mellum2.1 Thinking (12B-A2.5B) | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.48.0 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| Gemma 3 27B IT | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.55.7 tok/s |
| Ling Mini 2.0 (16B-A1.4B) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.75.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.198 tok/s |
| Gemma 4 E4B | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s |
| Qwen3.5 9B | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Qwen3-VL 32B Instruct | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.11.2 tok/s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.45.8 tok/s |
| Ornith 1.0 9B | Runs well71Great quality (A) · 71 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Qwen3 Coder Next (80B-A3B) | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs welloffloads to RAM72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| Kimi-VL A3B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| TMax 9B | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.32.2 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.57.7 tok/s |
| MedGemma 27B Text IT | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.21.4 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.5 tok/s |
| Llama 3.3 Nemotron Super 49B v1.5 | Slowtight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.5 tok/s | Runs well76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.57.0 tok/s |
| Nanbeige4.2 3B | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.48.0 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| Qwen3 32B | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Devstral Small 1.1 (24B) | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Magistral Small 2507 | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Apriel 1.5 15B Thinker | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.19.4 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.75.4 tok/s |
| Terminal RSI 27B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Granite 4.2 30B | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.12.7 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.62.1 tok/s |
| Nemotron Labs Dory 1.0 4B | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.50.3 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s |
| Mistral Small 3.2 24B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Magistral Small 2506 | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Phi-4 Reasoning Vision 15B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.19.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.74.1 tok/s |
| Hemmingway 1.5 | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.21.5 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.56.7 tok/s |
| Granite 4.2 8B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.32.8 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s |
| Spark-X2.6 4B | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71.5 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.126 tok/s |
| Moonlight 16B-A3B Instruct | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Mistral Small 3.1 24B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| GLM-4 32B 0414 | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Gemma 4 E2B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.58.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.224 tok/s |
| Qwen3 14B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.39.0 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.75.7 tok/s |
| Granite 4.1 30B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.12.9 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.20.0 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.62.9 tok/s |
| Qwen2.5 32B Instruct | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Baichuan M2 32B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Gemma 2 27B IT | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.21.2 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.1 tok/s |
| Olmo 3.1 32B Instruct | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| EXAONE 4.0 32B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.11.7 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.8 tok/s |
| Mistral Small 3 24B (2501) | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Qwen2.5-VL 32B Instruct | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.11.2 tok/s | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s |
| Hermes 4 14B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| MiMo V2.6 Distill Qwen 9B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| Qwen3.5 4B | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.111 tok/s |
| Voxtral Small 24B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.11.9 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.23.8 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.62.9 tok/s |
| Dolphin Mistral 24B Venice Edition | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Nous Hermes 2 Mixtral 8x7B DPO | Runs welltight fit49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs welltight fit49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| Phi-4 (14B) | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.19.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.76.2 tok/s |
| InternVL3 14B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.19.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.74.1 tok/s |
| TMax 4B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.47.6 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.69.8 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s |
| Ministral 3 14B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.20.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s |
| InternVL3.5 8B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.33.8 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.60.5 tok/s |
| Jamba 1.5 Mini | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.22.2 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Qwen3-VL 8B Instruct | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.32.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.58.8 tok/s |
| Agents-A1 4B | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.44.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Magistral Small 1.2 (24B) | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.12.0 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| DeepSeek Coder V2 Lite (16B) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| LFM2.5 8B-A1B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.137 tok/s |
| QwQ 32B | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Dolphin 3.0 R1 Mistral 24B | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Mixtral 8x7B Instruct | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.20.8 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| MiniCPM5 2B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.58.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.53.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.205 tok/s |
| DeepSeek V2 Lite Chat | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| Qwen2.5 14B Instruct | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.19.6 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.76.2 tok/s |
| Granite 4.0 H Tiny (7B-A1B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.52.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.85.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.158 tok/s |
| Solar Pro Preview 22B | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.13.0 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.26.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.50.6 tok/s |
| Qwen3 8B | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.35.2 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| Granite 4.1 8B | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.32.8 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s |
| InternVL3.5 38B | Runs well53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.12.3 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s | Runs welltight fit73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.47.3 tok/s |
| MiniCPM-V 4.5 8B | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.33.2 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.46.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.59.3 tok/s |
| Falcon-H1 34B Instruct | Runs well53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.2 tok/s | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s |
| OpenReasoning Nemotron 32B | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Skywork OR1 32B | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Fara 1.5 4B | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.44.1 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Mage-VL 4.7B | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| Phi-4 Reasoning Plus (14B) | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s |
| Qwen2.5 14B Instruct 1M | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| MiMo-VL 7B RL | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.34.7 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.62.1 tok/s |
| Nemotron Nano 9B v2 | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.32.5 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s |
| Command R (08-2024) 32B | Runs well52Fair quality (C) · 52 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s |
| EXAONE Deep 32B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.7 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.18.0 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.8 tok/s |
| GLM-4.1V 9B Thinking | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.28.0 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.38.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s |
| Qwen2 57B-A14B Instruct | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.122 tok/s |
| Olmo 3.1 32B Think | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs well72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| Gemma 3 12B IT | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.23.7 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| Phi-4 Reasoning (14B) | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s |
| MiMo 7B RL | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.36.9 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.51.1 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.65.9 tok/s |
| Qwen3-VL 4B Instruct | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.45.1 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s |
| SenseNova 6.8 Code2Video | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.5 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.21.1 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Qwen3 4B Instruct 2507 | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| UI-Mate 9B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| UI-Venus 2 9B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| Aya Vision 32B | Runs well50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.3 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.4 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| DeepSeek R1 Distill Qwen 32B | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Aya Expanse 32B | Runs well50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s |
| Ministral 3 8B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.32.4 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.57.9 tok/s |
| InternVL3 8B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.36.3 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s |
| InternVL3 38B | Runs well49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.12.3 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.15.0 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.47.3 tok/s |
| Devstral Small 2505 | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.12.2 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.24.5 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| OLMo 2 32B Instruct | Runs well50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.11.6 tok/s | Runs well57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.9 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| GLM-Z1 32B (0414) | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.17.7 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Mistral Small 22B (2409) | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.13.0 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| LFM3 1.7B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.84.9 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.298 tok/s |
| Granite 4.2 3B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s |
| Hermes 4 70B | Slowtight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Llama 3.3 70B Instruct | Slowtight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| Llama 3.1 Nemotron Nano 8B v1 | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| OpenHands LM 32B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Reka Flash 3.1 (21B) | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.13.8 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.27.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| Qwen2.5 72B Instruct | Slowtight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| Qwen2.5-VL 72B Instruct | Slowtight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| DeepSeek R1 Distill Llama 70B | Slowtight fit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Tülu 3 70B | Slowtight fit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Hunyuan 7B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.38.5 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.2 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.68.8 tok/s |
| Apriel Nemotron 15B Thinker | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.19.3 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.38.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.74.8 tok/s |
| MiniCPM4 8B | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.35.2 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| AceReason Nemotron 14B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| OpenReasoning Nemotron 14B | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Qwen2.5 Coder 14B Instruct | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Ministral 3 14B Reasoning | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.20.7 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s |
| Qwen3 4B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| Qwen2.5 7B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Olmo 3 7B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| Qwen2.5 Coder 32B Instruct | Runs well56Good quality (B) · 56 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Mage-VL-Cowork 4.7B | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| Llama 3.1 70B Instruct | Slowtight fit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| Codestral 22B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.13.0 tok/s | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| Command R 35B (v01) | Runs welltight fit45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s | Runs welltight fit45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.81.3 tok/s |
| Qwen3-Next 80B-A3B Thinking | Runs wellover GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs welloffloads to RAM72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.88.4 tok/s |
| Qwen2.5-VL 7B Instruct | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.34.8 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| Gemma 3n E4B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.50.1 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.129 tok/s |
| Falcon 3 10B Instruct | Runs well49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.28.0 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.38.8 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.50.1 tok/s |
| DeepCoder 14B Preview | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| QwQ 32B Preview | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.11.4 tok/s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.17.6 tok/s | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| MiniCPM-o 2.6 8B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.33.3 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.46.2 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.59.5 tok/s |
| DeepSeek R1 0528 Qwen3 8B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.35.2 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| Nemotron 3 Nano 4B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.74.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s |
| Foundation-Sec 1.1 8B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| InternVL2.5 8B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.35.7 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s |
| Mistral Nemo 12B Instruct | Runs well49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.23.7 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| Hermes 3 (Llama 3.1 70B) | Slowtight fit47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Gemma 2 9B IT | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.31.2 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.43.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.55.9 tok/s |
| GLM-4 9B 0414 | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Qwen2.5 7B Instruct 1M | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Qwen3 Next 80B-A3B Instruct | Runs wellover GPU memory limit58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs welloffloads to RAM70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.88.4 tok/s |
| NorthCUA Nano | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| QVQ 72B Preview | Slowtight fit47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs welloffloads to RAM47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| InternLM2.5 20B Chat | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.14.5 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| Falcon-H1 7B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.38.0 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.52.8 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| GLiDE 9B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.36.3 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s |
| LFM3-VL 2B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.269 tok/s |
| DeepSeek R1 Distill Qwen 14B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Llama 3.1 Nemotron 70B Instruct | Slowtight fit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| InternLM3 8B Instruct | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.32.8 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.45.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.58.6 tok/s |
| Apertus v1.5 70B | Slowtight fit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Runs welltight fit55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.4 tok/s | Runs welloffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.19.9 tok/s |
| JanusCoder 14B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.19.5 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Hermes 3 (Llama 3.1 8B) | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Llama 3.1 8B Instruct | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Pixtral 12B (2409) | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.23.3 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.38.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.90.3 tok/s |
| Granite 4.1 3B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.152 tok/s |
| DeepSeek MoE 16B Chat | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.43.7 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.75.0 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.159 tok/s |
| Aya 23 35B | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.81.3 tok/s |
| LLM4Decompile 22B V2 | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.13.0 tok/s | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.25.9 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| Yi 1.5 34B Chat | Runs well42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs well47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.64.0 tok/s |
| MiniCPM-V 2.6 8B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.6 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.7 tok/s |
| Kimi Dev 72B | Slowtight fit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs welloffloads to RAM56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| Qwen2 72B Instruct | Slowtight fit44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs welloffloads to RAM44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| Qwen2-VL 72B Instruct | Slowtight fit44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s | Runs welloffloads to RAM44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| Falcon 3 7B Instruct | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.69.2 tok/s |
| Qwen3 4B Thinking 2507 | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.8 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| LFM2 8B-A1B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.137 tok/s |
| EXAONE 3.5 7.8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.36.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Granite 3.2 8B Instruct | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Granite 3.3 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Ministral 8B (2410) | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.36.0 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.4 tok/s |
| GLM-4 9B Chat 1M | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.30.4 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.4 tok/s |
| Seed-Coder 8B Reasoning | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.0 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.62.6 tok/s |
| DeepHermes 3 Llama 3 8B Preview | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Dolphin 3.0 Llama 3.1 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Tülu 3 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Llama 3.1 Nemotron Nano 4B v1.1 | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.65.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| GLM-4 9B Chat (2024) | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Gemma 3 4B IT | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s |
| Phi-4 Multimodal Instruct | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.92.7 tok/s |
| Apertus v1.5 8B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.32.4 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.45.0 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.58.0 tok/s |
| Phi-4 Mini (3.8B) | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Phi-3 Small 8K Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.39.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.69.8 tok/s |
| OLMo 2 13B Instruct | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.21.0 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.81.6 tok/s |
| Ministral 3 3B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| EXAONE Deep 7.8B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.36.9 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| North Micro Vision | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.59.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.54.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.208 tok/s |
| MedGemma 4B IT | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.68.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s |
| LFM2.5-VL 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.47.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.165 tok/s |
| Phi-3 Medium 128K Instruct | Runs well43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.20.7 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.80.2 tok/s |
| Jamba Reasoning 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.45.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.42.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| Granite 3.1 8B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Phi-3.5 Mini Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Seed-Coder 8B Instruct | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.0 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.62.6 tok/s |
| Dolphin 2.9.4 Llama 3.1 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Hermes 2 Pro Llama 3 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| OpenChat 3.6 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| InternVL2 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.35.7 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s |
| GLM-Z1 9B (0414) | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Apertus 70B Instruct | Slowtight fit42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| Olmo 3 7B Think | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| Falcon-H1R 7B | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.38.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| LFM2.5 2.6B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.54.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.191 tok/s |
| Hunyuan 4B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.69.6 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.122 tok/s |
| Qwen2 7B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Qwen1.5 MoE A2.7B Chat | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.45.1 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| Qwen2.5-Omni 7B | Runs well43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.26.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| Phi-3 Medium (14B) | Runs well43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.20.7 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.80.2 tok/s |
| Command R7B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Qwen2.5 Coder 7B Instruct | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| SmolLM3 3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.47.7 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| Qwen2-VL 7B Instruct | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.34.8 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| Qwen1.5 32B Chat | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.11.5 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.17.8 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Llama 3 70B Instruct | Slowtight fit40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.6.7 tok/s | Runs welltight fit48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.10.6 tok/s | Runs welloffloads to RAM40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Phi-3.5 Vision Instruct | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| Aya Vision 8B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.33.4 tok/s | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.46.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.59.8 tok/s |
| Llama 3 8B Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Mistral-NeMo-Minitron 8B Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.34.3 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.47.6 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.61.4 tok/s |
| Jamba2 3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s |
| LLaVA 1.6 34B | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.16.6 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.3 tok/s |
| OpenReasoning Nemotron 7B | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| OLMoE 1B-7B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.145 tok/s |
| InternLM2 Chat 20B | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.14.5 tok/s | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| CogVLM2 Llama 3 Chat 19B | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.14.8 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.6 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.57.4 tok/s |
| Granite 4.0 Micro (3B) | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.152 tok/s |
| AFM 4.5B | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.43.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.112 tok/s |
| Command R7B Arabic | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Qianfan-OCR 4B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| Llama 3.2 11B Vision Instruct | Runs well39Basic quality (D) · 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.27.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s |
| InternLM2.5 7B Chat | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.37.3 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s |
| Foundation-Sec 8B Reasoning | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great51Fair quality (C) · 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Aya Expanse 8B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Apertus 8B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.8 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.1 tok/s |
| Phi-3 Mini 128K Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Qwen1.5 14B Chat | Runs well32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.20.4 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.79.0 tok/s |
| MiniCPM-V 4.7 1B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.43.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.333 tok/s |
| Gemma 3n E2B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.258 tok/s |
| Granite 3.0 8B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Mistral 7B Instruct v0.3 | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.2 tok/s |
| Qwen3.5 2B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.59.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.227 tok/s |
| OLMo 2 7B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| Zamba2 7B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.1 tok/s |
| DeepSeek R1 Distill Llama 8B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Unlimited OCR 2 (3B-A0.6B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.75.5 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s |
| Phi-3 Vision 128K Instruct | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| Yi 34B Chat | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.16.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.0 tok/s |
| TMax 2B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.78.1 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.275 tok/s |
| Granite 4.0 H Micro | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.46.1 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.162 tok/s |
| Dolphin 2.9 Llama 3 8B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| MiniCPM3 4B | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.49.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.127 tok/s |
| Voxtral Mini 3B | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.62.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.110 tok/s |
| Nous Hermes 2 SOLAR 10.7B | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| SOLAR 10.7B Instruct | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26.9 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| S5 nano | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.111 tok/s |
| Phi-3 Mini 4K Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| EXAONE 3.0 7.8B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.36.9 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.51.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Falcon3 Mamba 7B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.7 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.0 tok/s |
| LightOnOCR-3 4B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.44.1 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.64.7 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Chandra OCR 2 | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.37.8 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.55.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.97.4 tok/s |
| Fara 7B | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.34.8 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.48.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| Hermes 2 Pro Mistral 7B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Starling LM 7B Beta | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Hunyuan A13B (80B-A13B) | Runs wellover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.27.8 tok/s | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs welloffloads to RAM62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.25.1 tok/s |
| Llama 3.2 3B Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.45.8 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| Falcon-H1 3B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.164 tok/s |
| OpenChat 3.5 (0106) | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| WeVisOne-VL 4B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.60.8 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s |
| Raon 1 nano | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.83.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.146 tok/s |
| Qwen3-VL 2B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.69.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.242 tok/s |
| Phi-4 Mini Reasoning (3.8B) | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.52.1 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Mistral 7B Instruct v0.2 | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Nous Hermes 2 Mistral 7B DPO | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| OpenChat 3.5 (1210) | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| MiniCPM5 1B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.62.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.125 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.478 tok/s |
| Yi 1.5 9B Chat | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.32.7 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| Unlimited OCR (3B-A0.6B) | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.75.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s |
| Qwen2.5 3B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.47.5 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s |
| Aya 23 8B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| DeepSeek R1 Distill Qwen 7B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| DeepSeek Prover V2 7B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.41.8 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.74.7 tok/s |
| Qwen2.5-VL 3B Instruct | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.53.4 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.78.3 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s |
| Hermes 3 Llama 3.2 3B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.45.8 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| OpenHermes 2.5 Mistral 7B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Starling LM 7B Alpha | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| LFM2 2.6B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.57.2 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.201 tok/s |
| CodeGeeX4 All 9B | Runs great43Fair quality (C) · 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Gemma 1.1 7B IT | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.33.8 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| Qwen1.5 7B Chat | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.37.4 tok/s | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.66.9 tok/s |
| DeepSeek Coder 33B Instruct | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.11.2 tok/s | Runs well41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.17.3 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Mistral 7B Instruct v0.1 | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Falcon 3 3B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.45.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s |
| Qwen2.5-Math 72B Instruct | Slowtight fit40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Runs welltight fit48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.10.3 tok/s | Runs welloffloads to RAM40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.31.9 tok/s |
| LFM2.5 1.2B Instruct | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.57.9 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.441 tok/s |
| Qwen2.5-Omni 3B | Runs great29Basic quality (D) · 29 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.0 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.93.2 tok/s |
| RecurrentGemma 9B IT | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| DeepSeek LLM 67B Chat | Slowtight fit28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.7.0 tok/s | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs welltight fit28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| Gemma 7B IT | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.33.8 tok/s | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| Granite 3.2 2B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Granite 3.3 2B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Falcon Mamba 7B Instruct | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.7 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.71.0 tok/s |
| MiniCPM-OCR 1.0 1.3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.52.1 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.104 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.397 tok/s |
| EXAONE 3.5 2.4B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.61.0 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.56.2 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.214 tok/s |
| Qwen3 1.7B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.72.4 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.254 tok/s |
| EXAONE 4.0 1.2B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.52.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.403 tok/s |
| Neural Chat 7B v3.3 | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| TeleOCR | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.47.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.95.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.363 tok/s |
| LLaVA 1.6 Mistral 7B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.38.1 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.52.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.68.2 tok/s |
| PaddleOCR-VL 1.6 0.9B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.538 tok/s |
| Yi 1.5 6B Chat | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.85.2 tok/s |
| InternLM2 Math Plus 20B | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.14.5 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.29.1 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| DeepSeek Coder 6.7B Instruct | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Granite 3.1 2B Instruct | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Gemma 2 2B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.198 tok/s |
| Yi Coder 9B Chat | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.32.7 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| Zephyr 7B Beta | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Orca 2 13B | Runs well23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.28.7 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Granite 4.0 H 1B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.46.4 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.92.7 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.353 tok/s |
| WeVisOne VL 2B | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.55.5 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.212 tok/s |
| Llama 2 70B Chat | Slowtight fit25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.6.9 tok/s | Runs well30Basic quality (D) · 30 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs welltight fit25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s |
| Granite Vision 3.2 2B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.49.3 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s |
| CodeGemma 7B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.33.8 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.46.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| LLM4Decompile 6.7B V2 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Code Llama 34B Instruct | Runs well31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.65.2 tok/s |
| StarCoder2 15B Instruct | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.70.2 tok/s |
| Granite Code 34B Instruct | Runs well33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.11.1 tok/s | Runs well38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.17.1 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s |
| Falcon-H1 1.5B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.7 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.87.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.333 tok/s |
| LLM4Decompile 9B V2 | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.32.7 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| Llama 2 13B Chat | Runs well21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.28.7 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Granite 3.0 2B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.55.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.51.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s |
| Baichuan 2 13B Chat | Runs well21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.26.8 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.41.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.80.6 tok/s |
| Granite 3.1 3B-A800M Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.61.8 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.169 tok/s |
| Qwen2.5 Coder 3B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.47.5 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s |
| Orca 2 7B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| SmolVLM2 2.2B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.229 tok/s |
| LFM2 1.2B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.57.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.441 tok/s |
| SQLCoder 7B v2 | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Codestral Mamba 7B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s |
| Qwen2.5-Math 7B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.37.9 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.52.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| StableLM 2 12B Chat | Runs great23Basic quality (D) · 23 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.30.7 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.92.2 tok/s |
| Mathstral 7B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.71.2 tok/s |
| Qwen2.5 1.5B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| LFM2-VL 1.6B | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.42.9 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.85.7 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.327 tok/s |
| Qwen3.5 0.8B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.77.8 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.593 tok/s |
| Granite 3.0 3B-A800M Instruct | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.61.8 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.97.2 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.169 tok/s |
| Baichuan 2 7B Chat | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.38.4 tok/s | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.53.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.68.7 tok/s |
| Llama 2 7B Chat | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| SmolVLM 2.2B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.60.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.229 tok/s |
| Qwen2-VL 2B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.61.3 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.234 tok/s |
| Code Llama 13B Instruct | Runs well25Basic quality (D) · 25 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.28.7 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Granite Code 20B Instruct | Runs well34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.14.4 tok/s | Runs well34Basic quality (D) · 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.28.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Nemotron Mini 4B Instruct | Runs great21Basic quality (D) · 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.47.8 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.70.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s |
| Zamba2 2.7B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.55.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.50.9 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| InternLM2 Math Plus 7B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.37.3 tok/s | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.51.7 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s |
| Code Llama 70B Instruct | Slowtight fit30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.6.9 tok/s | Runs well36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.10.8 tok/s | Runs welltight fit30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s |
| Granite Code 8B Instruct (128K) | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.35.8 tok/s | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.49.7 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.64.1 tok/s |
| WizardLM 13B v1.2 | Runs great15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.36.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| DeepSeek LLM 7B Chat | Runs great17Basic quality (D) · 17 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.41.8 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.74.7 tok/s |
| Llama 3.2 1B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.54.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.416 tok/s |
| LLaVA 1.5 13B | Runs great15Basic quality (D) · 15 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.35.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.83.9 tok/s |
| Yi 6B Chat | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.39.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.85.2 tok/s |
| Hunyuan 1.8B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.82.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.288 tok/s |
| InternVL3 78B | Slowover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.5.1 tok/s | Runs well51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.12.1 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s |
| SmolLM2 1.7B Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.85.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| Vicuna 13B v1.5 | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.36.4 tok/s | Runs great19Basic quality (D) · 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.44.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| LLaVA 1.5 7B | Runs great15Basic quality (D) · 15 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.40.9 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s |
| Code Llama 7B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| StarCoder2 15B (base) | Runs well30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.36.2 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.70.2 tok/s |
| Gemma 3 1B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.516 tok/s |
| Vicuna 7B v1.5 | Runs great13Basic quality (D) · 13 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.43.6 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| StableLM Zephyr 3B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.52.5 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| InternLM2.5 1.8B Chat | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.77.7 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.273 tok/s |
| Falcon 3 1B Instruct | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.81.1 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.309 tok/s |
| WizardMath 7B v1.1 | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.40.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Phi-2 2.7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.52.8 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.186 tok/s |
| OLMo 7B Instruct | Runs great13Basic quality (D) · 13 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.74.9 tok/s |
| Qwen2 1.5B Instruct | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| LFM2 700M | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.183 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.697 tok/s |
| GLiDE 2B | Runs great17Basic quality (D) · 17 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.78.1 tok/s | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.275 tok/s |
| Falcon 40B Instruct | Runs well11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.11.3 tok/s | Runs well11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.22.7 tok/s | Runs great13Basic quality (D) · 13 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| Qwen2.5 Coder 1.5B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| InternLM 7B | Runs great11Basic quality (D) · 11 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s |
| OLMo 2 1B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.349 tok/s |
| Spark X2.6 1.7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.85.9 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| StarCoder2 7B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.41.0 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s |
| Granite Code 3B Instruct (128K) | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.148 tok/s |
| DeepScaleR 1.5B Preview | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.82.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.290 tok/s |
| Qwen3 0.6B | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.90.3 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.688 tok/s |
| SmolLM 1.7B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.85.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| Stable Code Instruct 3B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.52.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| Gemma 2B IT | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.58.5 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.206 tok/s |
| StableLM 2 1.6B Chat | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.315 tok/s |
| StableLM 2 Zephyr 1.6B | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.41.3 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.82.6 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.315 tok/s |
| Falcon-H1 0.5B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.260 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.992 tok/s |
| LFM2.5 350M | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.193 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.1,475 tok/s |
| StarCoder 15B | Runs well21Basic quality (D) · 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.23.6 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.36.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s |
| DeepSeek R1 Distill Qwen 1.5B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.82.5 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.76.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.290 tok/s |
| Qwen2.5-Math 1.5B Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.87.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| LFM2-VL 450M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.150 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.301 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,147 tok/s |
| GLiDE 0.8B | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.90.3 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.688 tok/s |
| LFM2 350M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.193 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,475 tok/s |
| Granite 4.0 H 350M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.199 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.398 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,518 tok/s |
| d1 2B | Runs great21Basic quality (D) · 21 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.76.5 tok/s | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.269 tok/s |
| Falcon 7B Instruct | Runs great5Basic quality (D) · 5 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.40.0 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.40.7 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.71.5 tok/s |
| SmolVLM 500M Instruct | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.133 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.266 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,012 tok/s |
| StarCoder2 3B | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.48.5 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.44.7 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s |
| Qwen2.5 0.5B Instruct | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| Yi Coder 1.5B Chat | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.45.7 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.349 tok/s |
| LFM2.5 230M | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.294 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.589 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.2,244 tok/s |
| Hunyuan 0.5B Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.125 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.251 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.956 tok/s |
| SmolLM2 360M Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| TinyLlama 1.1B Chat | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.61.5 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.469 tok/s |
| Qwen2 0.5B Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| ERNIE 4.5 0.3B | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| DeepSeek Coder 1.3B Instruct | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.382 tok/s |
| LLM4Decompile 1.3B V2 | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.382 tok/s |
| SmolVLM 256M Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.260 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,985 tok/s |
| CodeGemma 2B | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.58.5 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.206 tok/s |
| Gemma 3 270M | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.251 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.502 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,911 tok/s |
| SmolLM 360M Instruct | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.376 tok/s | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| surya ocr 2.5 | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.98.1 tok/s | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.748 tok/s |
| Llama 3.2 90B Vision Instruct | Slowfar over GPU memory limit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.4.5 tok/s | Runs welltight fit46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.10.7 tok/s | Slowoffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.5.4 tok/s |
| SmolLM2 135M Instruct | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3,970 tok/s |
| Falcon-H1 Tiny 90M Instruct | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.752 tok/s | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,505 tok/s | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.5,734 tok/s |
| SmolLM 135M Instruct | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.521 tok/s | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.1,042 tok/s | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.3,970 tok/s |
| Qwen2.5 Coder 0.5B Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.276 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| gpt-oss 120B | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K3.1 Pro (2.8T) | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4.1 Flash 1017 (552B-A16B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.3 (753B-A40B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.8 Max (2.4T-A95B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K3 (2.8T) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| LongCat 2.5 (1.6T-A48B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.6 Pro (1T-A42B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.1 Flash 1019 (560B-A25B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M3.1 Flash (428B-A23B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nex N2.6 (321B-A18B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Step 5 Preview (600B-A27B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.3 Flash (320B-A18B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.6 Flash (310B-A15B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4.1 Flash (552B-A16B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 4 (1T-A52B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.8 Flash Next (125B-A6B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.2 (753B-A40B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hunyuan Hy4 Preview (780B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Muse Spark Open (816B-A42B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Pro (1.6T-A49B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.6 (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.1 Flash (560B-A25B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.5 Pro (1T-A42B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Intern-S2 (397B-A17B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.1 (744B-A40B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nex N2 Pro (397B-A17B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Pro (April 2026) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Naive N0.5 Flash (309B-A15.5B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.5 397B-A17B | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M3 (427B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5 (744B-A40B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| K2-Horizon (375B-A23B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hunyuan Hy3 (295B-A21B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Beam 1.0 (501B-A23B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.5 (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.7 Code (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Flash (284B-A13B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.5 (310B-A15B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ornith 1.5 397B-A17B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3.5 Super VL 120B-A12B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Slowheavy RAM offload65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.24.4 tok/s |
| DeepSeek V3.2 (671B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Inkling Small (276B-A12B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.7 (355B-A32B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Inkling (975B-A41B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.7 (230B-A10B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Step 3.7 Flash (198B-A11B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3 Ultra 550B-A55B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek R1 0528 (671B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Thinking (1T) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ornith 1.5 122B-A10B | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Slowheavy RAM offload64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.28.4 tok/s |
| DeepSeek V3.2 Exp | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 2.6 1T | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.5 (230B-A10B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek R1 (671B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Flash (April 2026) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.5 122B-A10B | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Slowheavy RAM offload64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.28.4 tok/s |
| Step 3.5 Flash (196B-A11B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.1 Terminus | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.1 (671B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Instruct 0905 (1T) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.1 (230B-A10B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 235B-A22B Thinking 2507 | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 Coder 480B-A35B | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2 Flash (309B-A15B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 1T | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 3 (675B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 235B-A22B Instruct 2507 | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Solar Open 2 250B | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Instruct (July 2025) | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2 (230B-A10B) | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3-VL 235B-A22B Instruct | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A+ (218B-A25B) | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| K-EXAONE 236B-A23B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| LongCat Flash Chat (560B) | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3 Super 120B-A12B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Slowheavy RAM offload61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.24.6 tok/s |
| Qwen3 235B-A22B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Devstral 2 (123B) | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.6.4 tok/s | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| InternVL3.5 241B-A28B | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Laguna S 2.1 (118B-A8B) | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.73.5 tok/s | Slowheavy RAM offload60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s |
| Llama 3.1 Nemotron Ultra 253B v1 | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Baichuan M3 235B-A22B | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.6 (355B-A32B) | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3 0324 (671B) | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.6V | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Slowheavy RAM offload58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.27.8 tok/s |
| MiniMax-M1 80K | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A Reasoning (111B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.7.3 tok/s | Slowheavy RAM offload57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.3.3 tok/s |
| GLM-4.5 (355B-A32B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.0 Flash (127B-A5B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.99.9 tok/s | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax-M1 40K | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Small 4 (119B-A6B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s |
| Mistral Medium 3.5 (128B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.6.3 tok/s | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Trinity Large Thinking (400B-A13B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A (111B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.7.3 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.3 tok/s |
| ERNIE 4.5 300B-A47B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.5V | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.27.8 tok/s |
| Hermes 4 405B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 2.6 Flash (104B-A7.4B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.91.5 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Mistral Large 2.1 (2411) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Solar Open 100B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.63.4 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s |
| GLM-4.5 Air (106B-A12B) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.53.8 tok/s | Slowheavy RAM offload54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.26.7 tok/s |
| Mistral Large 2 (123B) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| NorthCUA Pro | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Pixtral Large 124B (2411) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ring Flash 2.0 (100B-A6B) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs welloffloads to RAM54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| dots.llm1 Instruct | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling Flash 2.0 (100B-A6B) | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs welloffloads to RAM53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| MiniMax-Text-01 | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Tülu 3 405B | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3 (671B) | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Sarvam 105B | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Slowheavy RAM offload52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.36.1 tok/s |
| DeepSeek Coder V2 (236B) | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Leanstral 119B-A6B | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.85.1 tok/s | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.44.2 tok/s |
| DeepSeek Coder V2 Instruct (0614) | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V2.5 1210 | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 3.1 405B Instruct | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 4 Maverick (17B x 128E) | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 4 Scout 17B-16E | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.40.4 tok/s | Slowheavy RAM offload50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.19.8 tok/s |
| Command R+ (08-2024) 104B | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.7.8 tok/s | Slowheavy RAM offload49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.3.6 tok/s |
| DeepSeek V2.5 | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Groot 65B A6B 261023 | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellover GPU memory limit49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Slowheavy RAM offload49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.47.9 tok/s |
| Jamba 1.5 Large | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.0 flash Sante | Won't run47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowfar over GPU memory limit47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.99.9 tok/s | Won't run47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command R+ 104B (04-2024) | Won't run44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.7.8 tok/s | Slowheavy RAM offload44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3.6 tok/s |
| DeepSeek V2 Chat | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mixtral 8x22B Instruct | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen1.5 110B Chat | Won't run36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Slowover GPU memory limit36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.7.2 tok/s | Very slowheavy RAM offload36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.2.9 tok/s |
| Grok-1 (314B) | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— |
| Snowflake Arctic Instruct | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— |
| Falcon 180B Chat | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— |
Image models
61 of 61 image models run on the Apple M4 Pro (48 GB). 36.0 GB usable for models.
61 of 61 image models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
60 of 61 image models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | M4 Profit · speed · quality | M4 Maxfit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| Z-Image Turbo | Runs well84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs well84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15 s | Runs great84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.2.1 s |
| ERNIE-Image Turbo | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.32 s | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.19 s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.2.7 s |
| FLUX.2 [klein] 4B | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.9 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.5.9 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.0.8 s |
| FLUX.2 [klein] 9B | Runs well82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs great84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.1.7 s |
| FLUX 3 Image Klein | Slow91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.56 s | Runs well91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.33 s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.4.8 s |
| Krea 2 Turbo | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.38 s | Runs well92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| Fibo 2.0 Dev | Very slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.3.7 min | Slow93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Runs welltight fit93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.19 s |
| Ideogram 4 | Very slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.3.7 min | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Runs welltight fit90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.19 s |
| Qwen-Image 2.1 Turbo | Slowtight fit87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Slow87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.61 s | Runs welltight fit87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.8.7 s |
| Z-Image | Very slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.2.8 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Runs well82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| FLUX.1 [schnell] | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.18 s | Runs great68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.1.9 s |
| ERNIE-Image | Very slow87Excellent quality (A+) · 87 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.4.2 min | Slow88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| Qwen-Image 2512 | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.17.7 min | Very slow89Excellent quality (A+) · 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.10.1 min | Slow83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.1.5 min |
| Chroma1-HD | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.3.3 min | Slow78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.2.0 min | Runs well76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.15 s |
| FireRed-Image-Edit 1.1 | Very slowtight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.15.1 min | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.9.0 min | Slow80Great quality (A) · 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.80 s |
| FLUX.1 Krea [dev] | Slow80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.70 s | Runs great80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.9.0 s |
| Qwen-Image-Edit 2511 | Very slowtight fit85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.15.1 min | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.9.0 min | Slow79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.80 s |
| FLUX.1 Kontext [dev] | Very slow80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.3.3 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| FLUX.1 [dev] | Slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.70 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.9.0 s |
| HiDream-I1 Full | Very slowtight fit82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.6.7 min | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.1 min | Runs wellencoder in RAM82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.36 s |
| Ming-Image 0.1 Design | Very slowencoder unloaded after use86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.2.8 min | Slow84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.86 s | Runs wellencoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| HiDream-I1 Dev | Very slowtight fit78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.2.5 min | Slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.90 s | Runs well74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 s |
| HiDream-I1 Fast | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.89 s | Slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.53 s | Runs great70Great quality (A) · 70 of 100: how capable the model is at NF4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.2 s |
| Qwen-Image 2.1 | Very slowtight fit · encoder unloaded after use92Excellent quality (A+) · 92 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.11.9 min | Very slowtight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.6.7 min | Slowtight fit · encoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.63 s |
| FLUX.1 Fill [dev] | Slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow76Great quality (A) · 76 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.73 s | Runs great74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.9.4 s |
| MOSS-Image SFT | Slow80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slow82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.82 s | Runs great80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11 s |
| Qwen-Image | Very slowencoder unloaded after use86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.14.0 min | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.9 min | Slowencoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71 s |
| Stable Diffusion 3.5 Large Turbo | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.14 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.9.4 s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.1.5 s |
| FLUX.2 [dev] | Very slowencoder unloaded after use85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.7.5 min | Very slowencoder unloaded after use91Excellent quality (A+) · 91 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.4.2 min | Runs wellencoder in RAM85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38 s |
| Stable Diffusion 3.5 Large | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.64 s | Runs great64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.8.3 s |
| MiniMax H3 Image | SlowCPU offload93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slowencoder unloaded after use91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.55 s | Runs greatCPU offload91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.9.8 s |
| Iris-3B | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.6 min | Slow72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Runs well72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 s |
| Kolors | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.23 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.3.3 s |
| CogView4 6B | Very slow68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.2.7 min | Slow70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| OmniGen2 | Very slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.2.6 min | Slow66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.1.6 min | Runs great64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.12 s |
| SDXL-Lightning (4-step) | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.6.9 s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.4.1 s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.0.6 s |
| Cosmos3-Super Text2Image (64B) | Very slowCPU offload90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.7.7 min | Very slowCPU offload90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.4.6 min | Runs wellCPU offload90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39 s |
| Stable Diffusion 3.5 Medium | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.59 s | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.35 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.5.0 s |
| Stable Diffusion 3 Medium | Runs well52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.31 s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.2.6 s |
| DreamShaper XL v2 Turbo | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.7.6 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.1 s |
| PixArt-Σ XL 1024 | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.15 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.8.8 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.1.3 s |
| Lumina-Image 2.0 | Slow62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.69 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.5.9 s |
| Kandinsky 3 | Slow48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.89 s | Slow48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.53 s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.7.5 s |
| HunyuanImage 3.0 (80B-A13B) | Very slowtight fit · CPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.11.5 min | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.6.9 min | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| Pony Diffusion V6 XL | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| Stable Cascade | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.49 s | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.2 s |
| Juggernaut XL v9 | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| RealVisXL V5.0 | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| Playground v2.5 | Slow60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.74 s | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.3 s |
| SDXL Turbo | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.0.9 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.0.1 s |
| Animagine XL 4.0 | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| Illustrious XL v0.1 | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| AuraFlow v0.3 | Slow56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.2.1 min | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.82 s | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.11 s |
| Animagine XL 3.1 | Runs well54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well54Fair quality (C) · 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.22 s | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| Hunyuan-DiT v1.2 | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.59 s | Runs well52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.35 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.5.0 s |
| Stable Diffusion XL 1.0 | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.40 s | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.24 s | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| Realistic Vision V5.1 | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| DreamShaper 8 | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| Stable Diffusion 1.5 | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| LCM DreamShaper v7 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.2.5 s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.0.2 s |
| SD Turbo | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.0.8 s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.0.5 s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.0.1 s |
Video models
41 of 43 video models run on the Apple M4 Pro (48 GB). 36.0 GB usable for models.
42 of 43 video models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
40 of 43 video models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | M4 Profit · speed · quality | M4 Maxfit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| Bernini-R A14B (video generation + editing) | Very slow83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.1.8 h | Very slow83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.3 min | Runs well83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.0 min |
| LTX-Video 13B 0.9.8 (distilled) | Runs well66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.8.9 min | Runs well66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.3 min | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.45 s |
| LTX-2.5 (22B, video + audio) | Slowencoder unloaded after use81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.18.8 min | Runs well81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.10.7 min | Runs greatencoder in RAM81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.1.6 min |
| SCAIL-2 14B (character animation) | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.3.0 h | Very slow78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.8 h | Slowtight fit78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.15.1 min |
| HY-OmniWeaving (8.3B, unified video) | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.78.3 min | Very slow74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.46.6 min | Runs well74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.6.7 min |
| FramePack I2V (HunyuanVideo 13B) | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.59.1 min | Slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.0 min |
| Wan 2.2 S2V 14B (speech to video) | Very slow75Great quality (A) · 75 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.3.1 h | Very slow75Great quality (A) · 75 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.1.9 h | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.12.5 min |
| Wan 2.2 I2V A14B | Very slowtight fit79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.5.3 h | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.0 h | Slowencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.28.1 min |
| Wan 2.2 T2V A14B | Very slowtight fit79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.5.3 h | Very slow85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.0 h | Slowencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.28.1 min |
| Wan 2.1 I2V 14B 480P | Very slow73Great quality (A) · 73 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.8 h | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.79.1 min | Runs well72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.11.3 min |
| SkyReels V2 T2V 14B 540P | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.4 h | Very slow72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.1 h | Slowtight fit72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.17.6 min |
| LTX-2.3 | Very slowtight fit · encoder unloaded after use85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.79.3 min | Slowtight fit84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Runs wellencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.0 min |
| HunyuanVideo 1.5 (8.3B) | Very slowencoder unloaded after use82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.7 h | Very slow82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.6 h | Slowtight fit · encoder in RAM82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.23.7 min |
| LTX-2 (19B, video + audio) | Very slowtight fit · encoder unloaded after use80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.62.1 min | Slowtight fit80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.35.2 min | Runs wellencoder in RAM76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.7.0 min |
| MiniMax H3 (33B, video + audio) | Very slowtight fit · encoder unloaded after use83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.3.6 h | Very slowtight fit · encoder unloaded after use89Excellent quality (A+) · 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | SlowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.0 min |
| Kandinsky 6.0 Video Lite (3B, video + audio) | Very slowtight fit68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.50.9 min | Slow68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.30.3 min | Runs wellencoder in RAM68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.5 min |
| Wan 2.2 Animate 14B | Very slowtight fit72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.2 h | Very slowtight fit77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.6 h | Slowtight fit · encoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.33.4 min |
| Wan 2.1 I2V 14B 720P | Very slowtight fit72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.6 h | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.7 h | Slowencoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.35.1 min |
| Wan 2.1 T2V 14B | Very slowtight fit72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.6 h | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.3.7 h | Slowencoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.35.1 min |
| Wan 2.1 VACE 14B | Very slowtight fit72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.7.2 h | Very slowtight fit77Great quality (A) · 77 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.4.1 h | Slowtight fit · encoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.38.6 min |
| Wan 2.2 TI2V 5B | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.1.7 h | Very slow68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.45.7 min | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.8.7 min |
| Cosmos-Predict2 2B Video2World | Slow56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.37.0 min | Slow58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.24.4 min | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.1 min |
| CogVideoX 5B | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.6 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.17.6 min | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.5 min |
| CogVideoX 5B I2V | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.29.6 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.17.6 min | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.5 min |
| Mochi 1 Preview (10B) | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.2.7 h | Very slow60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.1.6 h | Slow60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.13.8 min |
| Kandinsky 6.0 Pro (29B, video + audio) | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.10.5 h | Very slowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.6.2 h | Very slowtight fit · CPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.53.3 min |
| LTX-Video 2B | Runs great49Fair quality (C) · 49 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.3.7 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.2.4 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.21 s |
| SkyReels V2 Diffusion Forcing 1.3B | Very slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.49.3 min | Slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.29.3 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.8 min |
| Wan 2.1 T2V 1.3B | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.4 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.23.4 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.0 min |
| HunyuanVideo 13B | Very slowencoder unloaded after use66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.8.3 h | Very slow66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.7 h | SlowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| HunyuanVideo I2V 13B | Very slowencoder unloaded after use66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.8.3 h | Very slow66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.7 h | SlowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| Open-Sora 2.0 (11B) | Very slowtight fit · encoder unloaded after use70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.8.6 h | Very slowtight fit70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.9 h | Very slowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.54.4 min |
| Cosmos-Predict2 14B Video2World | Very slowtight fit · encoder unloaded after use68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.4.1 h | Very slow68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.2.3 h | SlowCPU offload68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.26.1 min |
| SkyReels V2 T2V 14B 720P | Very slowCPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.7.7 h | Very slowtight fit76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.5 h | SlowCPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| Wan 2.1 VACE 1.3B | Slow53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.38.4 min | Slow55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.4 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.3 min |
| Step-Video T2V (30B) | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.9.9 h | Very slowtight fit · encoder unloaded after use74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.4.8 h | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.50.6 min |
| CogVideoX 2B | Runs well44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.8.9 min | Runs well44Fair quality (C) · 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.5.3 min | Runs great45Fair quality (C) · 45 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.50 s |
| Allegro (2.8B) | Very slowencoder unloaded after use52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.3.4 h | Very slow52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Slowtight fit · encoder in RAM52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.17.6 min |
| AnimateDiff v3 (SD 1.5) | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.3.3 min | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.2.0 min | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| Cosmos 1.0 Diffusion 7B Text2World | Very slowCPU offload60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Very slowencoder unloaded after use60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.56.4 min | Runs welltight fit · CPU offload60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.10.0 min |
| CogVideoX 1.5 5B | Very slowtight fit · CPU offload60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.2 h | Very slow58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.79.1 min | Won't run58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Cosmos3-Super Image2Video (64B) | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| Wan-Dancer 14B (music to dance) | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.8.4 h | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— |
Decision models
69 of 70 decision models run on the Apple M4 Pro (48 GB). 36.0 GB usable for models.
69 of 70 decision models run on the Apple M4 Max (64 GB). 48.0 GB usable for models.
68 of 70 decision models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | M4 Profit · speed · quality | M4 Maxfit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| Jina Reranker v3.5 | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.67 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| Jina Reranker v3 | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.67 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| FRIDA-Decisions | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.50 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.27 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.7.5 ms |
| GLiNER2.5 Decide | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.31 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.6.1 ms |
| Qwen3 Reranker 0.6B | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.67 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| Llama Prompt Guard 2 86M | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms |
| mxbai-rerank-base v2 | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.55 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.30 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.8.3 ms |
| mxbai-rerank-large v2 | Runs well82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.153 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.84 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.17 ms |
| d1 3B | Runs well82Great quality (A) · 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.165 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.97 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.21 ms |
| GLiClass ModernBERT Large v3 | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.26 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| OpenAI Privacy Filter | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.6.8 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.5.4 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.4.2 ms |
| Qwen3 Reranker 4B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.392 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.198 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.39 ms |
| Intern-Decision 4B | Runs well84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.239 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.121 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.28 ms |
| zerank-1 small | Runs well80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.170 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.94 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.19 ms |
| BGE Reranker v2 M3 | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.68 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.36 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.8.8 ms |
| DeBERTa v3 Large Zero-shot v2.0 | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.29 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.5.9 ms |
| DuoGuard 0.5B | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.59 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.31 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.8.2 ms |
| zerank-2 | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.392 ms | Runs well88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.198 ms | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.39 ms |
| GTE Reranker ModernBERT Base | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.21 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms |
| Jina Reranker v2 Base Multilingual | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms |
| ModernBERT Large Zero-shot v2.0 | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.26 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| Shieldstral 1.0 3B | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.376 ms | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.190 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.38 ms |
| Skywork Reward V2 Qwen3 0.6B | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.121 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.14 ms |
| LemonDecider 4B | Runs well80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.225 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.115 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.23 ms |
| Qwen3 Reranker 8B | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.795 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.399 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.75 ms |
| GLiGuard 300M | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.28 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.8 ms |
| Granite Embedding Reranker English R2 | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.21 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms |
| Laya | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.28 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.8 ms |
| Llama Prompt Guard 2 22M | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.8.0 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.4.6 ms |
| Qwen3Guard Gen 0.6B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.102 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.71 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.18 ms |
| GLiClass Base v3 | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.15 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.9.3 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.4.8 ms |
| ProtectAI DeBERTa v3 Prompt Injection v2 | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.24 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.14 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.5.5 ms |
| BGE Reranker v2 Gemma | Runs well78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.246 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.125 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.26 ms |
| Skywork Reward V2 Llama 3.1 8B | Very slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Slow90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.736 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.115 ms |
| DeBERTa v3 Base Zero-shot v2.0 | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.14 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.0 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.8 ms |
| GLiNER2 Base v1 | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.9 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.9 ms |
| HHEM 2.1 Open | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.29 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.9 ms |
| LettuceDetect Large | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.94 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.49 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.11 ms |
| Laya Multilingual | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.22 ms | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 ms | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.4 ms |
| GLiDE 27B | Slowtight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.640 ms | Won't run86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| Skywork Reward V2 Qwen3 8B | Very slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.743 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.116 ms |
| Qwen3Guard Gen 4B | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.580 ms | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.268 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.85 ms |
| GLiNER Multi v2.1 | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.20 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.12 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.2 ms |
| Qwen3Guard Gen 8B | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.539 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.103 ms |
| BART Large MNLI | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.27 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| Llama 3.1 Nemotron Safety Guard 8B v3 | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.528 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.102 ms |
| Granite Guardian 4.1 8B | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.551 ms | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.94 ms |
| ms-marco MiniLM L6 v2 | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.6.6 ms | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.4.2 ms |
| NuExtract 2.0 4B | Slow75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.622 ms | Runs well75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.313 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.97 ms |
| WildGuard 7B | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.951 ms | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.477 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.92 ms |
| Granite Guardian 3.3 8B | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.538 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.103 ms |
| NuExtract 2.0 2B | Runs well69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.368 ms | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.145 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.59 ms |
| ShieldGemma 2B | Runs well58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.256 ms | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.130 ms | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.27 ms |
| ShieldGemma 9B | Slow64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.896 ms | Slow64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.450 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.85 ms |
| Llama Guard 3 1B | Runs well55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.184 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.102 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.32 ms |
| CompassJudger-2 32B | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.26.5 s | Very slow86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.19.7 s | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| ShieldGemma 27B | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.6 s | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.206 ms |
| Llama Guard 3 8B | Slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.528 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.102 ms |
| Nemotron Content Safety Reasoning 4B | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.5.1 s | Very slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.9 s | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.346 ms |
| DynaGuard 8B | Very slow79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.9.7 s | Very slow79Great quality (A) · 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s | Slow78Great quality (A) · 78 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.918 ms |
| gpt-oss-safeguard 20B | Very slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.2.7 s | Very slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.6 s | Slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.812 ms |
| Selene 1 Mini 8B | Very slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.9.6 s | Very slow80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.4.8 s | Slow79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.902 ms |
| CompassJudger-2 7B | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.9.2 s | Very slow78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.6 s | Slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.856 ms |
| Llama Guard 4 12B | Slow62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.788 ms | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.133 ms |
| DynaGuard 1.7B | Slow66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.906 ms | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.165 ms |
| Prometheus 2 7B | Very slow68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.8.7 s | Very slow67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.3.4 s | Slow67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.814 ms |
| Prometheus 2 8x7B | Very slowtight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.9.5 s | Very slow71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.5.7 s | Very slowtight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.1.6 s |
| GLIDER | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.3.6 s | Very slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.316 ms |
| Flow Judge v0.1 | Very slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.3.6 s | Very slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.316 ms |
| gpt-oss-safeguard 120B | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— |
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.