RTX 3090 vs RTX 4090 vs RTX 5090 for local AI
Pick two to four systems or devices to see which runs more models, faster, for the kind of work you do.
Winner: NVIDIA GeForce RTX 5090
For general chat: Its ten best models average a score of 100 vs 99; it runs 18 more LLMs (514 vs 496); it is about 1.3× faster on its top models (against the NVIDIA GeForce RTX 4090, the next best).
 NVIDIA GeForce RTX 3090
Remove
|
 NVIDIA GeForce RTX 4090
Remove
|
 NVIDIA GeForce RTX 5090
Remove
|
|
|---|---|---|---|
| Memory | 24.0 GB VRAM | 24.0 GB VRAM | 32.0 GB VRAM |
| Usable for models | 24.0 GB | 24.0 GB | 32.0 GB |
| Memory bandwidth | 936 GB/s | 1,008 GB/s | 1,792 GB/s |
| FP16 compute | 35.6 TFLOPS | 82.6 TFLOPS | 104.8 TFLOPS |
| Launch price | $1,499 | $1,599 | $1,999 |
| Released | 2020 | 2022 | 2025 |
| LLMs that run | 496 of 622 | 496 of 622 | 514 of 622 |
| LLMs that run well | 466 | 466 | 493 |
| Image models that run | 58 of 61 | 58 of 61 | 60 of 61 |
| Video models that run | 31 of 43 | 31 of 43 | 40 of 43 |
| Decision models that run | 68 of 70 | 68 of 70 | 68 of 70 |
| Top-10 LLM average score | 99 | 99 | 100 |
| Top-10 LLM average speed | 86.1 tok/s | 89.3 tok/s | 114 tok/s |
| Best LLM | Qwen3.8 27B | Qwen3.8 27B | Nanbeige4.3 33B |
Highlighted values are the best of the 3. Model results assume 8K of context.
LLMs
496 of 622 LLMs run on the NVIDIA GeForce RTX 3090. 24.0 GB usable for models.
496 of 622 LLMs run on the NVIDIA GeForce RTX 4090. 24.0 GB usable for models.
514 of 622 LLMs run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | RTX 3090fit · speed · quality | RTX 4090fit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| Qwen3.8 27B | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.36.8 tok/s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| Qwen3.6 27B | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.79.2 tok/s |
| Gemma 4 26B-A4B | Runs welltight fit91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.119 tok/s | Runs welltight fit91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.139 tok/s |
| Qwen3.5 27B | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Muse Glimmer 30B | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.38.6 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.41.6 tok/s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.51.3 tok/s |
| Qwen3.6 35B-A3B | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| MiniCPM-V 4.7 35B-A3B | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Nanbeige4.3 33B | Runs welltight fit90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.34.7 tok/s | Runs welltight fit90Excellent quality (A+) · 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.37.4 tok/s | Runs welltight fit94Excellent quality (A+) · 94 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.46.1 tok/s |
| Qwen3.5 35B-A3B | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| TMax 27B | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.42.0 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Nex N2 Mini (35B-A3B) | Runs welltight fit87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit87Excellent quality (A+) · 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Sage-A1.5 35B-A3B | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Fara 1.5 27B | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.42.0 tok/s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Raon-1 mini Preview (30B-A4B) | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.146 tok/s | Runs great87Excellent quality (A+) · 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s |
| Ornith 1.5 35B-A3B | Runs welltight fit85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Gemma 4 31B | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.36.7 tok/s | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s |
| Apodex 1.1 Mini (35B-A3B) | Runs welltight fit84Great quality (A) · 84 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit84Great quality (A) · 84 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Ornith 1.0 35B-A3B | Runs welltight fit84Great quality (A) · 84 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit84Great quality (A) · 84 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Gemma 4 12B | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.48.9 tok/s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.52.7 tok/s | Runs great83Great quality (A) · 83 of 100: how capable the model is at F16, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s |
| UI-Mate 27B | Runs great81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.42.0 tok/s | Runs great81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| GLM-4.7 Flash (30B-A3B) | Runs welltight fit83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Agents-A1 35B-A3B | Runs welltight fit82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Qwen AgentWorld 35B-A3B | Runs welltight fit82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit82Great quality (A) · 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Laguna XS 2.1 (33B-A3B) | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| DiffusionGemma 26B-A4B IT | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.121 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| Nemotron Cascade 2 30B-A3B | Runs welltight fit76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s | Runs welltight fit76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| Devstral Small 2 (24B) | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| K2 Horizon 7B | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.70.0 tok/s | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| Flint (40B-A2.5B) | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.203 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.208 tok/s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking.222 tok/s |
| KAT-Coder V2.5 Dev (35B-A3B) | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| EXAONE 4.5 33B | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.33.4 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.36.0 tok/s | Runs welltight fit81Great quality (A) · 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s |
| LFM2.5 24B-A2B | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.164 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.192 tok/s |
| Apriel 1.5 15B Thinker | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.39.4 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.42.4 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.75.4 tok/s |
| SenseNova Loop 9B Preview | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.60.5 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.65.2 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.53.4 tok/s |
| Nemotron 3 Nano 30B-A3B | Runs welltight fit73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s | Runs welltight fit73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| Qwen3 30B-A3B Instruct 2507 | Runs welltight fit73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s | Runs welltight fit73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.166 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Nemotron 3.5 Bolt E4B | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.59.9 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.5 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s |
| Qwen3-VL 30B-A3B Thinking | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.166 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Ornith 1.5 9B | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.60.6 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Granite 4.2 30B | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.39.2 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.42.3 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.62.1 tok/s |
| Phi-4 Reasoning Vision 15B | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.41.7 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.74.1 tok/s |
| Hunyuan Hy-Memory 30B-A3B | Runs welltight fit82Great quality (A) · 82 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.150 tok/s | Runs welltight fit82Great quality (A) · 82 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.155 tok/s | Runs great82Great quality (A) · 82 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Qwen3.5 9B | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.60.6 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Gemma 3 27B IT | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.55.7 tok/s |
| Qwen3 30B-A3B | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.166 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Qwen3-VL 30B-A3B Instruct | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.166 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Gemma 4 E4B | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.59.9 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.64.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s |
| Ornith 1.0 9B | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.60.6 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.65.3 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.53.5 tok/s |
| Devstral Small 1.1 (24B) | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Magistral Small 2507 | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Qwen3 32B | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.35.0 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.37.7 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Qwen3 14B | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.75.7 tok/s |
| MedGemma 27B Text IT | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.42.5 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.45.8 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.5 tok/s |
| gpt-oss 20B | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.159 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.164 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Nemotron 3.5 Lightning 30B-A3B | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Granite 4.1 30B | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.42.9 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.62.9 tok/s |
| GLM-4 32B 0414 | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.38.0 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Magistral Small 2506 | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Hermes 4 14B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Mistral Small 3.2 24B | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Qwen3-VL 32B Instruct | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.48.0 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.45.8 tok/s |
| TMax 9B | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.65.4 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.70.4 tok/s | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.57.7 tok/s |
| Seed-OSS 36B Instruct | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.44.2 tok/s | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.60.9 tok/s |
| Terminal RSI 27B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| EXAONE 4.0 32B | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.8 tok/s |
| Olmo 3.1 32B Instruct | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.35.6 tok/s | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.38.4 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| Sarvam 30B | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.183 tok/s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.189 tok/s | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.207 tok/s |
| North Mini Code 1.0 (30B-A3B) | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.150 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.155 tok/s | Runs great77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Hermes 4.3 36B | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.44.2 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.60.9 tok/s |
| Ministral 3 14B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s |
| Phi-4 (14B) | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.42.9 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.76.2 tok/s |
| Mistral Small 3.1 24B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Hemmingway 1.5 | Runs great63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.42.7 tok/s | Runs great63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.46.0 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.56.7 tok/s |
| InternVL3 14B | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.41.7 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.74.1 tok/s |
| ERNIE 4.5 21B-A3B | Runs great63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.136 tok/s | Runs great63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Trinity Mini (26B-A3B) | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.136 tok/s | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Baichuan M2 32B | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.35.1 tok/s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.37.8 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Qwen2.5 32B Instruct | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.35.0 tok/s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.37.7 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Mistral Small 3 24B (2501) | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Qwen2.5 14B Instruct | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.42.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.76.2 tok/s |
| Granite 4.2 8B | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.71.7 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s |
| InternVL3.5 38B | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.41.7 tok/s | Runs welltight fit73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.47.3 tok/s |
| Phi-4 Reasoning Plus (14B) | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s |
| Qwen3 30B-A3B Thinking 2507 | Runs welltight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s | Runs welltight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.166 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Qwen3 Coder 30B-A3B | Runs welltight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s | Runs welltight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.166 tok/s | Runs great75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.188 tok/s |
| Kimi-VL A3B Thinking 2506 | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.112 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.117 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Ling 3.0 Tiny (8B-A1.3B) | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.113 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.145 tok/s |
| Dolphin Mistral 24B Venice Edition | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Qwen2.5 14B Instruct 1M | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Voxtral Small 24B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.62.9 tok/s |
| Gemma 3 12B IT | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.47.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.51.6 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| Nanbeige4.2 3B | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.64.6 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.69.6 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| MiMo V2.6 Distill Qwen 9B | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.66.9 tok/s | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| Gemma 2 27B IT | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.42.2 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.45.5 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.1 tok/s |
| Phi-4 Reasoning (14B) | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.39.9 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.43.0 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s |
| Qwen3-Omni 30B-A3B Instruct | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.183 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.189 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s |
| QwQ 32B | Runs welltight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.35.1 tok/s | Runs welltight fit74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.37.8 tok/s | Runs welltight fit77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Magistral Small 1.2 (24B) | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.63.6 tok/s |
| Dolphin 3.0 R1 Mistral 24B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great59Good quality (B) · 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Solar Pro Preview 22B | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.42.9 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.46.2 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.50.6 tok/s |
| ERNIE 4.5 21B-A3B Thinking | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.136 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Maple Preview (20B-A1B) | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.183 tok/s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.187 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.195 tok/s |
| Mellum2.1 Thinking (12B-A2.5B) | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.129 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| Qwen2.5-VL 32B Instruct | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.44.4 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.47.8 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s |
| Olmo 3.1 32B Think | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.35.6 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.38.4 tok/s | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| OpenReasoning Nemotron 32B | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.35.1 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.37.8 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Skywork OR1 32B | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.35.1 tok/s | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.37.8 tok/s | Runs welltight fit75Great quality (A) · 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Axion 1.0 35B | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Granite 4.0 H Small (32B-A9B) | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.89.3 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.94.0 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.118 tok/s |
| Nemotron Labs Dory 1.0 4B | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.72.9 tok/s | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s |
| Groot 19B A3B 261023 | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.136 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Falcon-H1 34B Instruct | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.34.2 tok/s | Runs welltight fit60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.36.8 tok/s | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.45.4 tok/s |
| InternVL3 38B | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.41.7 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.47.3 tok/s |
| EXAONE Deep 32B | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.35.9 tok/s | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.38.7 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.8 tok/s |
| Qwen3-VL 8B Instruct | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.71.8 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.58.8 tok/s |
| GLM-Z1 32B (0414) | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs welltight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.38.0 tok/s | Runs great71Great quality (A) · 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| Ministral 3 14B Reasoning | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.80.3 tok/s |
| Command R (08-2024) 32B | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.35.6 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.38.3 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s |
| AceReason Nemotron 14B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| OpenReasoning Nemotron 14B | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Qwen2.5 Coder 14B Instruct | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Spark-X2.6 4B | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.65.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.70.6 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.126 tok/s |
| Gemma 4 E2B | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.117 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.126 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.224 tok/s |
| GLM-4.1V 9B Thinking | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.56.8 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.61.2 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.50.2 tok/s |
| Apriel Nemotron 15B Thinker | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.39.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.74.8 tok/s |
| InternVL3.5 8B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.60.5 tok/s |
| LFM2 24B-A2B | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.169 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.174 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.195 tok/s |
| NorthCUA Mini | Runs welltight fit56Good quality (B) · 56 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s | Runs welltight fit56Good quality (B) · 56 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s | Runs welltight fit58Good quality (B) · 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.179 tok/s |
| Reka Flash 3.1 (21B) | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.38.1 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s | Runs great66Good quality (B) · 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| Granite 4.1 8B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.71.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s |
| Qwen3.5 4B | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.57.8 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.111 tok/s |
| DeepSeek R1 Distill Qwen 32B | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.35.0 tok/s | Runs welltight fit68Good quality (B) · 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.37.7 tok/s | Runs welltight fit71Great quality (A) · 71 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| OLMo 2 32B Instruct | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.35.6 tok/s | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.38.4 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| Devstral Small 2505 | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.40.3 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.43.4 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.64.8 tok/s |
| Aya Expanse 32B | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.35.6 tok/s | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.38.3 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s |
| MiniCPM-V 4.5 8B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.67.2 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.72.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.59.3 tok/s |
| DeepCoder 14B Preview | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Aya Vision 32B | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.34.7 tok/s | Runs welltight fit57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.37.3 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| Mistral Small 22B (2409) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.42.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.45.9 tok/s | Runs great53Fair quality (C) · 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| LFM2.5 8B-A1B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.101 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.137 tok/s |
| Ling Mini 2.0 (16B-A1.4B) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.166 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.198 tok/s |
| Mistral Nemo 12B Instruct | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.47.9 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.51.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| Qwen3 8B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| SenseNova 6.8 Code2Video | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.42.0 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.45.2 tok/s | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| UI-Mate 9B | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.66.9 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| UI-Venus 2 9B | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.66.9 tok/s | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| OpenHands LM 32B | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.35.1 tok/s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.37.8 tok/s | Runs welltight fit69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| Nemotron Nano 9B v2 | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.65.8 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.70.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.58.1 tok/s |
| DeepSeek R1 Distill Qwen 14B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| JanusCoder 14B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.39.6 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s |
| Agents-A1 4B | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.59.4 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| TMax 4B | Runs great61Good quality (B) · 61 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.64.0 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.69.0 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s |
| Kimi-VL A3B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.112 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.117 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Pixtral 12B (2409) | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.47.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.50.8 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.90.3 tok/s |
| MiMo-VL 7B RL | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.70.4 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.62.1 tok/s |
| Kimi Linear 48B-A3B | Runs welltight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.212 tok/s | Runs welltight fit51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.217 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.233 tok/s |
| InternLM2.5 20B Chat | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great49Fair quality (C) · 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| Mage-VL 4.7B | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.56.9 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.61.2 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| Ministral 3 8B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.65.6 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.70.6 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.57.9 tok/s |
| Codestral 22B | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.42.7 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.45.9 tok/s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| Falcon 3 10B Instruct | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.56.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.61.1 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.50.1 tok/s |
| Fara 1.5 4B | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.59.4 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Qwen2.5 Coder 32B Instruct | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.35.0 tok/s | Runs welltight fit63Good quality (B) · 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.37.7 tok/s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| QwQ 32B Preview | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.35.1 tok/s | Runs welltight fit62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.37.8 tok/s | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.46.6 tok/s |
| MiMo 7B RL | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.74.7 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.80.4 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.65.9 tok/s |
| Qwen3-VL 4B Instruct | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.60.7 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.65.4 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s |
| Gemma 3n E4B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.67.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.72.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.129 tok/s |
| Moonlight 16B-A3B Instruct | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.112 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.117 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s |
| Phi-3 Medium (14B) | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.45.1 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.80.2 tok/s |
| Phi-3 Medium 128K Instruct | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.45.1 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.80.2 tok/s |
| InternVL3 8B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.73.7 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.79.3 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s |
| LLM4Decompile 22B V2 | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.42.7 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.45.9 tok/s | Runs great57Good quality (B) · 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.68.6 tok/s |
| Llama 3.1 Nemotron Nano 8B v1 | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Gemma 2 9B IT | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.68.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.55.9 tok/s |
| Phi-3.5 MoE (42B-A6.6B) | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.136 tok/s | Runs welltight fit53Fair quality (C) · 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.157 tok/s |
| GLM-4 9B 0414 | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.67.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| NorthCUA Nano | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.66.9 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s |
| Qwen3 4B Instruct 2507 | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.67.1 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| Granite 4.0 H Tiny (7B-A1B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.158 tok/s |
| OLMo 2 13B Instruct | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.42.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.45.9 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.81.6 tok/s |
| MiniCPM4 8B | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| MiniCPM-o 2.6 8B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.67.5 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.59.5 tok/s |
| DeepSeek R1 0528 Qwen3 8B | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71.4 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.76.9 tok/s | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.63.0 tok/s |
| Yi 1.5 34B Chat | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.33.4 tok/s | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.36.0 tok/s | Runs great47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.64.0 tok/s |
| Qwen2.5-VL 7B Instruct | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.70.6 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.76.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| Mage-VL-Cowork 4.7B | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.56.9 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.61.2 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| MiniCPM5 2B | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.205 tok/s |
| Qwen2.5-Omni 7B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.54.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| Hunyuan 7B Instruct | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.78.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.84.0 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.68.8 tok/s |
| InternLM2 Chat 20B | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| GLM-4 9B Chat 1M | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.61.7 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.66.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.4 tok/s |
| InternLM3 8B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.66.5 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.58.6 tok/s |
| InternVL2.5 8B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.72.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.78.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s |
| Foundation-Sec 1.1 8B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| DeepSeek Coder V2 Lite (16B) | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.126 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.131 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| Qwen2.5 7B Instruct | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.76.8 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.82.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Phi-4 Multimodal Instruct | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.52.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.92.7 tok/s |
| GLM-4 9B Chat (2024) | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.67.0 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| Qwen3 4B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.67.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| Olmo 3 7B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.80.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.86.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| DeepSeek V2 Lite Chat | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.126 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.131 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| Granite 4.2 3B | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.73.7 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.79.3 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.141 tok/s |
| Llama 3.2 11B Vision Instruct | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.59.0 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s |
| GLM-Z1 9B (0414) | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.0 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| GLiDE 9B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.73.7 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.79.3 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s |
| Hermes 3 (Llama 3.1 8B) | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Llama 3.1 8B Instruct | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Apertus v1.5 8B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.65.7 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.58.0 tok/s |
| Qwen2.5 7B Instruct 1M | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.76.8 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.82.7 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Falcon-H1 7B Instruct | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.77.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.83.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| Seed-Coder 8B Reasoning | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.70.9 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.62.6 tok/s |
| LFM2 8B-A1B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.101 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.137 tok/s |
| Granite 3.2 8B Instruct | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.77.1 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Granite 3.3 8B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.77.1 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| MiniCPM-V 2.6 8B | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.77.8 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.63.7 tok/s |
| Nemotron 3 Nano 4B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.67.9 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.130 tok/s |
| Qwen1.5 32B Chat | Runs welltight fit41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s | Runs welltight fit41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.38.1 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Ministral 8B (2410) | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.72.9 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.78.5 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.4 tok/s |
| DeepHermes 3 Llama 3 8B Preview | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Dolphin 3.0 Llama 3.1 8B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Tülu 3 8B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| LLaVA 1.6 34B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.42.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.46.0 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.3 tok/s |
| CogVLM2 Llama 3 Chat 19B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.57.4 tok/s |
| Nous Hermes 2 SOLAR 10.7B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.54.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| SOLAR 10.7B Instruct | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.54.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.58.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.48.1 tok/s |
| EXAONE 3.5 7.8B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.74.8 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.80.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Seed-Coder 8B Instruct | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.70.9 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.62.6 tok/s |
| Llama 3.1 Nemotron Nano 4B v1.1 | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.59.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Granite 3.1 8B Instruct | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.77.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Falcon 3 7B Instruct | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.84.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.69.2 tok/s |
| EXAONE Deep 7.8B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.74.8 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.80.6 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Qwen3 4B Thinking 2507 | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.67.1 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.72.2 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s |
| InternVL2 8B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.72.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.78.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s |
| Dolphin 2.9.4 Llama 3.1 8B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Hermes 2 Pro Llama 3 8B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| OpenChat 3.6 8B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Aya Vision 8B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.67.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.73.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.59.8 tok/s |
| Mistral-NeMo-Minitron 8B Instruct | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.69.5 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.74.9 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.61.4 tok/s |
| Gemma 3 4B IT | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.62.7 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.67.5 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s |
| Qwen2-VL 7B Instruct | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.70.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.76.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| Nous Hermes 2 Mixtral 8x7B DPO | Runs welltight fit36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs welltight fit36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Runs welltight fit49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| LFM3 1.7B | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.156 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.298 tok/s |
| Phi-3.5 Mini Instruct | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.70.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.76.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Qwen1.5 14B Chat | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.56.3 tok/s | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.60.6 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.79.0 tok/s |
| Yi 34B Chat | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs welltight fit38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.36.0 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.0 tok/s |
| Phi-3 Small 8K Instruct | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.79.1 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.85.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.69.8 tok/s |
| Command R7B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| MedGemma 4B IT | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.62.7 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.67.5 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s |
| Falcon-H1R 7B | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.77.1 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.83.0 tok/s | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| Phi-4 Mini (3.8B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Phi-3.5 Vision Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.0 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| Hunyuan 4B Instruct | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.68.8 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.122 tok/s |
| Olmo 3 7B Think | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.80.1 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.86.3 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| Llama 3 8B Instruct | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Ministral 3 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.75.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Jamba 1.5 Mini | Runs welltight fit35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs welltight fit35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.111 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Mixtral 8x7B Instruct | Runs welltight fit35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.100 tok/s | Runs welltight fit35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.106 tok/s | Runs welltight fit47Fair quality (C) · 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s |
| Qwen2 7B Instruct | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.76.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.82.7 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Qwen2.5 Coder 7B Instruct | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.76.8 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.82.7 tok/s | Runs great55Good quality (B) · 55 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Granite 4.1 3B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.79.3 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.85.4 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.152 tok/s |
| Qianfan-OCR 4B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.56.9 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.61.2 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s |
| Foundation-Sec 8B Reasoning | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| OpenReasoning Nemotron 7B | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.76.8 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.82.7 tok/s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| AFM 4.5B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.58.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.62.8 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.112 tok/s |
| Command R7B Arabic | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Nemotron 3.5 Bolt 74B-A9B | Runs welloffloads to RAM64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.41.4 tok/s | Runs welloffloads to RAM64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.41.9 tok/s | Runs welloffloads to RAM77Great quality (A) · 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.45.6 tok/s |
| Apertus 8B Instruct | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.78.2 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.1 tok/s |
| Aya Expanse 8B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Chandra OCR 2 | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.50.9 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.54.8 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.97.4 tok/s |
| Granite 3.0 8B Instruct | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.77.1 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.63.2 tok/s |
| Kolibri 1 (78B-A3.5B) | Runs welloffloads to RAM63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.83.3 tok/s | Runs welloffloads to RAM63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.83.9 tok/s | Runs welloffloads to RAM76Great quality (A) · 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.86.5 tok/s |
| InternLM2.5 7B Chat | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.81.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s |
| Gemma 3n E2B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.145 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.258 tok/s |
| DeepSeek R1 Distill Llama 8B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Phi-3 Mini 128K Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.70.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.76.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Fara 7B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.70.6 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.76.0 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s |
| Phi-3 Vision 128K Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.65.0 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.70.0 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s |
| Dolphin 2.9 Llama 3 8B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| LFM3-VL 2B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.140 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.151 tok/s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.269 tok/s |
| Voxtral Mini 3B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.57.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.62.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.110 tok/s |
| InternLM2 Math Plus 20B | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.43.2 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.56.4 tok/s |
| StarCoder2 15B Instruct | Runs great39Basic quality (D) · 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.70.2 tok/s |
| Jamba Reasoning 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.84.2 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.90.7 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| LFM2.5-VL 3B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.93.0 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.165 tok/s |
| S5 nano | Runs great37Basic quality (D) · 37 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.57.8 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.62.3 tok/s | Runs great37Basic quality (D) · 37 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.111 tok/s |
| Orca 2 13B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| StableLM 2 12B Chat | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.48.2 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.51.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.92.2 tok/s |
| OLMo 2 7B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.80.1 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.86.3 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.7 tok/s |
| WeVisOne-VL 4B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.60.1 tok/s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s |
| CodeGeeX4 All 9B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.62.2 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.67.0 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.54.9 tok/s |
| LightOnOCR-3 4B | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.59.4 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.63.9 tok/s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.114 tok/s |
| Yi 1.5 9B Chat | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| EXAONE 3.0 7.8B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.74.8 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.80.6 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Mistral 7B Instruct v0.3 | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.80.7 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.40.0 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.2 tok/s |
| North Micro Vision | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.117 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.208 tok/s |
| SmolLM3 3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.87.5 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.94.3 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.168 tok/s |
| Zamba2 7B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.79.5 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.85.6 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.70.1 tok/s |
| RecurrentGemma 9B IT | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.60.7 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.65.4 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| MiniCPM3 4B | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.127 tok/s |
| Granite 4.0 Micro (3B) | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.79.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.85.4 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.152 tok/s |
| LFM2.5 2.6B | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.99.8 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.108 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.191 tok/s |
| Phi-3 Mini 4K Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.70.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.76.0 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.135 tok/s |
| Baichuan 2 13B Chat | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.45.3 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.80.6 tok/s |
| DeepSeek MoE 16B Chat | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.121 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.159 tok/s |
| Llama 2 13B Chat | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| OLMoE 1B-7B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.113 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.145 tok/s |
| Gemma 1.1 7B IT | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.68.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| DeepSeek Coder 33B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.44.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.0 tok/s | Runs great41Fair quality (C) · 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.66.0 tok/s |
| Qwen3-Next 80B-A3B Thinking | Slowheavy RAM offload72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.58.6 tok/s | Slowheavy RAM offload72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.58.9 tok/s | Runs welloffloads to RAM72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.88.4 tok/s |
| Jamba2 3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.89.0 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.95.8 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s |
| Hermes 2 Pro Mistral 7B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Starling LM 7B Beta | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Falcon3 Mamba 7B Instruct | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.80.5 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.86.6 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.71.0 tok/s |
| Aya 23 8B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.72.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.78.4 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s |
| Yi 1.5 6B Chat | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.47.9 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.85.2 tok/s |
| Qwen2.5-Omni 3B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.48.7 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.93.2 tok/s |
| Granite Code 20B Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.39.7 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.42.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.55.8 tok/s |
| OpenChat 3.5 (0106) | Runs great35Basic quality (D) · 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Phi-4 Mini Reasoning (3.8B) | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.70.2 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.134 tok/s |
| Qwen1.5 MoE A2.7B Chat | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.119 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.124 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| Qwen3 Next 80B-A3B Instruct | Slowheavy RAM offload70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.58.6 tok/s | Slowheavy RAM offload70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.58.9 tok/s | Runs welloffloads to RAM70Great quality (A) · 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.88.4 tok/s |
| Gemma 7B IT | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.68.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| DeepSeek R1 Distill Qwen 7B | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.76.8 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.82.7 tok/s | Runs great46Fair quality (C) · 46 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| DeepSeek Prover V2 7B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.84.6 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.42.0 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.74.7 tok/s |
| Mistral 7B Instruct v0.2 | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Nous Hermes 2 Mistral 7B DPO | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| OpenChat 3.5 (1210) | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Qwen1.5 7B Chat | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.75.8 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.81.6 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.66.9 tok/s |
| Granite Code 34B Instruct | Runs great33Basic quality (D) · 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.44.1 tok/s | Runs welltight fit38Basic quality (D) · 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.36.7 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.53.9 tok/s |
| Granite 4.0 H Micro | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.84.5 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.91.0 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.162 tok/s |
| LLaVA 1.5 13B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.43.8 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.47.2 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.83.9 tok/s |
| WizardLM 13B v1.2 | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Llama 3.2 3B Instruct | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.84.0 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.90.4 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| OpenHermes 2.5 Mistral 7B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Starling LM 7B Alpha | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Qwen2.5-VL 3B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.71.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.77.4 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.138 tok/s |
| Yi Coder 9B Chat | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| Raon 1 nano | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.4 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.82.2 tok/s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.146 tok/s |
| Code Llama 34B Instruct | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.0 tok/s | Runs welltight fit36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.36.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.65.2 tok/s |
| CodeGemma 7B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.68.5 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.73.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.60.4 tok/s |
| Falcon-H1 3B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.85.6 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.92.2 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.164 tok/s |
| StarCoder2 15B (base) | Runs great31Basic quality (D) · 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.49.9 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.39.5 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.70.2 tok/s |
| Qwen3.5 2B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.119 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.128 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.227 tok/s |
| Vicuna 13B v1.5 | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.44.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.48.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Mistral 7B Instruct v0.1 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| LLM4Decompile 9B V2 | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.66.2 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s | Runs great38Basic quality (D) · 38 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.58.4 tok/s |
| TMax 2B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.143 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.275 tok/s |
| Unlimited OCR 2 (3B-A0.6B) | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.151 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.155 tok/s | Runs great44Fair quality (C) · 44 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s |
| Code Llama 13B Instruct | Runs great29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.61.2 tok/s | Runs great29Basic quality (D) · 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.65.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.0 tok/s |
| Qwen2.5 3B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.87.2 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.93.9 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s |
| Hermes 3 Llama 3.2 3B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.84.0 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.90.4 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s |
| Llama 3.3 Nemotron Super 49B v1.5 | Runs welloffloads to RAM56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.29.5 tok/s | Runs welloffloads to RAM56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.31.2 tok/s | Runs welltight fit67Good quality (B) · 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.57.0 tok/s |
| LLaVA 1.6 Mistral 7B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.77.3 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.83.2 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.68.2 tok/s |
| DeepSeek Coder 6.7B Instruct | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.86.8 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.43.1 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Qwen3-VL 2B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.127 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.136 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.242 tok/s |
| MiniCPM-V 4.7 1B | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.174 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.187 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.333 tok/s |
| Neural Chat 7B v3.3 | Runs great28Basic quality (D) · 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Falcon Mamba 7B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.80.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.86.6 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.71.0 tok/s |
| Yi 6B Chat | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.44.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.47.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.85.2 tok/s |
| Falcon 3 3B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.83.5 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.89.9 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.160 tok/s |
| LLM4Decompile 6.7B V2 | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.86.8 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.43.1 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| LFM2 2.6B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.105 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.113 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.201 tok/s |
| Zephyr 7B Beta | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Unlimited OCR (3B-A0.6B) | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.151 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.155 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.181 tok/s |
| Qwen3 Coder Next (80B-A3B) | Slowheavy RAM offload72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.59.7 tok/s | Slowheavy RAM offload72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.60.0 tok/s | Runs welloffloads to RAM72Great quality (A) · 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.91.8 tok/s |
| SQLCoder 7B v2 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.86.8 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.43.1 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| StarCoder 15B | Runs great25Basic quality (D) · 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.50.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.39.8 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s |
| Qwen2.5-Math 7B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.76.8 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.82.7 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.67.7 tok/s |
| Granite 3.2 2B Instruct | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Granite 3.3 2B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Orca 2 7B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.86.8 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.43.1 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| Granite Code 8B Instruct (128K) | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.72.7 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.78.2 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.64.1 tok/s |
| EXAONE 3.5 2.4B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.112 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.214 tok/s |
| Mathstral 7B | Runs great34Basic quality (D) · 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.80.7 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.40.0 tok/s | Runs great34Basic quality (D) · 34 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.71.2 tok/s |
| Baichuan 2 7B Chat | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.77.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.83.9 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.68.7 tok/s |
| Codestral Mamba 7B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.80.2 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.86.4 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.70.8 tok/s |
| Nemotron Mini 4B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.64.3 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.69.3 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.123 tok/s |
| MiniCPM5 1B | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.250 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.269 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.478 tok/s |
| InternLM2 Math Plus 7B | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.75.6 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.81.4 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.66.7 tok/s |
| Falcon 40B Instruct | Runs great11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.45.1 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.48.6 tok/s | Runs great13Basic quality (D) · 13 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.68.0 tok/s |
| Qwen3 1.7B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.133 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.143 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.254 tok/s |
| Granite 3.1 2B Instruct | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.115 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s |
| Gemma 2 2B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.103 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.111 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.198 tok/s |
| Granite Vision 3.2 2B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.90.5 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.97.4 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.173 tok/s |
| Llama 2 7B Chat | Runs great20Basic quality (D) · 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.86.8 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.43.1 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| WeVisOne VL 2B | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.110 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.119 tok/s | Runs great27Basic quality (D) · 27 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.212 tok/s |
| Qwen2.5 Coder 3B Instruct | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.87.2 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.93.9 tok/s | Runs great36Basic quality (D) · 36 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.167 tok/s |
| Granite 3.0 2B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.102 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.110 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s |
| DeepSeek LLM 7B Chat | Runs great18Basic quality (D) · 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.84.6 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.42.0 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.74.7 tok/s |
| Code Llama 7B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.86.8 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.43.1 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| LFM2.5 1.2B Instruct | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.230 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.248 tok/s | Runs great32Basic quality (D) · 32 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.441 tok/s |
| SmolVLM2 2.2B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.129 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.229 tok/s |
| TeleOCR | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.190 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.204 tok/s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.363 tok/s |
| MiniCPM-OCR 1.0 1.3B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.207 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.223 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.397 tok/s |
| LLaVA 1.5 7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.82.8 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.41.1 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.73.1 tok/s |
| EXAONE 4.0 1.2B | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.211 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.227 tok/s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.403 tok/s |
| PaddleOCR-VL 1.6 0.9B | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.281 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.538 tok/s |
| Granite 3.1 3B-A800M Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.136 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.140 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.169 tok/s |
| Ling Flash 2.0 (100B-A6B) | Slowheavy RAM offload53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.38.0 tok/s | Slowheavy RAM offload53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs welloffloads to RAM53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| SmolVLM 2.2B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.120 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.129 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.229 tok/s |
| Granite 4.0 H 1B | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.185 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.199 tok/s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.353 tok/s |
| Zamba2 2.7B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.101 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.109 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.194 tok/s |
| Qwen2-VL 2B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.122 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.131 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.234 tok/s |
| Vicuna 7B v1.5 | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.86.8 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.43.1 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.76.6 tok/s |
| WizardMath 7B v1.1 | Runs great26Basic quality (D) · 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.80.8 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.40.1 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.71.3 tok/s |
| Falcon-H1 1.5B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.174 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.187 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.333 tok/s |
| Granite 3.0 3B-A800M Instruct | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.136 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.140 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.169 tok/s |
| Hunyuan A13B (80B-A13B) | Slowheavy RAM offload62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.15.4 tok/s | Slowheavy RAM offload62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.15.5 tok/s | Runs welloffloads to RAM62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.25.1 tok/s |
| OLMo 7B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.84.9 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.42.1 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.74.9 tok/s |
| LFM2 1.2B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.230 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.248 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.441 tok/s |
| Qwen2 57B-A14B Instruct | Runs welloffloads to RAM35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.63.5 tok/s | Runs welloffloads to RAM35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.65.6 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.122 tok/s |
| Qwen2.5 1.5B Instruct | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.175 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.189 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| LFM2-VL 1.6B | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.171 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.327 tok/s |
| InternLM 7B | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.79.9 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.86.1 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.70.5 tok/s |
| StableLM Zephyr 3B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.96.3 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.104 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| Hunyuan 1.8B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.151 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.162 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.288 tok/s |
| SmolLM2 1.7B Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.158 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| StarCoder2 7B | Runs great24Basic quality (D) · 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.81.6 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.40.5 tok/s | Runs great24Basic quality (D) · 24 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.72.0 tok/s |
| Qwen3.5 0.8B | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.310 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.334 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.593 tok/s |
| Granite Code 3B Instruct (128K) | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.77.5 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.83.4 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.148 tok/s |
| Phi-2 2.7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.97.0 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.104 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.186 tok/s |
| Solar Open 100B | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.19.7 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.19.8 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.29.9 tok/s |
| Llama 3.2 1B Instruct | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.217 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.234 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.416 tok/s |
| InternLM2.5 1.8B Chat | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.143 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.273 tok/s |
| Sarvam 105B | Slowheavy RAM offload52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.23.9 tok/s | Slowheavy RAM offload52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.24.0 tok/s | Slowheavy RAM offload52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.36.1 tok/s |
| Stable Code Instruct 3B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.96.3 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.104 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.184 tok/s |
| Ring Flash 2.0 (100B-A6B) | Slowheavy RAM offload54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.38.0 tok/s | Slowheavy RAM offload54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.38.2 tok/s | Runs welloffloads to RAM54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.57.3 tok/s |
| GLiDE 2B | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.143 tok/s | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.154 tok/s | Runs great17Basic quality (D) · 17 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.275 tok/s |
| Falcon 3 1B Instruct | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.161 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.174 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.309 tok/s |
| Gemma 3 1B | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.270 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.290 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.516 tok/s |
| Spark X2.6 1.7B | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.158 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| Qwen2 1.5B Instruct | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.175 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.189 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| Gemma 2B IT | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.206 tok/s |
| DeepScaleR 1.5B Preview | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.151 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.163 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.290 tok/s |
| Falcon 7B Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.81.0 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.40.2 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.71.5 tok/s |
| SmolLM 1.7B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.158 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.302 tok/s |
| Qwen2.5 Coder 1.5B Instruct | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.175 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.189 tok/s | Runs great26Basic quality (D) · 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| Command R 35B (v01) | Runs welloffloads to RAM37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.16.1 tok/s | Runs welloffloads to RAM37Basic quality (D) · 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.16.4 tok/s | Runs welltight fit45Fair quality (C) · 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.81.3 tok/s |
| OLMo 2 1B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.182 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.349 tok/s |
| LFM2 700M | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.364 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.392 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.697 tok/s |
| Hermes 4 70B | Slowheavy RAM offload62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Slowheavy RAM offload62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Llama 3.3 70B Instruct | Slowheavy RAM offload62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.3.1 tok/s | Slowheavy RAM offload62Good quality (B) · 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| StarCoder2 3B | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.89.0 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.95.8 tok/s | Runs great18Basic quality (D) · 18 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.170 tok/s |
| DeepSeek R1 Distill Llama 70B | Slowheavy RAM offload60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Slowheavy RAM offload60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Tülu 3 70B | Slowheavy RAM offload60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Slowheavy RAM offload60Good quality (B) · 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Qwen3 0.6B | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.359 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.688 tok/s |
| DeepSeek R1 Distill Qwen 1.5B | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.151 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.163 tok/s | Runs great22Basic quality (D) · 22 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.290 tok/s |
| Aya 23 35B | Runs welloffloads to RAM35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.16.1 tok/s | Runs welloffloads to RAM35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.16.4 tok/s | Runs welltight fit42Fair quality (C) · 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.81.3 tok/s |
| StableLM 2 1.6B Chat | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.164 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.177 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.315 tok/s |
| StableLM 2 Zephyr 1.6B | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.164 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.177 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.315 tok/s |
| d1 2B | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.140 tok/s | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.151 tok/s | Runs great21Basic quality (D) · 21 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.269 tok/s |
| Llama 3.1 70B Instruct | Slowheavy RAM offload58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.3.1 tok/s | Slowheavy RAM offload58Good quality (B) · 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| Falcon-H1 0.5B Instruct | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.518 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.558 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.992 tok/s |
| Hermes 3 (Llama 3.1 70B) | Slowheavy RAM offload57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Slowheavy RAM offload57Good quality (B) · 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| Qwen2.5-Math 1.5B Instruct | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.175 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.189 tok/s | Runs great20Basic quality (D) · 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.335 tok/s |
| LFM2.5 350M | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.770 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.829 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.1,475 tok/s |
| Yi Coder 1.5B Chat | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.182 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.196 tok/s | Runs great16Basic quality (D) · 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.349 tok/s |
| GLiDE 0.8B | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.359 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.387 tok/s | Runs great11Basic quality (D) · 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.688 tok/s |
| Llama 3.1 Nemotron 70B Instruct | Slowheavy RAM offload55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Slowheavy RAM offload55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| LFM2-VL 450M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.599 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.645 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,147 tok/s |
| Apertus v1.5 70B | Slowheavy RAM offload55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.0 tok/s | Slowheavy RAM offload55Good quality (B) · 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.0 tok/s | Runs welloffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.19.9 tok/s |
| CodeGemma 2B | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.107 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.116 tok/s | Runs great14Basic quality (D) · 14 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.206 tok/s |
| LFM2 350M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.770 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.829 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,475 tok/s |
| Granite 4.0 H 350M | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.793 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.854 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,518 tok/s |
| SmolVLM 500M Instruct | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.529 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.569 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.1,012 tok/s |
| Qwen2.5 0.5B Instruct | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.550 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.592 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| Qwen2.5 72B Instruct | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.4.9 tok/s | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.5.0 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| Qwen2.5-VL 72B Instruct | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.4.8 tok/s | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.4.8 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| Hunyuan 0.5B Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.499 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.538 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.956 tok/s |
| InternVL3 78B | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.4.1 tok/s | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.4.1 tok/s | Runs welloffloads to RAM51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.10.2 tok/s |
| LFM2.5 230M | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.1,172 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.1,262 tok/s | Runs great10Basic quality (D) · 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.2,244 tok/s |
| TinyLlama 1.1B Chat | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.245 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.264 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.469 tok/s |
| SmolLM2 360M Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.749 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.806 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| DeepSeek Coder 1.3B Instruct | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.200 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.215 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.382 tok/s |
| LLM4Decompile 1.3B V2 | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.200 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking.215 tok/s | Runs great12Basic quality (D) · 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.382 tok/s |
| Apertus 70B Instruct | Slowheavy RAM offload50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.3.1 tok/s | Slowheavy RAM offload50Fair quality (C) · 50 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM42Fair quality (C) · 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.25.2 tok/s |
| QVQ 72B Preview | Slowheavy RAM offload47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.4.8 tok/s | Slowheavy RAM offload47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.4.8 tok/s | Runs welloffloads to RAM47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| Qwen2 0.5B Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.550 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.592 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| ERNIE 4.5 0.3B | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.749 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.806 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| Llama 3 70B Instruct | Slowheavy RAM offload48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Slowheavy RAM offload48Fair quality (C) · 48 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.3.2 tok/s | Runs welloffloads to RAM40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.25.4 tok/s |
| SmolVLM 256M Instruct | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.1,037 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.1,117 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,985 tok/s |
| Gemma 3 270M | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.998 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,075 tok/s | Runs great6Basic quality (D) · 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,911 tok/s |
| surya ocr 2.5 | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.391 tok/s | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.421 tok/s | Runs great15Basic quality (D) · 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.748 tok/s |
| SmolLM 360M Instruct | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.749 tok/s | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.806 tok/s | Runs great5Basic quality (D) · 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.1,434 tok/s |
| Kimi Dev 72B | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.4.9 tok/s | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.5.0 tok/s | Runs welloffloads to RAM56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| Qwen2 72B Instruct | Slowheavy RAM offload44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.4.9 tok/s | Slowheavy RAM offload44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.5.0 tok/s | Runs welloffloads to RAM44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.18.1 tok/s |
| Qwen2-VL 72B Instruct | Slowheavy RAM offload44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.4.8 tok/s | Slowheavy RAM offload44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.4.8 tok/s | Runs welloffloads to RAM44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.16.5 tok/s |
| SmolLM2 135M Instruct | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.2,074 tok/s | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.2,233 tok/s | Runs great4Basic quality (D) · 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3,970 tok/s |
| Llama 3.2 90B Vision Instruct | Very slowheavy RAM offload46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3.0 tok/s | Slowheavy RAM offload46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3.0 tok/s | Slowoffloads to RAM46Fair quality (C) · 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.5.4 tok/s |
| Qwen2.5-Math 72B Instruct | Slowoffloads to RAM40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.5.5 tok/s | Slowoffloads to RAM40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.5.6 tok/s | Runs welloffloads to RAM40Fair quality (C) · 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.31.9 tok/s |
| DeepSeek LLM 67B Chat | Slowoffloads to RAM28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.6.9 tok/s | Slowoffloads to RAM28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.7.0 tok/s | Runs welltight fit28Basic quality (D) · 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.53.6 tok/s |
| Falcon-H1 Tiny 90M Instruct | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.2,995 tok/s | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3,226 tok/s | Runs great3Basic quality (D) · 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.5,734 tok/s |
| SmolLM 135M Instruct | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.2,074 tok/s | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.2,233 tok/s | Runs great2Basic quality (D) · 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.3,970 tok/s |
| Llama 2 70B Chat | Slowoffloads to RAM25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Slowoffloads to RAM25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Runs welltight fit25Basic quality (D) · 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s |
| Qwen2.5 Coder 0.5B Instruct | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.550 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 46/100 (quality, fit, speed and memory use) sets the ranking.592 tok/s | Runs great8Basic quality (D) · 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking.1,053 tok/s |
| Code Llama 70B Instruct | Slowoffloads to RAM30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.6.5 tok/s | Slowoffloads to RAM30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.6.6 tok/s | Runs welltight fit30Basic quality (D) · 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.52.4 tok/s |
| gpt-oss 120B | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run84Great quality (A) · 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K3.1 Pro (2.8T) | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run69Good quality (B) · 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4.1 Flash 1017 (552B-A16B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.3 (753B-A40B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.8 Max (2.4T-A95B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K3 (2.8T) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| LongCat 2.5 (1.6T-A48B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.6 Pro (1T-A42B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.1 Flash 1019 (560B-A25B) | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M3.1 Flash (428B-A23B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nex N2.6 (321B-A18B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Step 5 Preview (600B-A27B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.3 Flash (320B-A18B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.6 Flash (310B-A15B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4.1 Flash (552B-A16B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 4 (1T-A52B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.8 Flash Next (125B-A6B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.2 (753B-A40B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hunyuan Hy4 Preview (780B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Muse Spark Open (816B-A42B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Pro (1.6T-A49B) | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run67Good quality (B) · 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.6 (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.1 Flash (560B-A25B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.5 Pro (1T-A42B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| Intern-S2 (397B-A17B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5.1 (744B-A40B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nex N2 Pro (397B-A17B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Pro (April 2026) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Naive N0.5 Flash (309B-A15.5B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.5 397B-A17B | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M3 (427B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-5 (744B-A40B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| K2-Horizon (375B-A23B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Hunyuan Hy3 (295B-A21B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Beam 1.0 (501B-A23B) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.5 (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2.7 Code (1T) | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Flash (284B-A13B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2.5 (310B-A15B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ornith 1.5 397B-A17B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3.5 Super VL 120B-A12B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.24.4 tok/s |
| DeepSeek V3.2 (671B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Inkling Small (276B-A12B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.7 (355B-A32B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Inkling (975B-A41B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.7 (230B-A10B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Step 3.7 Flash (198B-A11B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3 Ultra 550B-A55B | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek R1 0528 (671B) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Thinking (1T) | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run65Good quality (B) · 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ornith 1.5 122B-A10B | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.28.4 tok/s |
| DeepSeek V3.2 Exp | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 2.6 1T | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.5 (230B-A10B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek R1 (671B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V4 Flash (April 2026) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3.5 122B-A10B | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.28.4 tok/s |
| Step 3.5 Flash (196B-A11B) | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.1 Terminus | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run64Good quality (B) · 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3.1 (671B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Instruct 0905 (1T) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2.1 (230B-A10B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 235B-A22B Thinking 2507 | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 Coder 480B-A35B | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiMo V2 Flash (309B-A15B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 1T | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Large 3 (675B) | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3 235B-A22B Instruct 2507 | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Solar Open 2 250B | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run63Good quality (B) · 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kimi K2 Instruct (July 2025) | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax M2 (230B-A10B) | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen3-VL 235B-A22B Instruct | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run62Good quality (B) · 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A+ (218B-A25B) | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| K-EXAONE 236B-A23B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| LongCat Flash Chat (560B) | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Nemotron 3 Super 120B-A12B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.24.6 tok/s |
| Qwen3 235B-A22B | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run61Good quality (B) · 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Devstral 2 (123B) | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| InternVL3.5 241B-A28B | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Laguna S 2.1 (118B-A8B) | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.35.3 tok/s |
| Llama 3.1 Nemotron Ultra 253B v1 | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— |
| Baichuan M3 235B-A22B | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.6 (355B-A32B) | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3 0324 (671B) | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.6V | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.27.8 tok/s |
| MiniMax-M1 80K | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A Reasoning (111B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking.3.3 tok/s |
| GLM-4.5 (355B-A32B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.0 Flash (127B-A5B) | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax-M1 40K | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run57Good quality (B) · 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mistral Small 4 (119B-A6B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.47.4 tok/s |
| Mistral Medium 3.5 (128B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Trinity Large Thinking (400B-A13B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command A (111B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking.3.3 tok/s |
| ERNIE 4.5 300B-A47B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.5V | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.27.8 tok/s |
| Hermes 4 405B | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 2.6 Flash (104B-A7.4B) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.46.5 tok/s |
| Mistral Large 2.1 (2411) | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run56Good quality (B) · 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| GLM-4.5 Air (106B-A12B) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.26.7 tok/s |
| Mistral Large 2 (123B) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| NorthCUA Pro | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Pixtral Large 124B (2411) | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run54Fair quality (C) · 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| dots.llm1 Instruct | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| MiniMax-Text-01 | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Tülu 3 405B | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run53Fair quality (C) · 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V3 (671B) | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek Coder V2 (236B) | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Leanstral 119B-A6B | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload51Fair quality (C) · 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.44.2 tok/s |
| DeepSeek Coder V2 Instruct (0614) | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| DeepSeek V2.5 1210 | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 3.1 405B Instruct | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 4 Maverick (17B x 128E) | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Llama 4 Scout 17B-16E | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload50Fair quality (C) · 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.19.8 tok/s |
| Command R+ (08-2024) 104B | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking.3.6 tok/s |
| DeepSeek V2.5 | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Groot 65B A6B 261023 | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.47.9 tok/s |
| Jamba 1.5 Large | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Ling 3.0 flash Sante | Won't run47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run47Fair quality (C) · 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— |
| Command R+ 104B (04-2024) | Won't run44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking.— | Slowheavy RAM offload44Fair quality (C) · 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking.3.6 tok/s |
| DeepSeek V2 Chat | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run43Fair quality (C) · 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— |
| Mixtral 8x22B Instruct | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run38Basic quality (D) · 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen1.5 110B Chat | Won't run36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Very slowheavy RAM offload36Basic quality (D) · 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking.2.9 tok/s |
| Grok-1 (314B) | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— |
| Snowflake Arctic Instruct | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run35Basic quality (D) · 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking.— |
| Falcon 180B Chat | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run23Basic quality (D) · 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking.— |
Image models
58 of 61 image models run on the NVIDIA GeForce RTX 3090. 24.0 GB usable for models.
58 of 61 image models run on the NVIDIA GeForce RTX 4090. 24.0 GB usable for models.
60 of 61 image models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | RTX 3090fit · speed · quality | RTX 4090fit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| Krea 2 Turbo | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.7.3 s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.3.9 s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| ERNIE-Image Turbo | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.6.9 s | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.3.7 s | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.2.7 s |
| FLUX.2 [klein] 9B | Runs great82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.3.0 s | Runs great82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.1.6 s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.1.7 s |
| FLUX 3 Image Klein | Runs great89Excellent quality (A+) · 89 of 100: how capable the model is at FP8, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.9.6 s | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.6.6 s | Runs great91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.4.8 s |
| FLUX.2 [klein] 4B | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.1.9 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.1.0 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.0.8 s |
| Z-Image Turbo | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.5.4 s | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.2.9 s | Runs great84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.2.1 s |
| FLUX.1 Krea [dev] | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs great80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.9.0 s |
| ERNIE-Image | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs well86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| MOSS-Image SFT | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.24 s | Runs great80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.13 s | Runs great80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11 s |
| FLUX.1 [schnell] | Runs great68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.3.4 s | Runs great68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.1.9 s |
| FLUX.1 [dev] | Runs well78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.9.0 s |
| Fibo 2.0 Dev | Slowencoder in RAM93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.44 s | Runs wellencoder in RAM93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.24 s | Runs welltight fit93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.19 s |
| FLUX.1 Kontext [dev] | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.37 s | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| FLUX.1 Fill [dev] | Runs well74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.21 s | Runs great74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.9.4 s |
| Ideogram 4 | Slowtight fit85Excellent quality (A+) · 85 of 100: how capable the model is at NF4, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.52 s | Runs welltight fit85Excellent quality (A+) · 85 of 100: how capable the model is at NF4, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.28 s | Runs welltight fit90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.19 s |
| Qwen-Image 2.1 Turbo | Runs welltight fit · encoder in RAM87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.21 s | Runs welltight fit · encoder in RAM87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs welltight fit87Excellent quality (A+) · 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.8.7 s |
| Stable Diffusion 3.5 Large Turbo | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.3.5 s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Runs great61Good quality (B) · 61 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.1.5 s |
| Chroma1-HD | Runs well76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.34 s | Runs well76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.18 s | Runs well76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.15 s |
| Z-Image | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.29 s | Runs well80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.15 s | Runs well82Great quality (A) · 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| Kolors | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.7.5 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.4.0 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.3.3 s |
| Iris-3B | Runs well72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.30 s | Runs well72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs well72Great quality (A) · 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 s |
| Stable Diffusion 3.5 Large | Runs well64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.19 s | Runs great64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.9.9 s | Runs great64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.8.3 s |
| HiDream-I1 Fast | Runs wellencoder in RAM74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.18 s | Runs greatencoder in RAM74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.9.5 s | Runs great70Great quality (A) · 70 of 100: how capable the model is at NF4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.2 s |
| Ming-Image 0.1 Design | Runs welltight fit · encoder in RAM84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.29 s | Runs welltight fit · encoder in RAM84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.15 s | Runs wellencoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| CogView4 6B | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.30 s | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs well68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.14 s |
| HiDream-I1 Dev | Runs wellencoder in RAM78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.30 s | Runs wellencoder in RAM78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.16 s | Runs well74Great quality (A) · 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.16 s |
| Stable Diffusion 3 Medium | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.5.8 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.3.1 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.2.6 s |
| OmniGen2 | Runs well66Good quality (B) · 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.30 s | Runs well64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.14 s | Runs great64Good quality (B) · 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.12 s |
| MiniMax H3 Image | Runs wellCPU offload91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.22 s | Runs greatCPU offload91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs greatCPU offload91Excellent quality (A+) · 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.9.8 s |
| HiDream-I1 Full | Slowencoder in RAM82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.80 s | Slowencoder in RAM82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.43 s | Runs wellencoder in RAM82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.36 s |
| Stable Cascade | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.9.4 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.5.0 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.2 s |
| FLUX.2 [dev] | Slowtight fit · encoder in RAM85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.85 s | Slowtight fit · encoder in RAM85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.45 s | Runs wellencoder in RAM85Excellent quality (A+) · 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.38 s |
| Juggernaut XL v9 | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.8.5 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.4.5 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| RealVisXL V5.0 | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.8.5 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.4.5 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| Qwen-Image 2512 | Very slowencoder in RAM83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.3.6 min | Slowencoder in RAM83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.1.9 min | Slow83Great quality (A) · 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.1.5 min |
| Animagine XL 4.0 | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.7.9 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.4.2 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| Illustrious XL v0.1 | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.7.9 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.4.2 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| Stable Diffusion 3.5 Medium | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.10 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.6.0 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.5.0 s |
| AuraFlow v0.3 | Runs well58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.26 s | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.13 s | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.11 s |
| FireRed-Image-Edit 1.1 | Very slowencoder in RAM80Great quality (A) · 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.2 min | Slowencoder in RAM80Great quality (A) · 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Slow80Great quality (A) · 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.80 s |
| PixArt-Σ XL 1024 | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.8 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.1.5 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.1.3 s |
| DreamShaper XL v2 Turbo | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.2.4 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.1 s |
| Pony Diffusion V6 XL | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.7.1 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.8 s | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| Qwen-Image-Edit 2511 | Very slowencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.2 min | Slowencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.1.7 min | Slow79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.80 s |
| Animagine XL 3.1 | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.7.9 s | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.2 s | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.3.5 s |
| Stable Diffusion XL 1.0 | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.8.5 s | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.5 s | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.3.8 s |
| Lumina-Image 2.0 | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.12 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.0 s | Runs great62Good quality (B) · 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.5.9 s |
| SDXL-Lightning (4-step) | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.0.7 s | Runs great54Fair quality (C) · 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.0.6 s |
| Hunyuan-DiT v1.2 | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.11 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.6.0 s | Runs great52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.5.0 s |
| Playground v2.5 | Runs great58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.13 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.7.5 s | Runs great60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.6.3 s |
| Qwen-Image | Very slowCPU offload86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.3.3 min | SlowCPU offload86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.1.8 min | Slowencoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.71 s |
| SDXL Turbo | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.0.3 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.0.2 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.0.1 s |
| Realistic Vision V5.1 | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.2.3 s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.1.2 s | Runs great42Fair quality (C) · 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| Kandinsky 3 | Runs wellencoder in RAM48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.18 s | Runs greatencoder in RAM48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.9.5 s | Runs great48Fair quality (C) · 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.7.5 s |
| DreamShaper 8 | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.2.3 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.2 s | Runs great40Fair quality (C) · 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| Stable Diffusion 1.5 | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.2.3 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking.1.2 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking.1.0 s |
| LCM DreamShaper v7 | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.0.5 s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking.0.3 s | Runs great30Basic quality (D) · 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.0.2 s |
| SD Turbo | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.0.2 s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking.0.1 s | Runs great28Basic quality (D) · 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking.0.1 s |
| Cosmos3-Super Text2Image (64B) | Won't run90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking.— | Runs wellCPU offload90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.39 s |
| HunyuanImage 3.0 (80B-A13B) | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| Qwen-Image 2.1 | Won't run86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Slowtight fit · encoder in RAM86Excellent quality (A+) · 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.63 s |
Video models
31 of 43 video models run on the NVIDIA GeForce RTX 3090. 24.0 GB usable for models.
31 of 43 video models run on the NVIDIA GeForce RTX 4090. 24.0 GB usable for models.
40 of 43 video models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | RTX 3090fit · speed · quality | RTX 4090fit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| LTX-2.5 (22B, video + audio) | Runs greatCPU offload86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.3 min | Runs greatCPU offload87Excellent quality (A+) · 87 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.2.3 min | Runs greatencoder in RAM81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.1.6 min |
| Bernini-R A14B (video generation + editing) | Slowencoder in RAM83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.21.3 min | Runs wellencoder in RAM83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.11.3 min | Runs well83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.0 min |
| LTX-Video 2B | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.47 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.25 s | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.21 s |
| CogVideoX 2B | Runs great45Fair quality (C) · 45 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.1.9 min | Runs great45Fair quality (C) · 45 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.60 s | Runs great45Fair quality (C) · 45 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.50 s |
| FramePack I2V (HunyuanVideo 13B) | Runs wellencoder in RAM72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.11.8 min | Runs wellencoder in RAM72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.6.3 min | Runs welltight fit72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.0 min |
| CogVideoX 5B I2V | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.5.1 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.2.7 min | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.5 min |
| HY-OmniWeaving (8.3B, unified video) | Slowencoder in RAM74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.15.7 min | Runs wellencoder in RAM74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.8.3 min | Runs well74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.6.7 min |
| CogVideoX 5B | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.5.1 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.2.7 min | Runs great55Good quality (B) · 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.2.5 min |
| Wan 2.2 S2V 14B (speech to video) | Slowtight fit70Great quality (A) · 70 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.37.6 min | Slowtight fit70Great quality (A) · 70 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.20.0 min | Slowtight fit74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.12.5 min |
| Wan 2.1 I2V 14B 480P | Slowtight fit68Good quality (B) · 68 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.33.8 min | Slowtight fit68Good quality (B) · 68 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.18.0 min | Runs well72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.11.3 min |
| Kandinsky 6.0 Video Lite (3B, video + audio) | Runs wellencoder in RAM68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.10.2 min | Runs wellencoder in RAM68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.5.4 min | Runs wellencoder in RAM68Good quality (B) · 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.4.5 min |
| Wan 2.1 VACE 1.3B | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.7.3 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.3.9 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.3 min |
| LTX-Video 13B 0.9.8 (distilled) | Runs greatCPU offload66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.2.2 min | Runs greatCPU offload66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.70 s | Runs welltight fit66Good quality (B) · 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.45 s |
| SkyReels V2 Diffusion Forcing 1.3B | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.8.5 min | Runs well50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.4.5 min | Runs great50Fair quality (C) · 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.8 min |
| LTX-2.3 | SlowCPU offload84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.14.7 min | Runs wellCPU offload84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.7.8 min | Runs wellencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.0 min |
| Wan 2.1 T2V 1.3B | Runs well53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.6.8 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.3.6 min | Runs great53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.3.0 min |
| LTX-2 (19B, video + audio) | SlowCPU offload80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.14.7 min | Runs wellCPU offload80Great quality (A) · 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.7.8 min | Runs wellencoder in RAM76Great quality (A) · 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.7.0 min |
| MiniMax H3 (33B, video + audio) | Slowtight fit · CPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.38.1 min | Slowtight fit · CPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.20.3 min | SlowCPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.17.0 min |
| Cosmos-Predict2 2B Video2World | Runs welltight fit · encoder in RAM56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.7.4 min | Runs welltight fit · encoder in RAM56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.3.9 min | Runs great56Good quality (B) · 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.3.1 min |
| SCAIL-2 14B (character animation) | Very slowCPU offload80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.48.9 min | SlowCPU offload80Great quality (A) · 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.26.0 min | Slowtight fit78Great quality (A) · 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.15.1 min |
| Wan 2.2 I2V A14B | Very slowtight fit · CPU offload84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.58.7 min | Slowtight fit · CPU offload84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.31.2 min | Slowencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.28.1 min |
| Wan 2.2 T2V A14B | Very slowtight fit · CPU offload84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.58.7 min | Slowtight fit · CPU offload84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.31.2 min | Slowencoder in RAM79Great quality (A) · 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.28.1 min |
| Mochi 1 Preview (10B) | Slowtight fit · encoder in RAM60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.32.6 min | Slowtight fit · encoder in RAM60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking.17.3 min | Slow60Good quality (B) · 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.13.8 min |
| SkyReels V2 T2V 14B 540P | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.57.0 min | SlowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.30.3 min | Slowtight fit72Great quality (A) · 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.17.6 min |
| Wan 2.1 I2V 14B 720P | Very slowtight fit · CPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.73.3 min | Slowtight fit · CPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.39.0 min | Slowencoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.35.1 min |
| Wan 2.1 T2V 14B | Very slowtight fit · CPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.73.3 min | Slowtight fit · CPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking.39.0 min | Slowencoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.35.1 min |
| Wan 2.2 TI2V 5B | Slowtight fit · CPU offload68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.19.1 min | Runs welltight fit · CPU offload68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.10.1 min | Runs welltight fit65Good quality (B) · 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.8.7 min |
| Open-Sora 2.0 (11B) | Very slowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.2.0 h | Very slowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.65.0 min | Very slowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.54.4 min |
| Wan 2.2 Animate 14B | Very slowtight fit · CPU offload72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.5 h | Very slowtight fit · CPU offload72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.49.4 min | Slowtight fit · encoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.33.4 min |
| AnimateDiff v3 (SD 1.5) | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.38 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.20 s | Runs great35Basic quality (D) · 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking.17 s |
| Wan 2.1 VACE 14B | Very slowtight fit · CPU offload72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.1.8 h | Very slowtight fit · CPU offload72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking.57.2 min | Slowtight fit · encoder in RAM72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.38.6 min |
| Cosmos3-Super Image2Video (64B) | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| Kandinsky 6.0 Pro (29B, video + audio) | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Very slowtight fit · CPU offload88Excellent quality (A+) · 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.53.3 min |
| HunyuanVideo 1.5 (8.3B) | Won't run82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Slowtight fit · encoder in RAM82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.23.7 min |
| SkyReels V2 T2V 14B 720P | Won't run76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | SlowCPU offload76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| Step-Video T2V (30B) | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Very slowCPU offload74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking.50.6 min |
| Wan-Dancer 14B (music to dance) | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— |
| Cosmos-Predict2 14B Video2World | Won't run68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | SlowCPU offload68Good quality (B) · 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.26.1 min |
| HunyuanVideo 13B | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | SlowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| HunyuanVideo I2V 13B | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run66Good quality (B) · 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking.— | SlowCPU offload70Great quality (A) · 70 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.39.1 min |
| Cosmos 1.0 Diffusion 7B Text2World | Won't run60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking.— | Runs welltight fit · CPU offload60Good quality (B) · 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking.10.0 min |
| CogVideoX 1.5 5B | Won't run58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run58Good quality (B) · 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— |
| Allegro (2.8B) | Won't run52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking.— | Slowtight fit · encoder in RAM52Fair quality (C) · 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking.17.6 min |
Decision models
68 of 70 decision models run on the NVIDIA GeForce RTX 3090. 24.0 GB usable for models.
68 of 70 decision models run on the NVIDIA GeForce RTX 4090. 24.0 GB usable for models.
68 of 70 decision models run on the NVIDIA GeForce RTX 5090. 32.0 GB usable for models.
| Model | RTX 3090fit · speed · quality | RTX 4090fit · speed · quality | RTX 5090fit · speed · quality |
|---|---|---|---|
| Qwen3 Reranker 4B | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.99 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.52 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.39 ms |
| zerank-2 | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.99 ms | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.52 ms | Runs great88Excellent quality (A+) · 88 of 100: how capable the model is at F16, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.39 ms |
| Jina Reranker v3.5 | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.18 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.11 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| Shieldstral 1.0 3B | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.95 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.50 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.38 ms |
| Intern-Decision 4B | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.66 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.39 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.28 ms |
| Jina Reranker v3 | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.18 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.11 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| d1 3B | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.47 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.28 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.21 ms |
| mxbai-rerank-large v2 | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.40 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.23 ms | Runs great82Great quality (A) · 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.17 ms |
| Qwen3 Reranker 8B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.182 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.88 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at F16, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking.75 ms |
| FRIDA-Decisions | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.14 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.8.4 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.7.5 ms |
| GLiNER2.5 Decide | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.10 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.6.6 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.6.1 ms |
| LemonDecider 4B | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.61 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.28 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.23 ms |
| Qwen3 Reranker 0.6B | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.18 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.11 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms |
| zerank-1 small | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.45 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.25 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.19 ms |
| BGE Reranker v2 Gemma | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.63 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.34 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.26 ms |
| Llama Prompt Guard 2 86M | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.11 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.7.0 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms |
| mxbai-rerank-base v2 | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.9.9 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.8.3 ms |
| Skywork Reward V2 Llama 3.1 8B | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.330 ms | Runs well90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.144 ms | Runs great90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.115 ms |
| GLiClass ModernBERT Large v3 | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.9.0 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.6.2 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| OpenAI Privacy Filter | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.4.6 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.4.3 ms | Runs great78Great quality (A) · 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.4.2 ms |
| Qwen3Guard Gen 4B | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.125 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.84 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.85 ms |
| BGE Reranker v2 M3 | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.18 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.10 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.8.8 ms |
| DeBERTa v3 Large Zero-shot v2.0 | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.9.5 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.5.9 ms |
| DuoGuard 0.5B | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.16 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.9.3 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.8.2 ms |
| Skywork Reward V2 Qwen3 0.6B | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.34 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.17 ms | Runs great72Great quality (A) · 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.14 ms |
| GTE Reranker ModernBERT Base | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.7.7 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.5.6 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms |
| Jina Reranker v2 Base Multilingual | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.11 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.7.0 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms |
| ModernBERT Large Zero-shot v2.0 | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.9.0 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.6.2 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| Skywork Reward V2 Qwen3 8B | Runs well84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.333 ms | Runs well84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.146 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.116 ms |
| GLiGuard 300M | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.6.3 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.8 ms |
| Granite Embedding Reranker English R2 | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.7.7 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.6 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.3 ms |
| Laya | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.9.2 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.6.3 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.8 ms |
| Llama Prompt Guard 2 22M | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.5.8 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.4.8 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.4.6 ms |
| Qwen3Guard Gen 0.6B | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.36 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.26 ms | Runs great74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.18 ms |
| Qwen3Guard Gen 8B | Runs well85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.229 ms | Runs great85Excellent quality (A+) · 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.132 ms | Runs great86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.103 ms |
| GLiClass Base v3 | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.5.0 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.4.8 ms |
| ProtectAI DeBERTa v3 Prompt Injection v2 | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.8.5 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.5.9 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.5.5 ms |
| Llama 3.1 Nemotron Safety Guard 8B v3 | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.225 ms | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.129 ms | Runs great84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.102 ms |
| Granite Guardian 4.1 8B | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.234 ms | Runs well83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.135 ms | Runs great83Great quality (A) · 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.94 ms |
| NuExtract 2.0 4B | Runs great72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.120 ms | Runs great73Great quality (A) · 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.108 ms | Runs great76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.97 ms |
| NuExtract 2.0 2B | Runs great69Good quality (B) · 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.93 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.98 ms | Runs great70Great quality (A) · 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking.59 ms |
| DeBERTa v3 Base Zero-shot v2.0 | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.6.2 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.5.0 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.8 ms |
| GLiNER2 Base v1 | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.6.6 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.5.1 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.4.9 ms |
| WildGuard 7B | Runs well79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.203 ms | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.117 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.92 ms |
| HHEM 2.1 Open | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.9.5 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.6.4 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.9 ms |
| LettuceDetect Large | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.24 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.13 ms | Runs great66Good quality (B) · 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.11 ms |
| Granite Guardian 3.3 8B | Runs well79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.229 ms | Runs great79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking.131 ms | Runs great80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking.103 ms |
| Laya Multilingual | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.8.0 ms | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms | Runs great60Good quality (B) · 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.5.4 ms |
| ShieldGemma 2B | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.66 ms | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.35 ms | Runs great58Good quality (B) · 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking.27 ms |
| GLiNER Multi v2.1 | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.7.6 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.6 ms | Runs great62Good quality (B) · 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.2 ms |
| BART Large MNLI | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.9.1 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.6.2 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.5.7 ms |
| ShieldGemma 9B | Runs well64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.204 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.98 ms | Runs great64Good quality (B) · 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking.85 ms |
| ms-marco MiniLM L6 v2 | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.4.6 ms | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.4.2 ms | Runs great52Fair quality (C) · 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking.4.2 ms |
| Llama Guard 3 8B | Runs well67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.225 ms | Runs great67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking.129 ms | Runs great68Good quality (B) · 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.102 ms |
| Llama Guard 3 1B | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.67 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.48 ms | Runs great56Good quality (B) · 56 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.32 ms |
| Llama Guard 4 12B | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.334 ms | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.191 ms | Runs great65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.133 ms |
| Nemotron Content Safety Reasoning 4B | Slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.953 ms | Slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.842 ms | Runs well74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.346 ms |
| ShieldGemma 27B | Slow62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.571 ms | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.260 ms | Runs well65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking.206 ms |
| gpt-oss-safeguard 20B | Slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.0 s | Slow76Great quality (A) · 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.812 ms |
| DynaGuard 1.7B | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.327 ms | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.284 ms | Runs well66Good quality (B) · 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking.165 ms |
| Selene 1 Mini 8B | Very slow79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.8 s | Slow77Great quality (A) · 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow79Great quality (A) · 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.902 ms |
| DynaGuard 8B | Slow75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow75Great quality (A) · 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.1.1 s | Slow78Great quality (A) · 78 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.918 ms |
| CompassJudger-2 32B | Very slowtight fit81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.5.6 s | Very slowtight fit81Great quality (A) · 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.4.5 s | Very slow83Great quality (A) · 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.3.2 s |
| CompassJudger-2 7B | Slow74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow75Great quality (A) · 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.1.3 s | Slow77Great quality (A) · 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.856 ms |
| GLIDER | Slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.875 ms | Slow65Good quality (B) · 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.737 ms | Runs well62Good quality (B) · 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking.316 ms |
| Flow Judge v0.1 | Slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.875 ms | Slow63Good quality (B) · 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.737 ms | Runs well60Good quality (B) · 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.316 ms |
| Prometheus 2 7B | Slow64Good quality (B) · 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking.1.2 s | Slow65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.1.2 s | Slow67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking.814 ms |
| Prometheus 2 8x7B | Very slowoffloads to RAM70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.7.9 s | Very slowoffloads to RAM70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking.7.4 s | Very slowtight fit70Great quality (A) · 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking.1.6 s |
| GLiDE 27B | Won't run86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking.— |
| gpt-oss-safeguard 120B | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— | Won't run80Great quality (A) · 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking.— |
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.