Intel Arc Pro B70 vs Apple M5 Max (36 GB) for local AI

Pick two to four systems or devices to see which runs more models, faster, for the kind of work you do.

Hardware to compare, two to four

Winner: Intel Arc Pro B70

For general chat: Its ten best models average a score of 100 vs 97; it runs 50 more LLMs (514 vs 464). On the other hand, the Apple M5 Max (36 GB) is about 1.1ร— faster on its own top models.

Side by side

Swap sides All models on the Arc Pro B70 All models on the M5 Max

  Intel Arc Pro B70Intel Arc Pro B70 Apple MacBook Pro (14-inch, M5 Pro or M5 Max)Apple M5 Max (36 GB)
Memory 32.0 GB VRAM 36.0 GB unified
Usable for models 32.0 GB 24.1 GB
Memory bandwidth 608 GB/s 460 GB/s
FP16 compute 37.4 TFLOPS 15.1 TFLOPS
Launch price $949 โ€”
Released 2026 2026
LLMs that run 514 of 622 464 of 622
LLMs that run well 485 462
Image models that run 60 of 61 58 of 61
Video models that run 40 of 43 31 of 43
Decision models that run 68 of 70 67 of 70
Top-10 LLM average score 100 97
Top-10 LLM average speed 84.3 tok/s 96.1 tok/s
Best LLM MiniCPM-V 4.7 35B-A3B Gemma 4 26B-A4B

Highlighted values are the best of the two. Model results assume 8K of context and the default macOS GPU memory limit on Macs.

LLMs

514 of 622 LLMs run on the Intel Arc Pro B70. 32.0 GB usable for models.

464 of 622 LLMs run on the Apple M5 Max (36 GB). 24.1 GB usable for models.

Model QualityArc Pro B70 SpeedArc Pro B70 RunsArc Pro B70 QualityM5 Max SpeedM5 Max RunsM5 Max
MiniCPM-V 4.7 35B-A3BOpenBMB ยท 35B (3.0B active) 92Excellent quality (A+) ยท 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 78Great quality (A) ยท 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Qwen3.5 35B-A3BAlibaba ยท 36B (3.0B active) 92Excellent quality (A+) ยท 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 78Great quality (A) ยท 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Qwen3.6 35B-A3BAlibaba ยท 36B (3.0B active) 92Excellent quality (A+) ยท 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 78Great quality (A) ยท 78 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Gemma 4 26B-A4BGoogle ยท 26B (3.8B active) 92Excellent quality (A+) ยท 92 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking. 59.2 tok/s Runs welltight fit 91Excellent quality (A+) ยท 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 100/100 (quality, fit, speed and memory use) sets the ranking. 67.2 tok/s Runs welltight fit
Nex N2 Mini (35B-A3B)Nex-AGI ยท 35B (3.0B active) 91Excellent quality (A+) ยท 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 87Excellent quality (A+) ยท 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
Sage-A1.5 35B-A3BInternScience ยท 36B (3.0B active) 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 76Great quality (A) ยท 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Raon-1 mini Preview (30B-A4B)KRAFTON ยท 29B (4.3B active) 87Excellent quality (A+) ยท 87 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking. 69.1 tok/s Runs great 83Great quality (A) ยท 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking. 81.9 tok/s Runs great
Ornith 1.5 35B-A3BDeepReinforce ยท 36B (3.0B active) 89Excellent quality (A+) ยท 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 99/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 75Great quality (A) ยท 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Apodex 1.1 Mini (35B-A3B)Apodex ยท 36B (3.0B active) 88Excellent quality (A+) ยท 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 74Great quality (A) ยท 74 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Ornith 1.0 35B-A3BDeepReinforce ยท 36B (3.0B active) 88Excellent quality (A+) ยท 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 74Great quality (A) ยท 74 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Flint (40B-A2.5B)Recursive ยท 40B (2.5B active) 85Excellent quality (A+) ยท 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking. 129 tok/s Runs great 75Great quality (A) ยท 75 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 139 tok/s Runs welltight fit
GLM-4.7 Flash (30B-A3B)Zhipu AI ยท 31B (3.0B active) 85Excellent quality (A+) ยท 85 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 98/100 (quality, fit, speed and memory use) sets the ranking. 101 tok/s Runs great 83Great quality (A) ยท 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
Qwen AgentWorld 35B-A3BAlibaba ยท 35B (3.0B active) 84Great quality (A) ยท 84 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking. 101 tok/s Runs great 82Great quality (A) ยท 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
Agents-A1 35B-A3BInternScience ยท 35B (3.0B active) 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 97/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 82Great quality (A) ยท 82 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
Qwen3.8 27BAlibaba ยท 28B 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 20.5 tok/s Runs well 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 17.5 tok/s Runs well
Nanbeige4.3 33BBOSS Zhipin (Nanbeige) ยท 33B 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 17.2 tok/s Runs well 80Great quality (A) ยท 80 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 19.0 tok/s Runs welltight fit
Laguna XS 2.1 (33B-A3B)Poolside ยท 33B (3.0B active) 81Great quality (A) ยท 81 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 68Good quality (B) ยท 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Qwen3.6 27BAlibaba ยท 28B 89Excellent quality (A+) ยท 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 20.5 tok/s Runs well 89Excellent quality (A+) ยท 89 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. 17.5 tok/s Runs well
Nemotron Cascade 2 30B-A3BNVIDIA ยท 32B (3.5B active) 77Great quality (A) ยท 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 91.5 tok/s Runs great 76Great quality (A) ยท 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 94.7 tok/s Runs welltight fit
DiffusionGemma 26B-A4B ITGoogle ยท 26B (4.0B active) 77Great quality (A) ยท 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 72.9 tok/s Runs great 77Great quality (A) ยท 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 64.7 tok/s Runs welltight fit
KAT-Coder V2.5 Dev (35B-A3B)Kuaishou ยท 35B (3.0B active) 88Excellent quality (A+) ยท 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 101 tok/s Runs great 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
Qwen3.5 27BAlibaba ยท 28B 88Excellent quality (A+) ยท 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 20.5 tok/s Runs well 88Excellent quality (A+) ยท 88 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 17.5 tok/s Runs well
Muse Glimmer 30BMeta ยท 30B 87Excellent quality (A+) ยท 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs well 87Excellent quality (A+) ยท 87 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 16.3 tok/s Runs welltight fit
Gemma 4 31BGoogle ยท 31B 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 18.2 tok/s Runs well 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 15.5 tok/s Runs welltight fit
Nemotron 3 Nano 30B-A3BNVIDIA ยท 32B (3.5B active) 74Great quality (A) ยท 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 91.5 tok/s Runs great 73Great quality (A) ยท 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 94.7 tok/s Runs welltight fit
Qwen3 30B-A3B Instruct 2507Alibaba ยท 31B (3.3B active) 74Great quality (A) ยท 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 95.2 tok/s Runs great 73Great quality (A) ยท 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 98.5 tok/s Runs welltight fit
TMax 27BAi2 ยท 27B 85Excellent quality (A+) ยท 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 20.8 tok/s Runs well 85Excellent quality (A+) ยท 85 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 17.8 tok/s Runs well
Qwen3-VL 30B-A3B ThinkingAlibaba ยท 31B (3.3B active) 73Great quality (A) ยท 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 95.2 tok/s Runs great 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 98.5 tok/s Runs welltight fit
LFM2.5 24B-A2BLiquid AI ยท 24B (2.5B active) 73Great quality (A) ยท 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 102 tok/s Runs great 71Great quality (A) ยท 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 104 tok/s Runs great
Fara 1.5 27BMicrosoft ยท 27B 83Great quality (A) ยท 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 20.8 tok/s Runs well 83Great quality (A) ยท 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 17.8 tok/s Runs well
Qwen3 30B-A3BAlibaba ยท 31B (3.3B active) 71Great quality (A) ยท 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 95.2 tok/s Runs great 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 98.5 tok/s Runs welltight fit
Qwen3-VL 30B-A3B InstructAlibaba ยท 31B (3.3B active) 71Great quality (A) ยท 71 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 95.2 tok/s Runs great 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 98.5 tok/s Runs welltight fit
Hunyuan Hy-Memory 30B-A3BTencent ยท 30B (3.0B active) 82Great quality (A) ยท 82 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 101 tok/s Runs great 80Great quality (A) ยท 80 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
UI-Mate 27BTencent ยท 27B 81Great quality (A) ยท 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 20.8 tok/s Runs well 81Great quality (A) ยท 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 17.8 tok/s Runs well
Nemotron 3.5 Lightning 30B-A3BNVIDIA ยท 32B (3.0B active) 69Good quality (B) ยท 69 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 101 tok/s Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
Sarvam 30BSarvam AI ยท 32B (2.4B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 116 tok/s Runs great 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 120 tok/s Runs welltight fit
Qwen3-Omni 30B-A3B InstructAlibaba ยท 35B (3.3B active) 69Good quality (B) ยท 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 83.7 tok/s Runs welltight fit 58Good quality (B) ยท 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 118 tok/s Runs welltight fit
North Mini Code 1.0 (30B-A3B)Cohere ยท 30B (3.0B active) 77Great quality (A) ยท 77 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 101 tok/s Runs great 76Great quality (A) ยท 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
EXAONE 4.5 33BLG AI Research ยท 34B 77Great quality (A) ยท 77 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 16.6 tok/s Runs well 68Good quality (B) ยท 68 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 18.3 tok/s Runs well
Trinity Mini (26B-A3B)Arcee AI ยท 26B (3.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs great 65Good quality (B) ยท 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 80.0 tok/s Runs welltight fit
Gemma 4 12BGoogle ยท 12B 80Great quality (A) ยท 80 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 39.4 tok/s Runs great 78Great quality (A) ยท 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great
Qwen3 30B-A3B Thinking 2507Alibaba ยท 31B (3.3B active) 75Great quality (A) ยท 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 95.2 tok/s Runs great 74Great quality (A) ยท 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 98.5 tok/s Runs welltight fit
Qwen3 Coder 30B-A3BAlibaba ยท 31B (3.3B active) 75Great quality (A) ยท 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 95.2 tok/s Runs great 74Great quality (A) ยท 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 98.5 tok/s Runs welltight fit
ERNIE 4.5 21B-A3BBaidu ยท 22B (3.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 70.7 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 91.2 tok/s Runs great
Seed-OSS 36B InstructByteDance ยท 36B 76Great quality (A) ยท 76 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 15.8 tok/s Runs well 67Good quality (B) ยท 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs welltight fit
K2 Horizon 7BInstitute of Foundation Models ยท 9.0B 79Great quality (A) ยท 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 44.0 tok/s Runs great 79Great quality (A) ยท 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 37.5 tok/s Runs great
Hermes 4.3 36BNous Research ยท 36B 75Great quality (A) ยท 75 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 15.8 tok/s Runs well 66Good quality (B) ยท 66 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs welltight fit
Qwen3-VL 32B InstructAlibaba ยท 33B 74Great quality (A) ยท 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 17.1 tok/s Runs well 65Good quality (B) ยท 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 18.8 tok/s Runs welltight fit
Kimi Linear 48B-A3BMoonshot AI ยท 49B (3.0B active) 62Good quality (B) ยท 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 137 tok/s Runs great 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 145 tok/s Runs welltight fit
Granite 4.2 30BIBM ยท 29B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 19.5 tok/s Runs well 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 16.6 tok/s Runs welltight fit
SenseNova Loop 9B PreviewSenseTime ยท 9.7B 77Great quality (A) ยท 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great 75Great quality (A) ยท 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great
Qwen3 32BAlibaba ยท 33B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 63Good quality (B) ยท 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
Devstral Small 2 (24B)Mistral AI ยท 24B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Runs well 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 20.3 tok/s Runs well
Granite 4.0 H Small (32B-A9B)IBM ยท 32B (9.0B active) 60Good quality (B) ยท 60 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 44.2 tok/s Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 45.7 tok/s Runs welltight fit
gpt-oss 20BOpenAI ยท 21B (3.6B active) 70Great quality (A) ยท 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 108 tok/s Runs great 70Great quality (A) ยท 70 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 97.2 tok/s Runs great
Ornith 1.5 9BDeepReinforce ยท 9.7B 76Great quality (A) ยท 76 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 74Great quality (A) ยท 74 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great
Axion 1.0 35BAxion ยท 35B (3.0B active) 61Good quality (B) ยท 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
GLM-4 32B 0414Zhipu AI ยท 33B 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 17.5 tok/s Runs well 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 14.9 tok/s Runs welltight fit
ERNIE 4.5 21B-A3B ThinkingBaidu ยท 22B (3.0B active) 70Great quality (A) ยท 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 70.7 tok/s Runs great 67Good quality (B) ยท 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 91.2 tok/s Runs great
Maple Preview (20B-A1B)DeepGrove ยท 20B (1.5B active) 70Great quality (A) ยท 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 112 tok/s Runs great 69Good quality (B) ยท 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 125 tok/s Runs great
Mellum2.1 Thinking (12B-A2.5B)JetBrains ยท 12B (2.5B active) 70Great quality (A) ยท 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 43.4 tok/s Runs great 70Great quality (A) ยท 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 71.9 tok/s Runs great
Qwen3.5 9BAlibaba ยท 9.7B 75Great quality (A) ยท 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 73Great quality (A) ยท 73 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great
Granite 4.1 30BIBM ยท 29B 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 19.7 tok/s Runs well 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 16.8 tok/s Runs welltight fit
Kimi-VL A3B Thinking 2506Moonshot AI ยท 16B (3.0B active) 60Good quality (B) ยท 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 70.7 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 62.8 tok/s Runs great
Ornith 1.0 9BDeepReinforce ยท 9.7B 74Great quality (A) ยท 74 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 72Great quality (A) ยท 72 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great
Gemma 3 27B ITGoogle ยท 27B 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 20.8 tok/s Runs well 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs well
EXAONE 4.0 32BLG AI Research ยท 32B 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 17.8 tok/s Runs well 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 15.2 tok/s Runs welltight fit
Olmo 3.1 32B InstructAi2 ยท 32B 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs well 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs welltight fit
Groot 19B A3B 261023GrootTeam ยท 19B (3.0B active) 57Good quality (B) ยท 57 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 70.7 tok/s Runs great 56Good quality (B) ยท 56 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 80.0 tok/s Runs great
InternVL3.5 38BShanghai AI Lab ยท 38B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 14.9 tok/s Runs welltight fit 63Good quality (B) ยท 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs welltight fit
NorthCUA Minibeipo-ai ยท 35B (3.0B active) 58Good quality (B) ยท 58 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 89.3 tok/s Runs welltight fit 56Good quality (B) ยท 56 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs welltight fit
Ling 3.0 Tiny (8B-A1.3B)Ant Group (inclusionAI) ยท 7.9B (1.3B active) 60Good quality (B) ยท 60 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 71.0 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 63.4 tok/s Runs great
TMax 9BAi2 ยท 9.0B 73Great quality (A) ยท 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 44.2 tok/s Runs great 73Great quality (A) ยท 73 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 37.7 tok/s Runs great
LFM2 24B-A2BLiquid AI ยท 24B (2.3B active) 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 106 tok/s Runs great 54Fair quality (C) ยท 54 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 108 tok/s Runs great
MedGemma 27B Text ITGoogle ยท 27B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 21.1 tok/s Runs well 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 18.0 tok/s Runs well
Baichuan M2 32BBaichuan ยท 33B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 58Good quality (B) ยท 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
Qwen2.5 32B InstructAlibaba ยท 33B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 58Good quality (B) ยท 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
Apriel 1.5 15B ThinkerServiceNow ยท 15B 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 32.7 tok/s Runs great
Devstral Small 1.1 (24B)Mistral AI ยท 24B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 24.2 tok/s Runs well 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Magistral Small 2507Mistral AI ยท 24B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 24.2 tok/s Runs well 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Nemotron 3.5 Bolt E4BNVIDIA ยท 8.0B 70Great quality (A) ยท 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 64.5 tok/s Runs great 70Great quality (A) ยท 70 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 55.0 tok/s Runs great
Terminal RSI 27BTeriminal-RSI ยท 28B 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 20.5 tok/s Runs well 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 17.5 tok/s Runs well
Phi-4 Reasoning Vision 15BMicrosoft ยท 15B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 37.7 tok/s Runs great 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 32.2 tok/s Runs great
Qwen2.5-VL 32B InstructAlibaba ยท 33B 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 17.0 tok/s Runs well 57Good quality (B) ยท 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 18.8 tok/s Runs welltight fit
Mistral Small 3.2 24BMistral AI ยท 24B 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Runs well 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 20.3 tok/s Runs well
Magistral Small 2506Mistral AI ยท 24B 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 24.2 tok/s Runs well 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Gemma 4 E4BGoogle ยท 8.0B 68Good quality (B) ยท 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 64.5 tok/s Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 55.0 tok/s Runs great
Hemmingway 1.5Hemmingway ยท 27B 63Good quality (B) ยท 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 21.2 tok/s Runs well 63Good quality (B) ยท 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 18.1 tok/s Runs well
Llama 3.3 Nemotron Super 49B v1.5NVIDIA ยท 50B 67Good quality (B) ยท 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 14.8 tok/s Runs welltight fit 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 13.6 tok/s Runs wellover GPU memory limit
QwQ 32BAlibaba ยท 33B 74Great quality (A) ยท 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 65Good quality (B) ยท 65 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
InternVL3 38BShanghai AI Lab ยท 38B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 14.9 tok/s Runs welltight fit 58Good quality (B) ยท 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs welltight fit
Granite 4.2 8BIBM ยท 8.8B 67Good quality (B) ยท 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 45.0 tok/s Runs great 67Good quality (B) ยท 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
Phi-3.5 MoE (42B-A6.6B)Microsoft ยท 42B (6.6B active) 53Fair quality (C) ยท 53 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 66.9 tok/s Runs welltight fit 47Fair quality (C) ยท 47 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 72.8 tok/s Runs welltight fit
Mistral Small 3.1 24BMistral AI ยท 24B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 23.8 tok/s Runs well 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 20.3 tok/s Runs well
Qwen3 14BAlibaba ยท 15B 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 38.5 tok/s Runs great 64Good quality (B) ยท 64 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Ling Mini 2.0 (16B-A1.4B)Ant Group (inclusionAI) ยท 16B (1.4B active) 52Fair quality (C) ยท 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 116 tok/s Runs great 52Fair quality (C) ยท 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 106 tok/s Runs great
Olmo 3.1 32B ThinkAi2 ยท 32B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs well 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs welltight fit
OpenReasoning Nemotron 32BNVIDIA ยท 33B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 63Good quality (B) ยท 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
Skywork OR1 32BSkywork ยท 33B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 63Good quality (B) ยท 63 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
LFM2.5 8B-A1BLiquid AI ยท 8.5B (1.5B active) 52Fair quality (C) ยท 52 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 64.1 tok/s Runs great 52Fair quality (C) ยท 52 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 57.1 tok/s Runs great
Gemma 2 27B ITGoogle ยท 27B 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 21.0 tok/s Runs well 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 17.9 tok/s Runs welltight fit
Falcon-H1 34B InstructTII ยท 34B 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 16.9 tok/s Runs well 53Fair quality (C) ยท 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 18.7 tok/s Runs well
MiMo V2.6 Distill Qwen 9BXiaomi ยท 9.4B 77Great quality (A) ยท 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 75Great quality (A) ยท 75 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great
Hermes 4 14BNous Research ยท 15B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Mistral Small 3 24B (2501)Mistral AI ยท 24B 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 24.2 tok/s Runs well 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Gemma 4 E2BGoogle ยท 5.1B 62Good quality (B) ยท 62 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 58.2 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 49.6 tok/s Runs great
Kimi-VL A3B InstructMoonshot AI ยท 16B (3.0B active) 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 70.7 tok/s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 62.8 tok/s Runs great
EXAONE Deep 32BLG AI Research ยท 32B 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 17.8 tok/s Runs well 62Good quality (B) ยท 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 19.6 tok/s Runs well
GLM-Z1 32B (0414)Zhipu AI ยท 33B 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 17.5 tok/s Runs well 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 14.9 tok/s Runs welltight fit
Command R (08-2024) 32BCohere ยท 32B 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs well 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs welltight fit
Nanbeige4.2 3BBOSS Zhipin (Nanbeige) ยท 4.2B 68Good quality (B) ยท 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 69.6 tok/s Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 59.4 tok/s Runs great
Phi-4 (14B)Microsoft ยท 15B 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 38.8 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 33.1 tok/s Runs great
InternVL3 14BShanghai AI Lab ยท 15B 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 37.7 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 32.2 tok/s Runs great
Voxtral Small 24BMistral AI ยท 24B 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 23.5 tok/s Runs well 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 20.0 tok/s Runs well
Dolphin Mistral 24B Venice EditionCognitive Computations ยท 24B 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Runs well 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 20.3 tok/s Runs well
Ministral 3 14BMistral AI ยท 14B 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 34.9 tok/s Runs great
Nous Hermes 2 Mixtral 8x7B DPONous Research ยท 47B (12.9B active) 49Fair quality (C) ยท 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 38.5 tok/s Runs welltight fit 36Basic quality (D) ยท 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs welltight fit
DeepSeek R1 Distill Qwen 32BDeepSeek ยท 33B 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 60Good quality (B) ยท 60 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
OLMo 2 32B InstructAi2 ยท 32B 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs well 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs welltight fit
Aya Expanse 32BCohere ยท 32B 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 17.7 tok/s Runs well 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 15.1 tok/s Runs welltight fit
Nemotron Labs Dory 1.0 4BNVIDIA ยท 4.0B 68Good quality (B) ยท 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 72.9 tok/s Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 62.2 tok/s Runs great
Qwen2.5 14B InstructAlibaba ยท 15B 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 38.8 tok/s Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 33.1 tok/s Runs great
Aya Vision 32BCohere ยท 33B 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 17.2 tok/s Runs well 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 14.7 tok/s Runs welltight fit
Magistral Small 1.2 (24B)Mistral AI ยท 24B 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Runs well 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 20.3 tok/s Runs well
Qwen3-VL 8B InstructAlibaba ยท 8.8B 61Good quality (B) ยท 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 45.1 tok/s Runs great 61Good quality (B) ยท 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 38.5 tok/s Runs great
Dolphin 3.0 R1 Mistral 24BCognitive Computations ยท 24B 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 24.2 tok/s Runs well 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
InternVL3.5 8BShanghai AI Lab ยท 8.5B 61Good quality (B) ยท 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great 61Good quality (B) ยท 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 39.6 tok/s Runs great
Mixtral 8x7B InstructMistral AI ยท 47B (12.9B active) 47Fair quality (C) ยท 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 38.5 tok/s Runs welltight fit 35Basic quality (D) ยท 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs welltight fit
Solar Pro Preview 22BUpstage ยท 22B 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 25.8 tok/s Runs well 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 22.0 tok/s Runs well
OpenHands LM 32BAll Hands AI ยท 33B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 58Good quality (B) ยท 58 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
Phi-4 Reasoning Plus (14B)Microsoft ยท 15B 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 33.2 tok/s Runs great
Moonlight 16B-A3B InstructMoonshot AI ยท 16B (3.0B active) 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 70.7 tok/s Runs great 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 62.8 tok/s Runs great
Spark-X2.6 4BSparkLLM ยท 4.1B 66Good quality (B) ยท 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 70.6 tok/s Runs great 66Good quality (B) ยท 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 60.2 tok/s Runs great
Qwen2.5 14B Instruct 1MAlibaba ยท 15B 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Granite 4.1 8BIBM ยท 8.8B 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 45.0 tok/s Runs great 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great
Kolibri 1 (78B-A3.5B)Aleph Alpha ยท 78B (3.5B active) 76Great quality (A) ยท 76 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 70.1 tok/s Runs welloffloads to RAM 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-4.1V 9B ThinkingZhipu AI ยท 10B 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 39.0 tok/s Runs great
MiniCPM-V 4.5 8BOpenBMB ยท 8.7B 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 45.5 tok/s Runs great 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 38.8 tok/s Runs great
Gemma 3 12B ITGoogle ยท 12B 58Good quality (B) ยท 58 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 57Good quality (B) ยท 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great
Qwen3 8BAlibaba ยท 8.2B 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 48.3 tok/s Runs great 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 41.2 tok/s Runs great
Phi-4 Reasoning (14B)Microsoft ยท 15B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 33.2 tok/s Runs great
Nemotron 3.5 Bolt 74B-A9BNVIDIA ยท 75B (9.0B active) 77Great quality (A) ยท 77 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 34.2 tok/s Runs welloffloads to RAM 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Nemotron Nano 9B v2NVIDIA ยท 8.9B 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 44.5 tok/s Runs great 59Good quality (B) ยท 59 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
Qwen2.5 Coder 32B InstructAlibaba ยท 33B 63Good quality (B) ยท 63 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 56Good quality (B) ยท 56 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
UI-Mate 9BTencent ยท 9.4B 69Good quality (B) ยท 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 67Good quality (B) ยท 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great
UI-Venus 2 9BAnt Group (inclusionAI) ยท 9.4B 69Good quality (B) ยท 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 67Good quality (B) ยท 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great
Qwen3.5 4BAlibaba ยท 4.7B 62Good quality (B) ยท 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 62.3 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 53.1 tok/s Runs great
MiMo-VL 7B RLXiaomi ยท 8.3B 57Good quality (B) ยท 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 47.6 tok/s Runs great 57Good quality (B) ยท 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 40.6 tok/s Runs great
QwQ 32B PreviewAlibaba ยท 33B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 17.4 tok/s Runs well 55Good quality (B) ยท 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 19.2 tok/s Runs welltight fit
SenseNova 6.8 Code2VideoSenseTime ยท 27B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 20.8 tok/s Runs well 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 17.8 tok/s Runs well
Jamba 1.5 MiniAI21 Labs ยท 52B (12.0B active) 42Fair quality (C) ยท 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 51.2 tok/s Runs welltight fit 35Basic quality (D) ยท 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 55.1 tok/s Runs welltight fit
Qwen2 57B-A14B InstructAlibaba ยท 57B (14.0B active) 42Fair quality (C) ยท 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 45.0 tok/s Runs welltight fit 35Basic quality (D) ยท 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 41.1 tok/s Runs wellover GPU memory limit
Mistral Small 22B (2409)Mistral AI ยท 22B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 25.6 tok/s Runs well 51Fair quality (C) ยท 51 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 21.9 tok/s Runs well
Devstral Small 2505Mistral AI ยท 24B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 24.2 tok/s Runs well 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 20.6 tok/s Runs well
Ministral 3 8BMistral AI ยท 8.9B 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 37.8 tok/s Runs great
Qwen3-Next 80B-A3B ThinkingAlibaba ยท 81B (3.0B active) 72Great quality (A) ยท 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 73.8 tok/s Runs welloffloads to RAM 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniCPM5 2BOpenBMB ยท 2.5B 60Good quality (B) ยท 60 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 53.1 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 45.3 tok/s Runs great
TMax 4BAi2 ยท 4.2B 61Good quality (B) ยท 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 68.9 tok/s Runs great 61Good quality (B) ยท 61 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 58.8 tok/s Runs great
Agents-A1 4BInternScience ยท 4.5B 60Good quality (B) ยท 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 63.9 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 54.5 tok/s Runs great
Apriel Nemotron 15B ThinkerServiceNow ยท 15B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 38.1 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 32.5 tok/s Runs great
AceReason Nemotron 14BNVIDIA ยท 15B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
OpenReasoning Nemotron 14BNVIDIA ยท 15B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Qwen2.5 Coder 14B InstructAlibaba ยท 15B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Reka Flash 3.1 (21B)Reka AI ยท 21B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 27.3 tok/s Runs well 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 23.3 tok/s Runs well
DeepSeek Coder V2 Lite (16B)DeepSeek ยท 16B (2.4B active) 52Fair quality (C) ยท 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 82.8 tok/s Runs great 52Fair quality (C) ยท 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 74.1 tok/s Runs great
MiMo 7B RLXiaomi ยท 7.8B 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 50.5 tok/s Runs great 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great
Ministral 3 14B ReasoningMistral AI ยท 14B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 34.9 tok/s Runs great
DeepSeek V2 Lite ChatDeepSeek ยท 16B (2.4B active) 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 82.8 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 74.1 tok/s Runs great
Qwen3 Next 80B-A3B InstructAlibaba ยท 81B (3.0B active) 70Great quality (A) ยท 70 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 73.8 tok/s Runs welloffloads to RAM 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
InternVL3 8BShanghai AI Lab ยท 7.9B 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 49.8 tok/s Runs great 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs great
Falcon 3 10B InstructTII ยท 10B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 38.4 tok/s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 39.0 tok/s Runs great
Granite 4.0 H Tiny (7B-A1B)IBM ยท 6.9B (1.0B active) 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 84.4 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 76.1 tok/s Runs great
Fara 1.5 4BMicrosoft ยท 4.5B 58Good quality (B) ยท 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 63.9 tok/s Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 54.5 tok/s Runs great
DeepCoder 14B PreviewAgentica ยท 15B 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Mage-VL 4.7BMicrosoft ยท 4.7B 56Good quality (B) ยท 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 61.2 tok/s Runs great 56Good quality (B) ยท 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 52.2 tok/s Runs great
Codestral 22BMistral AI ยท 22B 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 25.6 tok/s Runs well 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 21.9 tok/s Runs well
Mistral Nemo 12B InstructMistral AI ยท 12B 50Fair quality (C) ยท 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great
Llama 3.1 Nemotron Nano 8B v1NVIDIA ยท 8.0B 53Fair quality (C) ยท 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 53Fair quality (C) ยท 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Yi 1.5 34B Chat01.AI ยท 34B 47Fair quality (C) ยท 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 16.6 tok/s Runs well 42Fair quality (C) ยท 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 18.3 tok/s Runs well
LFM2 8B-A1BLiquid AI ยท 8.3B (1.5B active) 38Basic quality (D) ยท 38 of 100: how capable the model is at F16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 64.1 tok/s Runs great 38Basic quality (D) ยท 38 of 100: how capable the model is at F16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 57.1 tok/s Runs great
Olmo 3 7B InstructAi2 ยท 7.3B 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 39.8 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 46.2 tok/s Runs great
MiniCPM4 8BOpenBMB ยท 8.2B 53Fair quality (C) ยท 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 48.3 tok/s Runs great 53Fair quality (C) ยท 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 41.2 tok/s Runs great
InternLM2.5 20B ChatShanghai AI Lab ยท 20B 47Fair quality (C) ยท 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 28.7 tok/s Runs well 47Fair quality (C) ยท 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 24.5 tok/s Runs well
GLM-4 9B 0414Zhipu AI ยท 9.4B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.1 tok/s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great
NorthCUA Nanobeipo-ai ยท 9.4B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great
Gemma 2 9B ITGoogle ยท 9.2B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.8 tok/s Runs great 48Fair quality (C) ยท 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 43.5 tok/s Runs great
DeepSeek R1 Distill Qwen 14BDeepSeek ยท 15B 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
Pixtral 12B (2409)Mistral AI ยท 12B 48Fair quality (C) ยท 48 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 38.0 tok/s Runs great 47Fair quality (C) ยท 47 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 39.2 tok/s Runs great
Command R 35B (v01)Cohere ยท 35B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 21.1 tok/s Runs welltight fit 37Basic quality (D) ยท 37 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Slowfar over GPU memory limit
JanusCoder 14BShanghai AI Lab ยท 15B 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 38.6 tok/s Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 32.9 tok/s Runs great
MiniCPM-o 2.6 8BOpenBMB ยท 8.7B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 45.6 tok/s Runs great 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great
Qwen3-VL 4B InstructAlibaba ยท 4.4B 54Fair quality (C) ยท 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 65.4 tok/s Runs great 54Fair quality (C) ยท 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 55.8 tok/s Runs great
Qwen2.5-VL 7B InstructAlibaba ยท 8.3B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 47.7 tok/s Runs great 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great
Hunyuan 7B InstructTencent ยท 7.5B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 52.7 tok/s Runs great 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 45.0 tok/s Runs great
DeepSeek R1 0528 Qwen3 8BDeepSeek ยท 8.2B 61Good quality (B) ยท 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 48.3 tok/s Runs great 61Good quality (B) ยท 61 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 41.2 tok/s Runs great
Qwen3 4B Instruct 2507Alibaba ยท 4.0B 54Fair quality (C) ยท 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 72.2 tok/s Runs great 54Fair quality (C) ยท 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 61.6 tok/s Runs great
Qwen2.5 7B InstructAlibaba ยท 7.6B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great
LLM4Decompile 22B V2LLM4Binary ยท 22B 55Good quality (B) ยท 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 25.6 tok/s Runs well 55Good quality (B) ยท 55 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 21.9 tok/s Runs well
InternVL2.5 8BShanghai AI Lab ยท 8.1B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 49.0 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great
Foundation-Sec 1.1 8B InstructCisco Foundation AI ยท 8.0B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
InternLM3 8B InstructShanghai AI Lab ยท 8.8B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 45.0 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 38.3 tok/s Runs great
LFM3 1.7BLiquid AI ยท 1.7B 56Good quality (B) ยท 56 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 77.3 tok/s Runs great 56Good quality (B) ยท 56 of 100: how capable the model is at F16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 65.9 tok/s Runs great
GLM-4 9B Chat 1MZhipu AI ยท 9.5B 47Fair quality (C) ยท 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great 46Fair quality (C) ยท 46 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.4 tok/s Runs great
OLMo 2 13B InstructAi2 ยท 14B 41Fair quality (C) ยท 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Runs great 41Fair quality (C) ยท 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 35.4 tok/s Runs great
Aya 23 35BCohere ยท 35B 42Fair quality (C) ยท 42 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 21.1 tok/s Runs welltight fit 35Basic quality (D) ยท 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 19.4 tok/s Slowfar over GPU memory limit
GLM-4 9B Chat (2024)Zhipu AI ยท 9.4B 47Fair quality (C) ยท 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 42.1 tok/s Runs great 46Fair quality (C) ยท 46 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great
GLiDE 9BFastino ยท 7.9B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 49.8 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs great
Qwen1.5 32B ChatAlibaba ยท 33B 41Fair quality (C) ยท 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 17.5 tok/s Runs well 36Basic quality (D) ยท 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 19.3 tok/s Runs welltight fit
Qwen2.5 7B Instruct 1MAlibaba ยท 7.6B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great
Falcon-H1 7B InstructTII ยท 7.6B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 52.1 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 44.4 tok/s Runs great
Phi-3 Medium 128K InstructMicrosoft ยท 14B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 40.8 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 34.8 tok/s Runs great
Hermes 3 (Llama 3.1 8B)Nous Research ยท 8.0B 47Fair quality (C) ยท 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 47Fair quality (C) ยท 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Llama 3.1 8B InstructMeta ยท 8.0B 47Fair quality (C) ยท 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 47Fair quality (C) ยท 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Falcon 3 7B InstructTII ยท 7.5B 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 38.9 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 45.2 tok/s Runs great
LLaVA 1.6 34BLLaVA ยท 35B 41Fair quality (C) ยท 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 16.4 tok/s Runs well 36Basic quality (D) ยท 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 18.1 tok/s Runs welltight fit
Gemma 3n E4BGoogle ยท 7.9B 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 72.6 tok/s Runs great 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 61.9 tok/s Runs great
Granite 4.2 3BIBM ยท 3.7B 52Fair quality (C) ยท 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 79.3 tok/s Runs great 52Fair quality (C) ยท 52 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 67.6 tok/s Runs great
GLM-Z1 9B (0414)Zhipu AI ยท 9.4B 57Good quality (B) ยท 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 42.1 tok/s Runs great 56Good quality (B) ยท 56 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great
Apertus v1.5 8BSwiss AI ยท 8.9B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 44.4 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 37.9 tok/s Runs great
MiniCPM-V 2.6 8BOpenBMB ยท 8.1B 47Fair quality (C) ยท 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 48.8 tok/s Runs great 47Fair quality (C) ยท 47 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great
Mage-VL-Cowork 4.7BMicrosoft ยท 4.7B 60Good quality (B) ยท 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 61.2 tok/s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 52.2 tok/s Runs great
Seed-Coder 8B ReasoningByteDance ยท 8.3B 57Good quality (B) ยท 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs great 57Good quality (B) ยท 57 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great
Qwen3 4BAlibaba ยท 4.0B 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 72.2 tok/s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 61.6 tok/s Runs great
Phi-3 Medium (14B)Microsoft ยท 14B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 40.8 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 34.8 tok/s Runs great
Granite 3.2 8B InstructIBM ยท 8.2B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 48.4 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 41.3 tok/s Runs great
Granite 3.3 8BIBM ยท 8.2B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 48.4 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 41.3 tok/s Runs great
Ministral 8B (2410)Mistral AI ยท 8.0B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.1 tok/s Runs great
Phi-3 Small 8K InstructMicrosoft ยท 7.4B 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 39.3 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 45.7 tok/s Runs great
DeepHermes 3 Llama 3 8B PreviewNous Research ยท 8.0B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Dolphin 3.0 Llama 3.1 8BCognitive Computations ยท 8.0B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Tรผlu 3 8BAi2 ยท 8.0B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Qwen3 Coder Next (80B-A3B)Alibaba ยท 80B (3.0B active) 72Great quality (A) ยท 72 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 75.9 tok/s Runs welloffloads to RAM 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen2.5-Omni 7BAlibaba ยท 11B 44Fair quality (C) ยท 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 43.9 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 45.3 tok/s Runs great
Olmo 3 7B ThinkAi2 ยท 7.3B 54Fair quality (C) ยท 54 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 39.8 tok/s Runs great 53Fair quality (C) ยท 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 46.2 tok/s Runs great
EXAONE 3.5 7.8BLG AI Research ยท 7.8B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 50.6 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great
CogVLM2 Llama 3 Chat 19BZhipu AI ยท 20B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 29.2 tok/s Runs well 36Basic quality (D) ยท 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 24.9 tok/s Runs well
Nemotron 3 Nano 4BNVIDIA ยท 4.0B 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 73.1 tok/s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 62.4 tok/s Runs great
Qwen1.5 14B ChatAlibaba ยท 14B 32Basic quality (D) ยท 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 40.2 tok/s Runs great 32Basic quality (D) ยท 32 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 34.3 tok/s Runs great
Seed-Coder 8B InstructByteDance ยท 8.3B 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs great 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 40.9 tok/s Runs great
InternLM2 Chat 20BShanghai AI Lab ยท 20B 39Basic quality (D) ยท 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 28.7 tok/s Runs well 39Basic quality (D) ยท 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 24.5 tok/s Runs well
Granite 3.1 8B InstructIBM ยท 8.2B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 48.4 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 41.3 tok/s Runs great
EXAONE Deep 7.8BLG AI Research ยท 7.8B 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 50.6 tok/s Runs great 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great
LFM3-VL 2BLiquid AI ยท 1.9B 50Fair quality (C) ยท 50 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 69.7 tok/s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at F16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 59.4 tok/s Runs great
Yi 34B Chat01.AI ยท 34B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 16.6 tok/s Runs well 33Basic quality (D) ยท 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 18.3 tok/s Runs well
InternVL2 8BShanghai AI Lab ยท 8.1B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 49.0 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great
Dolphin 2.9.4 Llama 3.1 8BCognitive Computations ยท 8.0B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Hermes 2 Pro Llama 3 8BNous Research ยท 8.0B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
OpenChat 3.6 8BOpenChat ยท 8.0B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Aya Vision 8BCohere ยท 8.6B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 45.8 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 39.1 tok/s Runs great
DeepSeek MoE 16B ChatDeepSeek ยท 16B (2.8B active) 26Basic quality (D) ยท 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 74.3 tok/s Runs great 25Basic quality (D) ยท 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 84.0 tok/s Runs great
Qwen2-VL 7B InstructAlibaba ยท 8.3B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 47.7 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great
Phi-4 Multimodal InstructMicrosoft ยท 5.6B 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 52.1 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 44.4 tok/s Runs great
Llama 3.2 11B Vision InstructMeta ยท 11B 40Fair quality (C) ยท 40 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 44.1 tok/s Runs great 39Basic quality (D) ยท 39 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 45.6 tok/s Runs great
Falcon-H1R 7BTII ยท 7.6B 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 52.1 tok/s Runs great 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 44.4 tok/s Runs great
Mistral-NeMo-Minitron 8B InstructNVIDIA ยท 8.4B 41Fair quality (C) ยท 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 47.0 tok/s Runs great 41Fair quality (C) ยท 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great
Command R7BCohere ยท 8.0B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Qwen2 7B InstructAlibaba ยท 7.6B 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great 43Fair quality (C) ยท 43 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great
Jamba Reasoning 3BAI21 Labs ยท 3.2B 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 41.8 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 77.4 tok/s Runs great
LFM2.5-VL 3BLiquid AI ยท 3.1B 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 42.9 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 79.3 tok/s Runs great
Qwen2.5 Coder 7B InstructAlibaba ยท 7.6B 54Fair quality (C) ยท 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great 54Fair quality (C) ยท 54 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great
Llama 3 8B InstructMeta ยท 8.0B 41Fair quality (C) ยท 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 41Fair quality (C) ยท 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Qwen3 4B Thinking 2507Alibaba ยท 4.0B 56Good quality (B) ยท 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 72.2 tok/s Runs great 56Good quality (B) ยท 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 61.6 tok/s Runs great
Gemma 3n E2BGoogle ยท 5.4B 36Basic quality (D) ยท 36 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 66.9 tok/s Runs great 36Basic quality (D) ยท 36 of 100: how capable the model is at F16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 57.0 tok/s Runs great
North Micro VisionCohere ยท 2.5B 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 53.9 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 46.0 tok/s Runs great
Llama 3.1 Nemotron Nano 4B v1.1NVIDIA ยท 4.5B 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 64.4 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 54.9 tok/s Runs great
OpenReasoning Nemotron 7BNVIDIA ยท 7.6B 53Fair quality (C) ยท 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great 53Fair quality (C) ยท 53 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great
LFM2.5 2.6BLiquid AI ยท 2.7B 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 49.5 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 42.3 tok/s Runs great
Granite 4.1 3BIBM ยท 3.4B 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 85.4 tok/s Runs great 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 72.8 tok/s Runs great
SmolLM3 3BHugging Face ยท 3.1B 42Fair quality (C) ยท 42 of 100: how capable the model is at F16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 43.4 tok/s Runs great 42Fair quality (C) ยท 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 80.4 tok/s Runs great
Command R7B ArabicCohere ยท 8.0B 41Fair quality (C) ยท 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 41Fair quality (C) ยท 41 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Gemma 3 4B ITGoogle ยท 4.3B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 67.5 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 57.6 tok/s Runs great
Foundation-Sec 8B ReasoningCisco Foundation AI ยท 8.0B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 51Fair quality (C) ยท 51 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Nous Hermes 2 SOLAR 10.7BNous Research ยท 11B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 43.9 tok/s Runs great 38Basic quality (D) ยท 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 45.3 tok/s Runs great
SOLAR 10.7B InstructUpstage ยท 11B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 43.9 tok/s Runs great 38Basic quality (D) ยท 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 45.3 tok/s Runs great
Phi-4 Mini (3.8B)Microsoft ยท 3.8B 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 75.6 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 64.5 tok/s Runs great
Mistral 7B Instruct v0.3Mistral AI ยท 7.3B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 46.5 tok/s Runs great
Ministral 3 3BMistral AI ยท 3.9B 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 75.4 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 64.3 tok/s Runs great
Apertus 8B InstructSwiss AI ยท 8.1B 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 49.1 tok/s Runs great 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 41.9 tok/s Runs great
Aya Expanse 8BCohere ยท 8.0B 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
OLMo 2 7B InstructAi2 ยท 7.3B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 39.8 tok/s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 46.2 tok/s Runs great
Jamba2 3BAI21 Labs ยท 3.0B 42Fair quality (C) ยท 42 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 44.1 tok/s Runs great 42Fair quality (C) ยท 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 81.7 tok/s Runs great
MedGemma 4B ITGoogle ยท 4.3B 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 67.5 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 57.6 tok/s Runs great
Phi-3.5 Mini InstructMicrosoft ยท 3.8B 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 76.0 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 64.8 tok/s Runs great
InternLM2.5 7B ChatShanghai AI Lab ยท 7.7B 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 51.1 tok/s Runs great 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 43.6 tok/s Runs great
Zamba2 7B InstructZyphra ยท 7.4B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 39.4 tok/s Runs great 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 45.8 tok/s Runs great
Granite 3.0 8B InstructIBM ยท 8.2B 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 48.4 tok/s Runs great 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 41.3 tok/s Runs great
Hunyuan 4B InstructTencent ยท 4.2B 42Fair quality (C) ยท 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 68.8 tok/s Runs great 42Fair quality (C) ยท 42 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 58.7 tok/s Runs great
Qwen1.5 MoE A2.7B ChatAlibaba ยท 14B (2.7B active) 24Basic quality (D) ยท 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 76.3 tok/s Runs great 24Basic quality (D) ยท 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 68.0 tok/s Runs great
DeepSeek R1 Distill Llama 8BDeepSeek ยท 8.0B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Falcon3 Mamba 7B InstructTII ยท 7.3B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great
Phi-3.5 Vision InstructMicrosoft ยท 4.2B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 69.9 tok/s Runs great 38Basic quality (D) ยท 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 59.6 tok/s Runs great
Fara 7BMicrosoft ยท 8.3B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 47.7 tok/s Runs great 49Fair quality (C) ยท 49 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great
Hermes 2 Pro Mistral 7BNous Research ยท 7.2B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Starling LM 7B BetaNexusflow ยท 7.2B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Dolphin 2.9 Llama 3 8BCognitive Computations ยท 8.0B 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Ling Flash 2.0 (100B-A6B)Ant Group (inclusionAI) ยท 103B (6.1B active) 53Fair quality (C) ยท 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs welloffloads to RAM 53Fair quality (C) ยท 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
OLMoE 1B-7B InstructAi2 ยท 6.9B (1.3B active) 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 71.0 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 63.4 tok/s Runs great
Hunyuan A13B (80B-A13B)Tencent ยท 80B (13.0B active) 62Good quality (B) ยท 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 20.3 tok/s Runs welloffloads to RAM 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
OpenChat 3.5 (0106)OpenChat ยท 7.2B 35Basic quality (D) ยท 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 34Basic quality (D) ยท 34 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
DeepSeek Coder 33B InstructDeepSeek ยท 33B 41Fair quality (C) ยท 41 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 17.1 tok/s Runs well 36Basic quality (D) ยท 36 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 18.9 tok/s Runs welltight fit
Qianfan-OCR 4BBaidu ยท 4.7B 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 61.2 tok/s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 52.2 tok/s Runs great
AFM 4.5BArcee AI ยท 4.6B 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 62.8 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 53.6 tok/s Runs great
EXAONE 3.0 7.8B InstructLG AI Research ยท 7.8B 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 50.6 tok/s Runs great 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great
Granite 4.0 H MicroIBM ยท 3.2B 38Basic quality (D) ยท 38 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 41.9 tok/s Runs great 38Basic quality (D) ยท 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 77.6 tok/s Runs great
Mistral 7B Instruct v0.2Mistral AI ยท 7.2B 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Nous Hermes 2 Mistral 7B DPONous Research ยท 7.2B 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
OpenChat 3.5 (1210)OpenChat ยท 7.2B 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Yi 1.5 9B Chat01.AI ยท 8.8B 35Basic quality (D) ยท 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 44.8 tok/s Runs great 34Basic quality (D) ยท 34 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 45.5 tok/s Runs great
Phi-3 Mini 128K InstructMicrosoft ยท 3.8B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 76.0 tok/s Runs great 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 64.8 tok/s Runs great
Qwen3.5 2BAlibaba ยท 2.3B 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 58.9 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 50.3 tok/s Runs great
CodeGeeX4 All 9BZhipu AI ยท 9.4B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 42.1 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great
DeepSeek Prover V2 7BDeepSeek ยท 6.9B 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 48.8 tok/s Runs great
Granite 4.0 Micro (3B)IBM ยท 3.4B 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 85.4 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 72.8 tok/s Runs great
Llama 3.2 3B InstructMeta ยท 3.2B 35Basic quality (D) ยท 35 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 77.1 tok/s Runs great
Falcon-H1 3B InstructTII ยท 3.2B 36Basic quality (D) ยท 36 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 42.5 tok/s Runs great 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 78.6 tok/s Runs great
RecurrentGemma 9B ITGoogle ยท 9.6B 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 41.1 tok/s Runs great 32Basic quality (D) ยท 32 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great
Chandra OCR 2Datalab ยท 5.3B 48Fair quality (C) ยท 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 54.8 tok/s Runs great 48Fair quality (C) ยท 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 46.7 tok/s Runs great
Phi-3 Vision 128K InstructMicrosoft ยท 4.2B 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 69.9 tok/s Runs great 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 59.6 tok/s Runs great
TMax 2BAi2 ยท 1.9B 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 71.1 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 60.7 tok/s Runs great
OpenHermes 2.5 Mistral 7BTeknium ยท 7.2B 32Basic quality (D) ยท 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 31Basic quality (D) ยท 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Starling LM 7B AlphaBerkeley ยท 7.2B 32Basic quality (D) ยท 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 31Basic quality (D) ยท 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Aya 23 8BCohere ยท 8.0B 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 49.3 tok/s Runs great 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great
Voxtral Mini 3BMistral AI ยท 4.7B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 62.0 tok/s Runs great 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 52.9 tok/s Runs great
MiniCPM3 4BOpenBMB ยท 4.1B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 71.3 tok/s Runs great 38Basic quality (D) ยท 38 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 60.8 tok/s Runs great
S5 nanoMaritaca AI ยท 4.7B 37Basic quality (D) ยท 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 62.3 tok/s Runs great 37Basic quality (D) ยท 37 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 53.1 tok/s Runs great
DeepSeek R1 Distill Qwen 7BDeepSeek ยท 7.6B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great
MiniCPM-V 4.7 1BOpenBMB ยท 1.6B 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 86.3 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 73.6 tok/s Runs great
Phi-3 Mini 4K InstructMicrosoft ยท 3.8B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 76.0 tok/s Runs great 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 64.8 tok/s Runs great
Granite Code 34B InstructIBM ยท 34B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 16.9 tok/s Runs well 33Basic quality (D) ยท 33 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 18.6 tok/s Runs well
LightOnOCR-3 4BLightOn ยท 4.5B 48Fair quality (C) ยท 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 63.9 tok/s Runs great 48Fair quality (C) ยท 48 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 54.5 tok/s Runs great
Gemma 1.1 7B ITGoogle ยท 8.5B 27Basic quality (D) ยท 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 46.3 tok/s Runs great 27Basic quality (D) ยท 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 39.5 tok/s Runs great
Orca 2 13BMicrosoft ยท 13B 26Basic quality (D) ยท 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 43.8 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
Unlimited OCR 2 (3B-A0.6B)Baidu ยท 3.3B (0.6B active) 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 113 tok/s Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at F16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 104 tok/s Runs great
Qwen2.5 3B InstructAlibaba ยท 3.1B 34Basic quality (D) ยท 34 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 43.3 tok/s Runs great 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 80.1 tok/s Runs great
Qwen3-VL 2B InstructAlibaba ยท 2.1B 36Basic quality (D) ยท 36 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 62.8 tok/s Runs great 36Basic quality (D) ยท 36 of 100: how capable the model is at F16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 53.6 tok/s Runs great
WeVisOne-VL 4BTencent ยท 4.8B 45Fair quality (C) ยท 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 60.1 tok/s Runs great 45Fair quality (C) ยท 45 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 51.2 tok/s Runs great
Mistral 7B Instruct v0.1Mistral AI ยท 7.2B 30Basic quality (D) ยท 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 29Basic quality (D) ยท 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Hermes 3 Llama 3.2 3BNous Research ยท 3.2B 32Basic quality (D) ยท 32 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 41.7 tok/s Runs great 32Basic quality (D) ยท 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 77.1 tok/s Runs great
InternLM2 Math Plus 20BShanghai AI Lab ยท 20B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 28.7 tok/s Runs well 38Basic quality (D) ยท 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 24.5 tok/s Runs well
Baichuan 2 13B ChatBaichuan ยท 14B 24Basic quality (D) ยท 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great 24Basic quality (D) ยท 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 35.0 tok/s Runs great
Phi-4 Mini Reasoning (3.8B)Microsoft ยท 3.8B 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 75.6 tok/s Runs great 46Fair quality (C) ยท 46 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 64.5 tok/s Runs great
Code Llama 34B InstructMeta ยท 34B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 16.9 tok/s Runs well 31Basic quality (D) ยท 31 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 18.6 tok/s Runs welltight fit
Gemma 7B ITGoogle ยท 8.5B 25Basic quality (D) ยท 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 46.3 tok/s Runs great 25Basic quality (D) ยท 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 39.5 tok/s Runs great
Llama 2 13B ChatMeta ยท 13B 24Basic quality (D) ยท 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 43.8 tok/s Runs great 24Basic quality (D) ยท 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
Qwen1.5 7B ChatAlibaba ยท 7.7B 25Basic quality (D) ยท 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 51.2 tok/s Runs great 25Basic quality (D) ยท 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 43.7 tok/s Runs great
LFM2 2.6BLiquid AI ยท 2.6B 34Basic quality (D) ยท 34 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 52.0 tok/s Runs great 34Basic quality (D) ยท 34 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 44.4 tok/s Runs great
Falcon Mamba 7B InstructTII ยท 7.3B 30Basic quality (D) ยท 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 39.9 tok/s Runs great 29Basic quality (D) ยท 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 46.4 tok/s Runs great
Neural Chat 7B v3.3Intel ยท 7.2B 28Basic quality (D) ยท 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 27Basic quality (D) ยท 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
DeepSeek LLM 67B ChatDeepSeek ยท 67B 28Basic quality (D) ยท 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 13.9 tok/s Runs welltight fit 28Basic quality (D) ยท 28 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
StableLM 2 12B ChatStability AI ยท 12B 27Basic quality (D) ยท 27 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 38.8 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great
StarCoder2 15B InstructBigCode ยท 16B 38Basic quality (D) ยท 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 35.7 tok/s Runs great 38Basic quality (D) ยท 38 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 30.5 tok/s Runs great
Falcon 3 3B InstructTII ยท 3.2B 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 41.4 tok/s Runs great 30Basic quality (D) ยท 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 76.6 tok/s Runs great
Yi Coder 9B Chat01.AI ยท 8.8B 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 44.8 tok/s Runs great 39Basic quality (D) ยท 39 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 38.2 tok/s Runs great
Raon 1 nanoKRAFTON ยท 3.5B 35Basic quality (D) ยท 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 82.2 tok/s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 70.1 tok/s Runs great
DeepSeek Coder 6.7B InstructDeepSeek ยท 6.7B 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great
Ornith 1.5 122B-A10BDeepReinforce ยท 125B (10.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 25.1 tok/s Slowheavy RAM offload 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Code Llama 13B InstructMeta ยท 13B 28Basic quality (D) ยท 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 43.8 tok/s Runs great 28Basic quality (D) ยท 28 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
Qwen2.5-VL 3B InstructAlibaba ยท 3.8B 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 77.4 tok/s Runs great 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 66.0 tok/s Runs great
Qwen3.5 122B-A10BAlibaba ยท 125B (10.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 25.1 tok/s Slowheavy RAM offload 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Unlimited OCR (3B-A0.6B)Baidu ยท 3.3B (0.6B active) 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 113 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 104 tok/s Runs great
Zephyr 7B BetaHugging Face ยท 7.2B 26Basic quality (D) ยท 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 25Basic quality (D) ยท 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Ring Flash 2.0 (100B-A6B)Ant Group (inclusionAI) ยท 103B (6.1B active) 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs welloffloads to RAM 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen2.5-Omni 3BAlibaba ยท 5.5B 30Basic quality (D) ยท 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 52.4 tok/s Runs great 30Basic quality (D) ยท 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 44.7 tok/s Runs great
Nemotron 3.5 Super VL 120B-A12BNVIDIA ยท 124B (12.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 21.4 tok/s Slowheavy RAM offload 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
CodeGemma 7BGoogle ยท 8.5B 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 46.3 tok/s Runs great 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 39.5 tok/s Runs great
MiniCPM5 1BOpenBMB ยท 1.1B 36Basic quality (D) ยท 36 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 124 tok/s Runs great 36Basic quality (D) ยท 36 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 106 tok/s Runs great
Mistral Small 4 (119B-A6B)Mistral AI ยท 119B (6.0B active) 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 41.6 tok/s Slowheavy RAM offload 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
LLM4Decompile 9B V2LLM4Binary ยท 8.8B 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 44.8 tok/s Runs great 37Basic quality (D) ยท 37 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 38.2 tok/s Runs great
LLaVA 1.6 Mistral 7BLLaVA ยท 7.6B 27Basic quality (D) ยท 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 52.3 tok/s Runs great 27Basic quality (D) ยท 27 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 44.6 tok/s Runs great
LLaVA 1.5 13BLLaVA ยท 13B 20Basic quality (D) ยท 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 42.7 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 36.4 tok/s Runs great
LLM4Decompile 6.7B V2LLM4Binary ยท 6.7B 32Basic quality (D) ยท 32 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 31Basic quality (D) ยท 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great
Granite 3.2 2B InstructIBM ยท 2.5B 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 52.9 tok/s Runs great 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 45.1 tok/s Runs great
Granite 3.3 2BIBM ยท 2.5B 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 52.9 tok/s Runs great 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 45.1 tok/s Runs great
WizardLM 13B v1.2WizardLM ยท 13B 20Basic quality (D) ยท 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 43.8 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
Llama 2 70B ChatMeta ยท 69B 25Basic quality (D) ยท 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 13.6 tok/s Runs welltight fit 25Basic quality (D) ยท 25 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Ling 2.6 Flash (104B-A7.4B)Ant Group (inclusionAI) ยท 107B (7.4B active) 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 39.2 tok/s Slowheavy RAM offload 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
EXAONE 3.5 2.4BLG AI Research ยท 2.4B 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 55.5 tok/s Runs great 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 47.3 tok/s Runs great
Yi 1.5 6B Chat01.AI ยท 6.1B 28Basic quality (D) ยท 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs great 28Basic quality (D) ยท 28 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 40.8 tok/s Runs great
Qwen3 1.7BAlibaba ยท 2.0B 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 65.9 tok/s Runs great 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 56.2 tok/s Runs great
Granite Code 20B InstructIBM ยท 20B 34Basic quality (D) ยท 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 28.4 tok/s Runs well 34Basic quality (D) ยท 34 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 24.2 tok/s Runs well
Codestral Mamba 7BMistral AI ยท 7.3B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 39.8 tok/s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 46.3 tok/s Runs great
SQLCoder 7B v2Defog ยท 6.7B 30Basic quality (D) ยท 30 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 29Basic quality (D) ยท 29 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great
Granite 3.1 2B InstructIBM ยท 2.5B 28Basic quality (D) ยท 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 52.9 tok/s Runs great 28Basic quality (D) ยท 28 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 45.1 tok/s Runs great
Gemma 2 2BGoogle ยท 2.6B 28Basic quality (D) ยท 28 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 51.2 tok/s Runs great 28Basic quality (D) ยท 28 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 43.7 tok/s Runs great
Vicuna 13B v1.5LMSYS ยท 13B 19Basic quality (D) ยท 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 43.8 tok/s Runs great 19Basic quality (D) ยท 19 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 37.3 tok/s Runs great
Mathstral 7BMistral AI ยท 7.3B 34Basic quality (D) ยท 34 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 40.0 tok/s Runs great 33Basic quality (D) ยท 33 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 46.5 tok/s Runs great
Orca 2 7BMicrosoft ยท 6.7B 22Basic quality (D) ยท 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 21Basic quality (D) ยท 21 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great
Granite Vision 3.2 2BIBM ยท 3.0B 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 44.9 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 83.1 tok/s Runs great
WeVisOne VL 2BTencent ยท 2.4B 27Basic quality (D) ยท 27 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 54.8 tok/s Runs great 27Basic quality (D) ยท 27 of 100: how capable the model is at F16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 46.8 tok/s Runs great
LFM2.5 1.2B InstructLiquid AI ยท 1.2B 32Basic quality (D) ยท 32 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 114 tok/s Runs great 32Basic quality (D) ยท 32 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 97.5 tok/s Runs great
Nemotron 3 Super 120B-A12BNVIDIA ยท 124B (12.0B active) 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 21.5 tok/s Slowheavy RAM offload 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen2.5 Coder 3B InstructAlibaba ยท 3.1B 36Basic quality (D) ยท 36 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 43.3 tok/s Runs great 36Basic quality (D) ยท 36 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 80.1 tok/s Runs great
Qwen2.5 72B InstructAlibaba ยท 73B 61Good quality (B) ยท 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 4.2 tok/s Slowoffloads to RAM 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniCPM-OCR 1.0 1.3BOpenBMB ยท 1.3B 42Fair quality (C) ยท 42 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 103 tok/s Runs great 42Fair quality (C) ยท 42 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 87.8 tok/s Runs great
Qwen2.5-VL 72B InstructAlibaba ยท 73B 61Good quality (B) ยท 61 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 4.1 tok/s Slowoffloads to RAM 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
InternVL3 78BShanghai AI Lab ยท 78B 62Good quality (B) ยท 62 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.4 tok/s Slowoffloads to RAM 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Granite 3.0 2B InstructIBM ยท 2.6B 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 50.9 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 43.4 tok/s Runs great
TeleOCRXingChen-AGI ยท 1.4B 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 94.2 tok/s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 80.3 tok/s Runs great
Qwen2.5-Math 7B InstructAlibaba ยท 7.6B 35Basic quality (D) ยท 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 51.9 tok/s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 44.3 tok/s Runs great
EXAONE 4.0 1.2BLG AI Research ยท 1.3B 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 105 tok/s Runs great 30Basic quality (D) ยท 30 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 89.1 tok/s Runs great
Llama 2 7B ChatMeta ยท 6.7B 20Basic quality (D) ยท 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 19Basic quality (D) ยท 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great
Leanstral 119B-A6BMistral AI ยท 119B (6.5B active) 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 38.7 tok/s Slowheavy RAM offload 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
PaddleOCR-VL 1.6 0.9BBaidu ยท 1.0B 42Fair quality (C) ยท 42 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 139 tok/s Runs great 42Fair quality (C) ยท 42 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 119 tok/s Runs great
StarCoder2 15B (base)BigCode ยท 16B 30Basic quality (D) ยท 30 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 35.7 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 39.4 tok/s Runs great
SmolVLM2 2.2B InstructHugging Face ยท 2.3B 24Basic quality (D) ยท 24 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 59.4 tok/s Runs great 24Basic quality (D) ยท 24 of 100: how capable the model is at F16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 50.7 tok/s Runs great
GLM-4.6VZhipu AI ยท 108B (12.0B active) 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Slowheavy RAM offload 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Code Llama 70B InstructMeta ยท 69B 30Basic quality (D) ยท 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 13.6 tok/s Runs welltight fit 30Basic quality (D) ยท 30 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Baichuan 2 7B ChatBaichuan ยท 7.5B 19Basic quality (D) ยท 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 52.7 tok/s Runs great 19Basic quality (D) ยท 19 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 44.9 tok/s Runs great
Granite Code 8B Instruct (128K)IBM ยท 8.1B 31Basic quality (D) ยท 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 49.1 tok/s Runs great 31Basic quality (D) ยท 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 41.9 tok/s Runs great
DeepSeek LLM 7B ChatDeepSeek ยท 6.9B 18Basic quality (D) ยท 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 42.0 tok/s Runs great 17Basic quality (D) ยท 17 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 48.8 tok/s Runs great
Granite 4.0 H 1BIBM ยท 1.5B 28Basic quality (D) ยท 28 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 91.6 tok/s Runs great 28Basic quality (D) ยท 28 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 78.1 tok/s Runs great
InternLM2 Math Plus 7BShanghai AI Lab ยท 7.7B 31Basic quality (D) ยท 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 51.1 tok/s Runs great 31Basic quality (D) ยท 31 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 43.6 tok/s Runs great
Hermes 4 70BNous Research ยท 71B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs welloffloads to RAM 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Llama 3.3 70B InstructMeta ยท 71B 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs welloffloads to RAM 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Solar Open 100BUpstage ยท 103B (12.0B active) 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 25.1 tok/s Slowheavy RAM offload 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Groot 65B A6B 261023GrootTeam ยท 118B (6.0B active) 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 41.9 tok/s Slowheavy RAM offload 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
SmolVLM 2.2B InstructHugging Face ยท 2.3B 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 59.4 tok/s Runs great 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 50.7 tok/s Runs great
Qwen2-VL 2B InstructAlibaba ยท 2.2B 24Basic quality (D) ยท 24 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 60.5 tok/s Runs great 24Basic quality (D) ยท 24 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 51.6 tok/s Runs great
GLM-4.5VZhipu AI ยท 108B (12.0B active) 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 23.7 tok/s Slowheavy RAM offload 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Granite 3.1 3B-A800M InstructIBM ยท 3.3B (0.8B active) 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 96.6 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 87.8 tok/s Runs great
LLaVA 1.5 7BLLaVA ยท 7.1B 16Basic quality (D) ยท 16 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 41.1 tok/s Runs great 15Basic quality (D) ยท 15 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 47.8 tok/s Runs great
Falcon-H1 1.5B InstructTII ยท 1.6B 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 86.3 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 73.6 tok/s Runs great
Sarvam 105BSarvam AI ยท 106B (10.0B active) 52Fair quality (C) ยท 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 30.1 tok/s Slowheavy RAM offload 52Fair quality (C) ยท 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Code Llama 7B InstructMeta ยท 6.7B 24Basic quality (D) ยท 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 23Basic quality (D) ยท 23 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great
Zamba2 2.7B InstructZyphra ยท 2.7B 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 50.3 tok/s Runs great 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 42.9 tok/s Runs great
QVQ 72B PreviewAlibaba ยท 73B 57Good quality (B) ยท 57 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 4.1 tok/s Slowoffloads to RAM 47Fair quality (C) ยท 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek R1 Distill Llama 70BDeepSeek ยท 71B 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs welloffloads to RAM 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Tรผlu 3 70BAi2 ยท 71B 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs welloffloads to RAM 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Yi 6B Chat01.AI ยท 6.1B 20Basic quality (D) ยท 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 47.9 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 40.8 tok/s Runs great
Falcon 40B InstructTII ยท 42B 11Basic quality (D) ยท 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 22.4 tok/s Runs well 11Basic quality (D) ยท 11 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 19.1 tok/s Runs well
Kimi Dev 72BMoonshot AI ยท 73B 67Good quality (B) ยท 67 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 4.2 tok/s Slowoffloads to RAM 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Nemotron Mini 4B InstructNVIDIA ยท 4.2B 22Basic quality (D) ยท 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 69.3 tok/s Runs great 22Basic quality (D) ยท 22 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 59.1 tok/s Runs great
LFM2 1.2BLiquid AI ยท 1.2B 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 114 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 97.5 tok/s Runs great
GLM-4.5 Air (106B-A12B)Zhipu AI ยท 110B (12.0B active) 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 23.0 tok/s Slowheavy RAM offload 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Apertus v1.5 70BSwiss AI ยท 72B 55Good quality (B) ยท 55 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 4.3 tok/s Slowoffloads to RAM 46Fair quality (C) ยท 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Llama 3.1 70B InstructMeta ยท 71B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs welloffloads to RAM 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen2.5 1.5B InstructAlibaba ยท 1.5B 24Basic quality (D) ยท 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 86.9 tok/s Runs great 24Basic quality (D) ยท 24 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 74.1 tok/s Runs great
Granite 3.0 3B-A800M InstructIBM ยท 3.4B (0.8B active) 18Basic quality (D) ยท 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 96.6 tok/s Runs great 18Basic quality (D) ยท 18 of 100: how capable the model is at F16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 87.8 tok/s Runs great
LFM2-VL 1.6BLiquid AI ยท 1.6B 24Basic quality (D) ยท 24 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 84.7 tok/s Runs great 24Basic quality (D) ยท 24 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 72.2 tok/s Runs great
Qwen3.5 0.8BAlibaba ยท 0.9B 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 154 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 131 tok/s Runs great
WizardMath 7B v1.1WizardLM ยท 7.2B 26Basic quality (D) ยท 26 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 40.1 tok/s Runs great 25Basic quality (D) ยท 25 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 46.6 tok/s Runs great
Vicuna 7B v1.5LMSYS ยท 6.7B 14Basic quality (D) ยท 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 43.1 tok/s Runs great 13Basic quality (D) ยท 13 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 50.1 tok/s Runs great
Hunyuan 1.8B InstructTencent ยท 1.8B 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 74.7 tok/s Runs great 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 63.7 tok/s Runs great
Laguna S 2.1 (118B-A8B)Poolside ยท 118B (8.0B active) 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 31.2 tok/s Slowheavy RAM offload 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
SmolLM2 1.7B InstructHugging Face ยท 1.7B 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 78.2 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 66.7 tok/s Runs great
Hermes 3 (Llama 3.1 70B)Nous Research ยท 71B 47Fair quality (C) ยท 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs welloffloads to RAM 47Fair quality (C) ยท 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen2 72B InstructAlibaba ยท 73B 53Fair quality (C) ยท 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 4.2 tok/s Slowoffloads to RAM 44Fair quality (C) ยท 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
StarCoder 15BBigCode ยท 16B 24Basic quality (D) ยท 24 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 36.0 tok/s Runs great 21Basic quality (D) ยท 21 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 39.7 tok/s Runs great
Qwen2-VL 72B InstructAlibaba ยท 73B 53Fair quality (C) ยท 53 of 100: how capable the model is at Q3_K_M, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 4.1 tok/s Slowoffloads to RAM 44Fair quality (C) ยท 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
OLMo 7B InstructAi2 ยท 6.9B 14Basic quality (D) ยท 14 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 42.1 tok/s Runs great 13Basic quality (D) ยท 13 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 49.0 tok/s Runs great
StableLM Zephyr 3BStability AI ยท 2.8B 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 47.8 tok/s Runs great 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great
Llama 3.1 Nemotron 70B InstructNVIDIA ยท 71B 46Fair quality (C) ยท 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs welloffloads to RAM 46Fair quality (C) ยท 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Llama 3.2 1B InstructMeta ยท 1.2B 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 108 tok/s Runs great 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 92.0 tok/s Runs great
Phi-2 2.7BMicrosoft ยท 2.8B 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 48.1 tok/s Runs great 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 41.0 tok/s Runs great
StarCoder2 7BBigCode ยท 7.2B 24Basic quality (D) ยท 24 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 40.5 tok/s Runs great 23Basic quality (D) ยท 23 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 47.1 tok/s Runs great
InternLM2.5 1.8B ChatShanghai AI Lab ยท 1.9B 18Basic quality (D) ยท 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 70.8 tok/s Runs great 18Basic quality (D) ยท 18 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 60.4 tok/s Runs great
Gemma 3 1BGoogle ยท 1.0B 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 134 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 114 tok/s Runs great
InternLM 7BShanghai AI Lab ยท 7.3B 11Basic quality (D) ยท 11 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 54.0 tok/s Runs great 11Basic quality (D) ยท 11 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 46.1 tok/s Runs great
GLiDE 2BFastino ยท 1.9B 17Basic quality (D) ยท 17 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 71.1 tok/s Runs great 17Basic quality (D) ยท 17 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 60.7 tok/s Runs great
Apertus 70B InstructSwiss AI ยท 71B 42Fair quality (C) ยท 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.7 tok/s Runs welloffloads to RAM 42Fair quality (C) ยท 42 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Falcon 3 1B InstructTII ยท 1.7B 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 80.1 tok/s Runs great 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 68.3 tok/s Runs great
Stable Code Instruct 3BStability AI ยท 2.8B 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 47.8 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 40.7 tok/s Runs great
Llama 4 Scout 17B-16EMeta ยท 109B (17.0B active) 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 16.9 tok/s Slowheavy RAM offload 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Spark X2.6 1.7BSparkLLM ยท 1.7B 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 78.2 tok/s Runs great 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 66.7 tok/s Runs great
Qwen2 1.5B InstructAlibaba ยท 1.5B 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 86.9 tok/s Runs great 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 74.1 tok/s Runs great
Llama 3 70B InstructMeta ยท 71B 40Fair quality (C) ยท 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 10.8 tok/s Runs welloffloads to RAM 40Fair quality (C) ยท 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepScaleR 1.5B PreviewAgentica ยท 1.8B 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 75.1 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 64.1 tok/s Runs great
LFM2 700MLiquid AI ยท 0.7B 18Basic quality (D) ยท 18 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 181 tok/s Runs great 18Basic quality (D) ยท 18 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 154 tok/s Runs great
Gemma 2B ITGoogle ยท 2.5B 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 53.3 tok/s Runs great 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 45.5 tok/s Runs great
Qwen2.5 Coder 1.5B InstructAlibaba ยท 1.5B 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 86.9 tok/s Runs great 26Basic quality (D) ยท 26 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 74.1 tok/s Runs great
Llama 3.2 90B Vision InstructMeta ยท 89B 46Fair quality (C) ยท 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 4.2 tok/s Slowoffloads to RAM 46Fair quality (C) ยท 46 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
SmolLM 1.7B InstructHugging Face ยท 1.7B 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 78.2 tok/s Runs great 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 66.7 tok/s Runs great
OLMo 2 1B InstructAi2 ยท 1.5B 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 90.4 tok/s Runs great 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 77.1 tok/s Runs great
Granite Code 3B Instruct (128K)IBM ยท 3.5B 20Basic quality (D) ยท 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 83.4 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 71.1 tok/s Runs great
Falcon 7B InstructTII ยท 7.2B 6Basic quality (D) ยท 6 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 40.2 tok/s Runs great 5Basic quality (D) ยท 5 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 46.7 tok/s Runs great
Qwen3 0.6BAlibaba ยท 0.8B 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 178 tok/s Runs great 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 152 tok/s Runs great
StarCoder2 3BBigCode ยท 3.0B 18Basic quality (D) ยท 18 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 44.1 tok/s Runs great 18Basic quality (D) ยท 18 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 81.7 tok/s Runs great
DeepSeek R1 Distill Qwen 1.5BDeepSeek ยท 1.8B 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 75.1 tok/s Runs great 22Basic quality (D) ยท 22 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 64.1 tok/s Runs great
StableLM 2 1.6B ChatStability AI ยท 1.6B 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 81.6 tok/s Runs great 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 69.6 tok/s Runs great
StableLM 2 Zephyr 1.6BStability AI ยท 1.6B 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 81.6 tok/s Runs great 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 69.6 tok/s Runs great
d1 2BLiquid AI ยท 1.9B 21Basic quality (D) ยท 21 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 69.7 tok/s Runs great 21Basic quality (D) ยท 21 of 100: how capable the model is at F16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 59.4 tok/s Runs great
Falcon-H1 0.5B InstructTII ยท 0.5B 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 257 tok/s Runs great 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 219 tok/s Runs great
LFM2.5 350MLiquid AI ยท 0.4B 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 382 tok/s Runs great 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 326 tok/s Runs great
Command A (111B)Cohere ยท 111B 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen2.5-Math 1.5B InstructAlibaba ยท 1.5B 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 86.9 tok/s Runs great 20Basic quality (D) ยท 20 of 100: how capable the model is at F16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 74.1 tok/s Runs great
LFM2-VL 450MLiquid AI ยท 0.5B 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 297 tok/s Runs great 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 254 tok/s Runs great
GLiDE 0.8BFastino ยท 0.8B 11Basic quality (D) ยท 11 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 178 tok/s Runs great 11Basic quality (D) ยท 11 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 152 tok/s Runs great
LFM2 350MLiquid AI ยท 0.4B 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 382 tok/s Runs great 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 326 tok/s Runs great
Granite 4.0 H 350MIBM ยท 0.3B 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 393 tok/s Runs great 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 336 tok/s Runs great
Yi Coder 1.5B Chat01.AI ยท 1.5B 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 90.4 tok/s Runs great 16Basic quality (D) ยท 16 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 77.1 tok/s Runs great
SmolVLM 500M InstructHugging Face ยท 0.5B 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 262 tok/s Runs great 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 224 tok/s Runs great
Qwen2.5 0.5B InstructAlibaba ยท 0.5B 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 273 tok/s Runs great 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 233 tok/s Runs great
CodeGemma 2BGoogle ยท 2.5B 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 53.3 tok/s Runs great 14Basic quality (D) ยท 14 of 100: how capable the model is at F16, whatever it runs on. Match score 53/100 (quality, fit, speed and memory use) sets the ranking. 45.5 tok/s Runs great
Hunyuan 0.5B InstructTencent ยท 0.5B 8Basic quality (D) ยท 8 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 248 tok/s Runs great 8Basic quality (D) ยท 8 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 211 tok/s Runs great
LFM2.5 230MLiquid AI ยท 0.2B 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 582 tok/s Runs great 10Basic quality (D) ยท 10 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 496 tok/s Runs great
Qwen2.5-Math 72B InstructAlibaba ยท 73B 40Fair quality (C) ยท 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 11.4 tok/s Runs welloffloads to RAM 40Fair quality (C) ยท 40 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
SmolLM2 360M InstructHugging Face ยท 0.4B 8Basic quality (D) ยท 8 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 372 tok/s Runs great 8Basic quality (D) ยท 8 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 317 tok/s Runs great
TinyLlama 1.1B ChatTinyLlama ยท 1.1B 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 122 tok/s Runs great 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 52/100 (quality, fit, speed and memory use) sets the ranking. 104 tok/s Runs great
Command R+ (08-2024) 104BCohere ยท 104B 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slowheavy RAM offload 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek Coder 1.3B InstructDeepSeek ยท 1.4B 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 99.1 tok/s Runs great 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 84.5 tok/s Runs great
LLM4Decompile 1.3B V2LLM4Binary ยท 1.4B 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 99.1 tok/s Runs great 12Basic quality (D) ยท 12 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 84.5 tok/s Runs great
Qwen2 0.5B InstructAlibaba ยท 0.5B 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 273 tok/s Runs great 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 51/100 (quality, fit, speed and memory use) sets the ranking. 233 tok/s Runs great
ERNIE 4.5 0.3BBaidu ยท 0.4B 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 372 tok/s Runs great 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 317 tok/s Runs great
SmolVLM 256M InstructHugging Face ยท 0.3B 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 514 tok/s Runs great 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 439 tok/s Runs great
Gemma 3 270MGoogle ยท 0.3B 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 495 tok/s Runs great 6Basic quality (D) ยท 6 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 423 tok/s Runs great
SmolLM 360M InstructHugging Face ยท 0.4B 5Basic quality (D) ยท 5 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 372 tok/s Runs great 5Basic quality (D) ยท 5 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 317 tok/s Runs great
surya ocr 2.5Datalab ยท 0.7B 15Basic quality (D) ยท 15 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 194 tok/s Runs great 15Basic quality (D) ยท 15 of 100: how capable the model is at F16, whatever it runs on. Match score 50/100 (quality, fit, speed and memory use) sets the ranking. 165 tok/s Runs great
Command A Reasoning (111B)Cohere ยท 111B 57Good quality (B) ยท 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 2.8 tok/s Very slowheavy RAM offload 57Good quality (B) ยท 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
SmolLM2 135M InstructHugging Face ยท 0.1B 4Basic quality (D) ยท 4 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 1,029 tok/s Runs great 4Basic quality (D) ยท 4 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 878 tok/s Runs great
Command R+ 104B (04-2024)Cohere ยท 104B 44Fair quality (C) ยท 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 3.0 tok/s Slowheavy RAM offload 44Fair quality (C) ยท 44 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Falcon-H1 Tiny 90M InstructTII ยท 0.1B 3Basic quality (D) ยท 3 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 1,486 tok/s Runs great 3Basic quality (D) ยท 3 of 100: how capable the model is at F16, whatever it runs on. Match score 49/100 (quality, fit, speed and memory use) sets the ranking. 1,268 tok/s Runs great
SmolLM 135M InstructHugging Face ยท 0.1B 2Basic quality (D) ยท 2 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 1,029 tok/s Runs great 2Basic quality (D) ยท 2 of 100: how capable the model is at F16, whatever it runs on. Match score 48/100 (quality, fit, speed and memory use) sets the ranking. 878 tok/s Runs great
Qwen2.5 Coder 0.5B InstructAlibaba ยท 0.5B 8Basic quality (D) ยท 8 of 100: how capable the model is at F16, whatever it runs on. Match score 45/100 (quality, fit, speed and memory use) sets the ranking. 273 tok/s Runs great 8Basic quality (D) ยท 8 of 100: how capable the model is at F16, whatever it runs on. Match score 47/100 (quality, fit, speed and memory use) sets the ranking. 233 tok/s Runs great
Qwen1.5 110B ChatAlibaba ยท 111B 36Basic quality (D) ยท 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 44/100 (quality, fit, speed and memory use) sets the ranking. 2.5 tok/s Very slowheavy RAM offload 36Basic quality (D) ยท 36 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
gpt-oss 120BOpenAI ยท 117B (5.1B active) 84Great quality (A) ยท 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 84Great quality (A) ยท 84 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Kimi K3.1 Pro (2.8T)Moonshot AI ยท 2,780B (104.0B active) 69Good quality (B) ยท 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 69Good quality (B) ยท 69 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V4.1 Flash 1017 (552B-A16B)DeepSeek ยท 763B (16.0B active) 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-5.3 (753B-A40B)Zhipu AI ยท 753B (40.0B active) 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen3.8 Max (2.4T-A95B)Alibaba ยท 2,446B (95.0B active) 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Kimi K3 (2.8T)Moonshot AI ยท 2,780B (104.0B active) 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
LongCat 2.5 (1.6T-A48B)Meituan ยท 1,780B (48.0B active) 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiMo V2.6 Pro (1T-A42B)Xiaomi ยท 1,024B (42.0B active) 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Ling 3.1 Flash 1019 (560B-A25B)Ant Group (inclusionAI) ยท 572B (25.0B active) 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 68Good quality (B) ยท 68 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax M3.1 Flash (428B-A23B)MiniMax ยท 429B (23.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Nex N2.6 (321B-A18B)Nex-AGI ยท 321B (18.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Step 5 Preview (600B-A27B)StepFun ยท 604B (27.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-5.3 Flash (320B-A18B)Zhipu AI ยท 321B (18.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiMo V2.6 Flash (310B-A15B)Xiaomi ยท 311B (15.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V4.1 Flash (552B-A16B)DeepSeek ยท 763B (16.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Mistral Large 4 (1T-A52B)Mistral AI ยท 1,053B (52.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen3.8 Flash Next (125B-A6B)Alibaba ยท 180B (6.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-5.2 (753B-A40B)Zhipu AI ยท 753B (40.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Hunyuan Hy4 Preview (780B)Tencent ยท 780B (55.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Muse Spark Open (816B-A42B)Meta ยท 822B (42.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V4 Pro (1.6T-A49B)DeepSeek ยท 1,651B (49.0B active) 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 67Good quality (B) ยท 67 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Kimi K2.6 (1T)Moonshot AI ยท 1,027B (32.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Ling 3.1 Flash (560B-A25B)Ant Group (inclusionAI) ยท 572B (25.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiMo V2.5 Pro (1T-A42B)Xiaomi ยท 1,023B (42.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Intern-S2 (397B-A17B)Shanghai AI Lab ยท 403B (17.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-5.1 (744B-A40B)Zhipu AI ยท 754B (40.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Nex N2 Pro (397B-A17B)Nex-AGI ยท 397B (17.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V4 Pro (April 2026)DeepSeek ยท 1,599B (49.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Naive N0.5 Flash (309B-A15.5B)Naive AI ยท 309B (15.5B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen3.5 397B-A17BAlibaba ยท 403B (17.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax M3 (427B)MiniMax ยท 427B (25.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-5 (744B-A40B)Zhipu AI ยท 754B (40.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
K2-Horizon (375B-A23B)Institute of Foundation Models ยท 379B (23.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Hunyuan Hy3 (295B-A21B)Tencent ยท 299B (21.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Beam 1.0 (501B-A23B)Reflection AI ยท 501B (23.4B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Kimi K2.5 (1T)Moonshot AI ยท 1,027B (32.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Kimi K2.7 Code (1T)Moonshot AI ยท 1,027B (32.0B active) 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 66Good quality (B) ยท 66 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V4 Flash (284B-A13B)DeepSeek ยท 304B (13.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiMo V2.5 (310B-A15B)Xiaomi ยท 311B (15.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Ornith 1.5 397B-A17BDeepReinforce ยท 403B (17.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V3.2 (671B)DeepSeek ยท 685B (37.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Inkling Small (276B-A12B)Thinking Machines Lab ยท 266B (12.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-4.7 (355B-A32B)Zhipu AI ยท 358B (32.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Inkling (975B-A41B)Thinking Machines Lab ยท 952B (41.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax M2.7 (230B-A10B)MiniMax ยท 229B (10.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Step 3.7 Flash (198B-A11B)StepFun ยท 201B (11.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Nemotron 3 Ultra 550B-A55BNVIDIA ยท 561B (55.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek R1 0528 (671B)DeepSeek ยท 685B (37.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Kimi K2 Thinking (1T)Moonshot AI ยท 1,026B (32.0B active) 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 65Good quality (B) ยท 65 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V3.2 ExpDeepSeek ยท 685B (37.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Ling 2.6 1TAnt Group (inclusionAI) ยท 1,025B (63.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax M2.5 (230B-A10B)MiniMax ยท 229B (10.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek R1 (671B)DeepSeek ยท 671B (37.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V4 Flash (April 2026)DeepSeek ยท 291B (13.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Step 3.5 Flash (196B-A11B)StepFun ยท 199B (11.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V3.1 TerminusDeepSeek ยท 685B (37.0B active) 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 64Good quality (B) ยท 64 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V3.1 (671B)DeepSeek ยท 685B (37.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Kimi K2 Instruct 0905 (1T)Moonshot AI ยท 1,026B (32.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax M2.1 (230B-A10B)MiniMax ยท 229B (10.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen3 235B-A22B Thinking 2507Alibaba ยท 235B (22.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen3 Coder 480B-A35BAlibaba ยท 480B (35.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiMo V2 Flash (309B-A15B)Xiaomi ยท 310B (15.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Ling 1TAnt Group (inclusionAI) ยท 1,000B (50.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Mistral Large 3 (675B)Mistral AI ยท 675B (41.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen3 235B-A22B Instruct 2507Alibaba ยท 235B (22.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Solar Open 2 250BUpstage ยท 250B (15.0B active) 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 63Good quality (B) ยท 63 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Kimi K2 Instruct (July 2025)Moonshot AI ยท 1,026B (32.0B active) 62Good quality (B) ยท 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 62Good quality (B) ยท 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax M2 (230B-A10B)MiniMax ยท 229B (10.0B active) 62Good quality (B) ยท 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 62Good quality (B) ยท 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen3-VL 235B-A22B InstructAlibaba ยท 236B (22.0B active) 62Good quality (B) ยท 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 62Good quality (B) ยท 62 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Command A+ (218B-A25B)Cohere ยท 219B (25.0B active) 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
K-EXAONE 236B-A23BLG AI Research ยท 237B (23.0B active) 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
LongCat Flash Chat (560B)Meituan ยท 562B (27.0B active) 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Qwen3 235B-A22BAlibaba ยท 235B (22.0B active) 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 61Good quality (B) ยท 61 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Devstral 2 (123B)Mistral AI ยท 125B 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
InternVL3.5 241B-A28BShanghai AI Lab ยท 241B (28.0B active) 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Llama 3.1 Nemotron Ultra 253B v1NVIDIA ยท 253B 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 60Good quality (B) ยท 60 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Baichuan M3 235B-A22BBaichuan ยท 235B (22.0B active) 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-4.6 (355B-A32B)Zhipu AI ยท 357B (32.0B active) 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V3 0324 (671B)DeepSeek ยท 685B (37.0B active) 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax-M1 80KMiniMax ยท 456B (45.9B active) 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 58Good quality (B) ยท 58 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLM-4.5 (355B-A32B)Zhipu AI ยท 358B (32.0B active) 57Good quality (B) ยท 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 57Good quality (B) ยท 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Ling 3.0 Flash (127B-A5B)Ant Group (inclusionAI) ยท 127B (5.1B active) 57Good quality (B) ยท 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 57Good quality (B) ยท 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax-M1 40KMiniMax ยท 456B (45.9B active) 57Good quality (B) ยท 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 57Good quality (B) ยท 57 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Mistral Medium 3.5 (128B)Mistral AI ยท 128B 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Trinity Large Thinking (400B-A13B)Arcee AI ยท 399B (13.0B active) 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
ERNIE 4.5 300B-A47BBaidu ยท 300B (47.0B active) 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Hermes 4 405BNous Research ยท 406B 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Mistral Large 2.1 (2411)Mistral AI ยท 123B 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 56Good quality (B) ยท 56 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Mistral Large 2 (123B)Mistral AI ยท 123B 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
NorthCUA Probeipo-ai ยท 397B (17.0B active) 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Pixtral Large 124B (2411)Mistral AI ยท 124B 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 54Fair quality (C) ยท 54 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
dots.llm1 Instructrednote ยท 143B (14.0B active) 53Fair quality (C) ยท 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 53Fair quality (C) ยท 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
MiniMax-Text-01MiniMax ยท 456B (45.9B active) 53Fair quality (C) ยท 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 53Fair quality (C) ยท 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Tรผlu 3 405BAi2 ยท 406B 53Fair quality (C) ยท 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 53Fair quality (C) ยท 53 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V3 (671B)DeepSeek ยท 671B (37.0B active) 52Fair quality (C) ยท 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 52Fair quality (C) ยท 52 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek Coder V2 (236B)DeepSeek ยท 236B (21.0B active) 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 51Fair quality (C) ยท 51 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek Coder V2 Instruct (0614)DeepSeek ยท 236B (21.0B active) 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V2.5 1210DeepSeek ยท 236B (21.0B active) 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Llama 3.1 405B InstructMeta ยท 406B 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Llama 4 Maverick (17B x 128E)Meta ยท 402B (17.0B active) 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 50Fair quality (C) ยท 50 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V2.5DeepSeek ยท 236B (21.0B active) 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Jamba 1.5 LargeAI21 Labs ยท 399B (94.0B active) 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 49Fair quality (C) ยท 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Ling 3.0 flash SanteAnt Group (inclusionAI) ยท 127B (5.1B active) 47Fair quality (C) ยท 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 47Fair quality (C) ยท 47 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 6/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
DeepSeek V2 ChatDeepSeek ยท 236B (21.0B active) 43Fair quality (C) ยท 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 43Fair quality (C) ยท 43 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Mixtral 8x22B InstructMistral AI ยท 141B (39.0B active) 38Basic quality (D) ยท 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 38Basic quality (D) ยท 38 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 5/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Grok-1 (314B)xAI ยท 314B (86.0B active) 35Basic quality (D) ยท 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 35Basic quality (D) ยท 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Snowflake Arctic InstructSnowflake ยท 479B (17.0B active) 35Basic quality (D) ยท 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 35Basic quality (D) ยท 35 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 4/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Falcon 180B ChatTII ยท 180B 23Basic quality (D) ยท 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 23Basic quality (D) ยท 23 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 3/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run

Image models

60 of 61 image models run on the Intel Arc Pro B70. 32.0 GB usable for models.

58 of 61 image models run on the Apple M5 Max (36 GB). 24.1 GB usable for models.

Model QualityArc Pro B70 SpeedArc Pro B70 RunsArc Pro B70 QualityM5 Max SpeedM5 Max RunsM5 Max
Krea 2 TurboKrea ยท 12.8B 90Excellent quality (A+) ยท 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 13 s Runs great 90Excellent quality (A+) ยท 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 26 s Runs well
ERNIE-Image TurboBaidu ยท 8.0B 85Excellent quality (A+) ยท 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 10 s Runs great 83Great quality (A) ยท 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 20 s Runs well
Z-Image TurboAlibaba (Tongyi) ยท 6.2B 84Great quality (A) ยท 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 8.2 s Runs great 82Great quality (A) ยท 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 15 s Runs well
FLUX.2 [klein] 9BBlack Forest Labs ยท 9.1B 83Great quality (A) ยท 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 6.8 s Runs great 82Great quality (A) ยท 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 11 s Runs great
FLUX 3 Image KleinBlack Forest Labs ยท 5.6B 91Excellent quality (A+) ยท 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 19 s Runs well 89Excellent quality (A+) ยท 89 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 35 s Runs well
FLUX.2 [klein] 4BBlack Forest Labs ยท 3.9B 78Great quality (A) ยท 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 3.3 s Runs great 78Great quality (A) ยท 78 of 100: how capable the model is at FP16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 6.8 s Runs great
FLUX.1 [schnell]Black Forest Labs ยท 11.9B 69Good quality (B) ยท 69 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 7.5 s Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great
Fibo 2.0 DevBria AI ยท 8.1B 93Excellent quality (A+) ยท 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 73 s Slowtight fit 93Excellent quality (A+) ยท 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 2.7 min Very slowencoder unloaded after use
Qwen-Image 2.1 TurboAlibaba ยท 7.1B 87Excellent quality (A+) ยท 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 34 s Runs welltight fit 87Excellent quality (A+) ยท 87 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 74 s Slowtight fit ยท encoder unloaded after use
Ideogram 4Ideogram ยท 18.6B 90Excellent quality (A+) ยท 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 73 s Slowtight fit 85Excellent quality (A+) ยท 85 of 100: how capable the model is at NF4, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 3.1 min Very slowtight fit
FLUX.1 Krea [dev]Black Forest Labs ยท 11.9B 80Great quality (A) ยท 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 35 s Runs well 80Great quality (A) ยท 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 73 s Slow
Z-ImageAlibaba (Tongyi) ยท 6.2B 82Great quality (A) ยท 82 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 55 s Slow 80Great quality (A) ยท 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 1.7 min Slow
ERNIE-ImageBaidu ยท 8.0B 86Excellent quality (A+) ยท 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 65 s Slow 86Excellent quality (A+) ยท 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 2.2 min Very slow
FLUX.1 [dev]Black Forest Labs ยท 11.9B 78Great quality (A) ยท 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 35 s Runs well 78Great quality (A) ยท 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 73 s Slow
FLUX.1 Kontext [dev]Black Forest Labs ยท 11.9B 80Great quality (A) ยท 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 65 s Slow 80Great quality (A) ยท 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 2.2 min Very slow
FLUX.1 Fill [dev]Black Forest Labs ยท 11.9B 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 37 s Runs well 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 76 s Slow
Qwen-Image 2512Alibaba ยท 20.4B 83Great quality (A) ยท 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 5.9 min Very slow 83Great quality (A) ยท 83 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 12.8 min Very slowencoder unloaded after use
Ming-Image 0.1 DesignAnt Group (inclusionAI) ยท 6.2B 86Excellent quality (A+) ยท 86 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 56 s Slowencoder in RAM 84Great quality (A) ยท 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 1.7 min Slowtight fit ยท encoder unloaded after use
MOSS-Image SFTOpenMOSS ยท 5.1B 80Great quality (A) ยท 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 41 s Slow 80Great quality (A) ยท 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 86 s Slow
Stable Diffusion 3.5 Large TurboStability AI ยท 8.1B 61Good quality (B) ยท 61 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 6.0 s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 9.8 s Runs great
HiDream-I1 FastHiDream.ai ยท 17.1B 70Great quality (A) ยท 70 of 100: how capable the model is at NF4, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 36 s Runs well 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 64 s Slowencoder unloaded after use
HiDream-I1 DevHiDream.ai ยท 17.1B 74Great quality (A) ยท 74 of 100: how capable the model is at NF4, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 61 s Slow 78Great quality (A) ยท 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 1.8 min Slowencoder unloaded after use
FireRed-Image-Edit 1.1rednote (FireRed Team) ยท 20.4B 80Great quality (A) ยท 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 5.2 min Very slow 80Great quality (A) ยท 80 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 11.4 min Very slowencoder unloaded after use
HiDream-I1 FullHiDream.ai ยท 17.1B 78Great quality (A) ยท 78 of 100: how capable the model is at NF4, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 2.7 min Very slow 82Great quality (A) ยท 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 4.8 min Very slowencoder unloaded after use
Qwen-Image-Edit 2511Alibaba ยท 20.4B 79Great quality (A) ยท 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 5.2 min Very slow 79Great quality (A) ยท 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 11.4 min Very slowencoder unloaded after use
FLUX.2 [dev]Black Forest Labs ยท 32.2B 85Excellent quality (A+) ยท 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 2.5 min Very slowencoder in RAM 85Excellent quality (A+) ยท 85 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 5.1 min Very slowtight fit ยท encoder unloaded after use
Chroma1-HDLodestone Rock ยท 8.9B 76Great quality (A) ยท 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 59 s Slow 76Great quality (A) ยท 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 2.0 min Slow
Qwen-ImageAlibaba ยท 20.4B 86Excellent quality (A+) ยท 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 4.6 min Very slowencoder in RAM 88Excellent quality (A+) ยท 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 13.3 min Very slowCPU offload
Iris-3BSperidlabs ยท 3.0B 72Great quality (A) ยท 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 52 s Slow 72Great quality (A) ยท 72 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 1.8 min Slow
Stable Diffusion 3 MediumStability AI ยท 2.0B 52Fair quality (C) ยท 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 10 s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 19 s Runs well
CogView4 6BZhipu AI ยท 6.4B 68Good quality (B) ยท 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 53 s Slow 68Good quality (B) ยท 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 1.8 min Slow
Qwen-Image 2.1Alibaba ยท 7.1B 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 4.1 min Very slowtight fit ยท encoder in RAM 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
OmniGen2BAAI (VectorSpaceLab) ยท 4.0B 66Good quality (B) ยท 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 52 s Slow 66Good quality (B) ยท 66 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 1.8 min Slow
Stable Diffusion 3.5 LargeStability AI ยท 8.1B 64Good quality (B) ยท 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 32 s Runs well 64Good quality (B) ยท 64 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 67 s Slow
KolorsKuaishou ยท 2.6B 58Good quality (B) ยท 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 27 s Runs well
DreamShaper XL v2 TurboLykon ยท 2.6B 58Good quality (B) ยท 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 4.2 s Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 8.8 s Runs great
MiniMax H3 ImageMiniMax ยท 33.1B 91Excellent quality (A+) ยท 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 38 s Runs wellCPU offload 91Excellent quality (A+) ยท 91 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 80 s SlowCPU offload
Lumina-Image 2.0Shanghai AI Lab ยท 2.6B 62Good quality (B) ยท 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 23 s Runs well 62Good quality (B) ยท 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 48 s Slow
PixArt-ฮฃ XL 1024PixArt ยท 0.6B 50Fair quality (C) ยท 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 4.9 s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 10 s Runs great
AuraFlow v0.3fal ยท 6.9B 58Good quality (B) ยท 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 46 s Slow 58Good quality (B) ยท 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 1.6 min Slow
Cosmos3-Super Text2Image (64B)NVIDIA ยท 64.0B 90Excellent quality (A+) ยท 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 2.5 min Very slowCPU offload 90Excellent quality (A+) ยท 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 12/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
SDXL-Lightning (4-step)ByteDance ยท 2.6B 54Fair quality (C) ยท 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 2.3 s Runs great 54Fair quality (C) ยท 54 of 100: how capable the model is at FP16, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 4.8 s Runs great
Kandinsky 3Sber AI ยท 3.0B 48Fair quality (C) ยท 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 29 s Runs well 48Fair quality (C) ยท 48 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 64 s Slowencoder unloaded after use
Juggernaut XL v9RunDiffusion ยท 2.6B 60Good quality (B) ยท 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 13 s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 31 s Runs well
RealVisXL V5.0SG161222 ยท 2.6B 60Good quality (B) ยท 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 13 s Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at FP16, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 31 s Runs well
Stable Diffusion 3.5 MediumStability AI ยท 2.5B 56Good quality (B) ยท 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 18 s Runs well 56Good quality (B) ยท 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 37 s Runs well
Animagine XL 4.0Cagliostro Lab ยท 2.6B 58Good quality (B) ยท 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 29 s Runs well
Illustrious XL v0.1OnomaAI ยท 2.6B 58Good quality (B) ยท 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 29 s Runs well
Pony Diffusion V6 XLPurpleSmartAI ยท 2.6B 56Good quality (B) ยท 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 11 s Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 26 s Runs well
Playground v2.5Playground AI ยท 2.6B 60Good quality (B) ยท 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 24 s Runs well 60Good quality (B) ยท 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 51 s Slow
Animagine XL 3.1Cagliostro Lab ยท 2.6B 54Fair quality (C) ยท 54 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 12 s Runs great 56Good quality (B) ยท 56 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 29 s Runs well
Stable Diffusion XL 1.0Stability AI ยท 2.6B 53Fair quality (C) ยท 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 13 s Runs great 55Good quality (B) ยท 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 31 s Runs well
Stable CascadeStability AI ยท 5.2B 50Fair quality (C) ยท 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 15 s Runs well 52Fair quality (C) ยท 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 34 s Runs well
Hunyuan-DiT v1.2Tencent ยท 1.5B 52Fair quality (C) ยท 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 20 s Runs well 52Fair quality (C) ยท 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 41 s Slow
SDXL TurboStability AI ยท 2.6B 40Fair quality (C) ยท 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 0.5 s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 1.0 s Runs great
Realistic Vision V5.1SG161222 ยท 0.9B 42Fair quality (C) ยท 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.9 s Runs great 42Fair quality (C) ยท 42 of 100: how capable the model is at FP16, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. 8.2 s Runs great
DreamShaper 8Lykon ยท 0.9B 40Fair quality (C) ยท 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 3.9 s Runs great 40Fair quality (C) ยท 40 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 8.2 s Runs great
Stable Diffusion 1.5Stability AI / Runway ยท 0.9B 35Basic quality (D) ยท 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. 3.9 s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 8.2 s Runs great
LCM DreamShaper v7Tsinghua (LCM) ยท 0.9B 30Basic quality (D) ยท 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 0.8 s Runs great 30Basic quality (D) ยท 30 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 1.7 s Runs great
SD TurboStability AI ยท 0.9B 28Basic quality (D) ยท 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 54/100 (quality, fit, speed and memory use) sets the ranking. 0.3 s Runs great 28Basic quality (D) ยท 28 of 100: how capable the model is at FP16, whatever it runs on. Match score 55/100 (quality, fit, speed and memory use) sets the ranking. 0.5 s Runs great
HunyuanImage 3.0 (80B-A13B)Tencent ยท 83.0B 88Excellent quality (A+) ยท 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 88Excellent quality (A+) ยท 88 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run

Video models

40 of 43 video models run on the Intel Arc Pro B70. 32.0 GB usable for models.

31 of 43 video models run on the Apple M5 Max (36 GB). 24.1 GB usable for models.

Model QualityArc Pro B70 SpeedArc Pro B70 RunsArc Pro B70 QualityM5 Max SpeedM5 Max RunsM5 Max
LTX-Video 13B 0.9.8 (distilled)Lightricks ยท 13.0B 66Good quality (B) ยท 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 2.9 min Runs welltight fit 66Good quality (B) ยท 66 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 8.0 min Runs wellCPU offload
LTX-2.5 (22B, video + audio)Lightricks ยท 21.0B 81Great quality (A) ยท 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 6.2 min Runs wellencoder in RAM 86Excellent quality (A+) ยท 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 12.1 min SlowCPU offload
Bernini-R A14B (video generation + editing)ByteDance ยท 14.3B 83Great quality (A) ยท 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 35.2 min Slow 83Great quality (A) ยท 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 77.1 min Very slowencoder unloaded after use
HY-OmniWeaving (8.3B, unified video)Tencent ยท 8.3B 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 25.9 min Slow 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 56.7 min Very slowencoder unloaded after use
SCAIL-2 14B (character animation)Zhipu AI ยท 16.4B 78Great quality (A) ยท 78 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 58.7 min Very slowtight fit 80Great quality (A) ยท 80 of 100: how capable the model is at FP16, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 2.9 h Very slowCPU offload
Wan 2.1 I2V 14B 480PAlibaba ยท 14.0B 72Great quality (A) ยท 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 44.0 min Very slow 68Good quality (B) ยท 68 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 2.0 h Very slowtight fit
FramePack I2V (HunyuanVideo 13B)lllyasviel ยท 12.9B 72Great quality (A) ยท 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 19.6 min Slowtight fit 72Great quality (A) ยท 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 42.8 min Very slowencoder unloaded after use
LTX-2.3Lightricks ยท 18.9B 79Great quality (A) ยท 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 27.4 min Slowencoder in RAM 84Great quality (A) ยท 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 53.0 min Very slowCPU offload
Wan 2.2 S2V 14B (speech to video)Alibaba ยท 16.3B 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 48.9 min Very slowtight fit 70Great quality (A) ยท 70 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 2.3 h Very slowtight fit
LTX-Video 2BLightricks ยท 1.9B 50Fair quality (C) ยท 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 82 s Runs great 50Fair quality (C) ยท 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 2.8 min Runs great
LTX-2 (19B, video + audio)Lightricks ยท 18.9B 76Great quality (A) ยท 76 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 27.4 min Slowencoder in RAM 80Great quality (A) ยท 80 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 53.0 min Very slowCPU offload
SkyReels V2 T2V 14B 540PSkywork ยท 14.0B 72Great quality (A) ยท 72 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 68.5 min Very slowtight fit 74Great quality (A) ยท 74 of 100: how capable the model is at FP16, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.4 h Very slowCPU offload
Wan 2.2 I2V A14BAlibaba ยท 14.0B 79Great quality (A) ยท 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 1.8 h Very slowencoder in RAM 84Great quality (A) ยท 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 3.5 h Very slowtight fit ยท CPU offload
Wan 2.2 T2V A14BAlibaba ยท 14.0B 79Great quality (A) ยท 79 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 1.8 h Very slowencoder in RAM 84Great quality (A) ยท 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 64/100 (quality, fit, speed and memory use) sets the ranking. 3.5 h Very slowtight fit ยท CPU offload
CogVideoX 5BZhipu AI ยท 5.0B 55Good quality (B) ยท 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 9.8 min Runs well 53Fair quality (C) ยท 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 18.3 min Slow
CogVideoX 5B I2VZhipu AI ยท 5.6B 55Good quality (B) ยท 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 9.8 min Runs well 53Fair quality (C) ยท 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 18.3 min Slow
Cosmos-Predict2 2B Video2WorldNVIDIA ยท 2.0B 56Good quality (B) ยท 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 12.2 min Slow 56Good quality (B) ยท 56 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 26.8 min Slowtight fit ยท encoder unloaded after use
HunyuanVideo 1.5 (8.3B)Tencent ยท 8.3B 82Great quality (A) ยท 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 1.5 h Very slowtight fit ยท encoder in RAM 82Great quality (A) ยท 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Wan 2.2 TI2V 5BAlibaba ยท 5.0B 65Good quality (B) ยท 65 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 33.9 min Slowtight fit 68Good quality (B) ยท 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 68.9 min Very slowtight fit ยท CPU offload
Wan 2.1 VACE 1.3BAlibaba ยท 2.2B 55Good quality (B) ยท 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 14.1 min Slow 53Fair quality (C) ยท 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 26.5 min Slow
Mochi 1 Preview (10B)Genmo ยท 10.0B 60Good quality (B) ยท 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 53.8 min Very slow 60Good quality (B) ยท 60 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 2.0 h Very slowtight fit ยท encoder unloaded after use
MiniMax H3 (33B, video + audio)MiniMax ยท 20.1B 89Excellent quality (A+) ยท 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 84.5 min Very slowCPU offload 89Excellent quality (A+) ยท 89 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 2.9 h Very slowtight fit ยท CPU offload
Wan 2.1 I2V 14B 720PAlibaba ยท 14.0B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 2.3 h Very slowencoder in RAM 76Great quality (A) ยท 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 4.4 h Very slowtight fit ยท CPU offload
Wan 2.1 T2V 14BAlibaba ยท 14.0B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 2.3 h Very slowencoder in RAM 76Great quality (A) ยท 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 4.4 h Very slowtight fit ยท CPU offload
SkyReels V2 Diffusion Forcing 1.3BSkywork ยท 1.4B 52Fair quality (C) ยท 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 16.3 min Slow 50Fair quality (C) ยท 50 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 30.6 min Slow
Wan 2.2 Animate 14BAlibaba ยท 17.3B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 2.2 h Very slowtight fit ยท encoder in RAM 72Great quality (A) ยท 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 5.6 h Very slowtight fit ยท CPU offload
CogVideoX 2BZhipu AI ยท 1.7B 44Fair quality (C) ยท 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 2.9 min Runs great 44Fair quality (C) ยท 44 of 100: how capable the model is at FP8, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 6.1 min Runs well
Wan 2.1 VACE 14BAlibaba ยท 17.3B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 2.5 h Very slowtight fit ยท encoder in RAM 72Great quality (A) ยท 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 58/100 (quality, fit, speed and memory use) sets the ranking. 6.5 h Very slowtight fit ยท CPU offload
Kandinsky 6.0 Video Lite (3B, video + audio)Sber AI ยท 3.2B 68Good quality (B) ยท 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 67/100 (quality, fit, speed and memory use) sets the ranking. 17.7 min Slowencoder in RAM 68Good quality (B) ยท 68 of 100: how capable the model is at FP16, whatever it runs on. Match score 68/100 (quality, fit, speed and memory use) sets the ranking. 36.9 min Slowencoder unloaded after use
Kandinsky 6.0 Pro (29B, video + audio)Sber AI ยท 30.1B 88Excellent quality (A+) ยท 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 3.5 h Very slowtight fit ยท CPU offload 88Excellent quality (A+) ยท 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Wan 2.1 T2V 1.3BAlibaba ยท 1.3B 53Fair quality (C) ยท 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 11.7 min Runs well 55Good quality (B) ยท 55 of 100: how capable the model is at FP16, whatever it runs on. Match score 69/100 (quality, fit, speed and memory use) sets the ranking. 27.2 min Slow
HunyuanVideo 13BTencent ยท 13.0B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 2.7 h Very slowtight fit ยท encoder in RAM 66Good quality (B) ยท 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
HunyuanVideo I2V 13BTencent ยท 13.0B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 2.7 h Very slowtight fit ยท encoder in RAM 66Good quality (B) ยท 66 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
SkyReels V2 T2V 14B 720PSkywork ยท 14.0B 76Great quality (A) ยท 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 63/100 (quality, fit, speed and memory use) sets the ranking. 2.5 h Very slowCPU offload 76Great quality (A) ยท 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Step-Video T2V (30B)StepFun ยท 30.0B 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 62/100 (quality, fit, speed and memory use) sets the ranking. 3.3 h Very slowCPU offload 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Cosmos-Predict2 14B Video2WorldNVIDIA ยท 14.0B 68Good quality (B) ยท 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 1.7 h Very slowCPU offload 68Good quality (B) ยท 68 of 100: how capable the model is at FP8, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Allegro (2.8B)Rhymes AI ยท 2.8B 52Fair quality (C) ยท 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 68.5 min Very slowtight fit ยท encoder in RAM 52Fair quality (C) ยท 52 of 100: how capable the model is at FP16, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Open-Sora 2.0 (11B)HPC-AI Tech ยท 11.0B 70Great quality (A) ยท 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 3.5 h Very slowCPU offload 70Great quality (A) ยท 70 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 7.4 h Very slowCPU offload
AnimateDiff v3 (SD 1.5)Shanghai AI Lab ยท 1.3B 35Basic quality (D) ยท 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 59/100 (quality, fit, speed and memory use) sets the ranking. 65 s Runs great 35Basic quality (D) ยท 35 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. 2.3 min Runs great
Cosmos 1.0 Diffusion 7B Text2WorldNVIDIA ยท 7.0B 60Good quality (B) ยท 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 56/100 (quality, fit, speed and memory use) sets the ranking. 38.9 min Slowtight fit ยท CPU offload 60Good quality (B) ยท 60 of 100: how capable the model is at FP16, whatever it runs on. Match score 8/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Cosmos3-Super Image2Video (64B)NVIDIA ยท 64.0B 88Excellent quality (A+) ยท 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 88Excellent quality (A+) ยท 88 of 100: how capable the model is at FP16, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
Wan-Dancer 14B (music to dance)Alibaba ยท 17.3B 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 74Great quality (A) ยท 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
CogVideoX 1.5 5BZhipu AI ยท 5.6B 58Good quality (B) ยท 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 58Good quality (B) ยท 58 of 100: how capable the model is at FP8, whatever it runs on. Match score 7/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run

Decision models

68 of 70 decision models run on the Intel Arc Pro B70. 32.0 GB usable for models.

67 of 70 decision models run on the Apple M5 Max (36 GB). 24.1 GB usable for models.

Model QualityArc Pro B70 SpeedArc Pro B70 RunsArc Pro B70 QualityM5 Max SpeedM5 Max RunsM5 Max
Qwen3 Reranker 4BAlibaba ยท 4.0B 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. 118 ms Runs great 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 240 ms Runs well
Jina Reranker v3.5Jina AI ยท 600M 86Excellent quality (A+) ยท 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 23 ms Runs great 86Excellent quality (A+) ยท 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 42 ms Runs great
zerank-2ZeroEntropy ยท 4.0B 88Excellent quality (A+) ยท 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. 118 ms Runs great 88Excellent quality (A+) ยท 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 240 ms Runs well
Shieldstral 1.0 3BMistral AI ยท 3.9B 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 113 ms Runs great 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 230 ms Runs well
Intern-Decision 4BShanghai AI Lab ยท 4.5B 84Great quality (A) ยท 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 94 ms Runs great 84Great quality (A) ยท 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 146 ms Runs well
Jina Reranker v3Jina AI ยท 600M 84Great quality (A) ยท 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 23 ms Runs great 84Great quality (A) ยท 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. 42 ms Runs great
d1 3BLiquid AI ยท 3.1B 82Great quality (A) ยท 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 66 ms Runs great 82Great quality (A) ยท 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 102 ms Runs great
mxbai-rerank-large v2Mixedbread ยท 1.5B 82Great quality (A) ยท 82 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 54 ms Runs great 82Great quality (A) ยท 82 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 94 ms Runs great
FRIDA-DecisionsSber AI ยท 820M 80Great quality (A) ยท 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 17 ms Runs great 80Great quality (A) ยท 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 32 ms Runs great
GLiNER2.5 DecideFastino ยท 490M 80Great quality (A) ยท 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 12 ms Runs great 80Great quality (A) ยท 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 21 ms Runs great
LemonDecider 4BAnt Group (inclusionAI) ยท 4.5B 80Great quality (A) ยท 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 75 ms Runs great 80Great quality (A) ยท 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 139 ms Runs well
Qwen3 Reranker 0.6BAlibaba ยท 600M 80Great quality (A) ยท 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 23 ms Runs great 80Great quality (A) ยท 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 42 ms Runs great
zerank-1 smallZeroEntropy ยท 1.7B 80Great quality (A) ยท 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. 60 ms Runs great 80Great quality (A) ยท 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 105 ms Runs great
Qwen3 Reranker 8BAlibaba ยท 8.2B 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 236 ms Runs well 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 485 ms Slow
BGE Reranker v2 GemmaBAAI ยท 2.5B 78Great quality (A) ยท 78 of 100: how capable the model is at F16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. 85 ms Runs great 78Great quality (A) ยท 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 151 ms Runs well
Llama Prompt Guard 2 86MMeta ยท 280M 76Great quality (A) ยท 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 76Great quality (A) ยท 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 23 ms Runs great
mxbai-rerank-base v2Mixedbread ยท 490M 76Great quality (A) ยท 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 20 ms Runs great 76Great quality (A) ยท 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. 35 ms Runs great
GLiClass ModernBERT Large v3Knowledgator ยท 400M 74Great quality (A) ยท 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great 74Great quality (A) ยท 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 18 ms Runs great
OpenAI Privacy FilterOpenAI ยท 1.4B (50M active) 78Great quality (A) ยท 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 4.8 ms Runs great 78Great quality (A) ยท 78 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. 5.7 ms Runs great
Skywork Reward V2 Llama 3.1 8BSkywork ยท 7.5B 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 410 ms Slow 90Excellent quality (A+) ยท 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 896 ms Slow
BGE Reranker v2 M3BAAI ยท 570M 72Great quality (A) ยท 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 22 ms Runs great 72Great quality (A) ยท 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 43 ms Runs great
DeBERTa v3 Large Zero-shot v2.0Moritz Laurer ยท 440M 72Great quality (A) ยท 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 11 ms Runs great 72Great quality (A) ยท 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 19 ms Runs great
DuoGuard 0.5BDuoGuard ยท 490M 72Great quality (A) ยท 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 19 ms Runs great 72Great quality (A) ยท 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 37 ms Runs great
Skywork Reward V2 Qwen3 0.6BSkywork ยท 600M 72Great quality (A) ยท 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 41 ms Runs great 72Great quality (A) ยท 72 of 100: how capable the model is at F16, whatever it runs on. Match score 87/100 (quality, fit, speed and memory use) sets the ranking. 86 ms Runs great
GTE Reranker ModernBERT BaseAlibaba ยท 150M 70Great quality (A) ยท 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 8.7 ms Runs great 70Great quality (A) ยท 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 14 ms Runs great
Jina Reranker v2 Base MultilingualJina AI ยท 280M 70Great quality (A) ยท 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 13 ms Runs great 70Great quality (A) ยท 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 23 ms Runs great
ModernBERT Large Zero-shot v2.0Moritz Laurer ยท 400M 70Great quality (A) ยท 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great 70Great quality (A) ยท 70 of 100: how capable the model is at F16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. 18 ms Runs great
GLiGuard 300MFastino ยท 210M 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 11 ms Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 18 ms Runs great
Granite Embedding Reranker English R2IBM ยท 150M 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 8.7 ms Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 14 ms Runs great
LayaConvai Innovations ยท 420M 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 11 ms Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 18 ms Runs great
Llama Prompt Guard 2 22MMeta ยท 71M 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 6.2 ms Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 8.9 ms Runs great
Qwen3Guard Gen 4BAlibaba ยท 4.4B 83Great quality (A) ยท 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 181 ms Runs well 84Great quality (A) ยท 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 352 ms Runs well
Qwen3Guard Gen 0.6BAlibaba ยท 750M 74Great quality (A) ยท 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 56 ms Runs great 74Great quality (A) ยท 74 of 100: how capable the model is at F16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 84 ms Runs great
Skywork Reward V2 Qwen3 8BSkywork ยท 7.6B 84Great quality (A) ยท 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. 414 ms Slow 84Great quality (A) ยท 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 904 ms Slow
GLiClass Base v3Knowledgator ยท 190M 66Good quality (B) ยท 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 7.0 ms Runs great 66Good quality (B) ยท 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great
ProtectAI DeBERTa v3 Prompt Injection v2Protect AI ยท 180M 66Good quality (B) ยท 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 9.6 ms Runs great 66Good quality (B) ยท 66 of 100: how capable the model is at F16, whatever it runs on. Match score 84/100 (quality, fit, speed and memory use) sets the ranking. 16 ms Runs great
DeBERTa v3 Base Zero-shot v2.0Moritz Laurer ยท 180M 64Good quality (B) ยท 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 6.8 ms Runs great 64Good quality (B) ยท 64 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great
GLiNER2 Base v1Fastino ยท 210M 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 7.3 ms Runs great 68Good quality (B) ยท 68 of 100: how capable the model is at F16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 11 ms Runs great
Qwen3Guard Gen 8BAlibaba ยท 8.2B 85Excellent quality (A+) ยท 85 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. 332 ms Runs well 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 650 ms Slow
HHEM 2.1 OpenVectara ยท 110M 62Good quality (B) ยท 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 11 ms Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 19 ms Runs great
Llama 3.1 Nemotron Safety Guard 8B v3NVIDIA ยท 8.0B 83Great quality (A) ยท 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 326 ms Runs well 84Great quality (A) ยท 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 638 ms Slow
LettuceDetect LargeKR Labs ยท 400M 66Good quality (B) ยท 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 29 ms Runs great 66Good quality (B) ยท 66 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 59 ms Runs great
Granite Guardian 4.1 8BIBM ยท 8.4B 83Great quality (A) ยท 83 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 340 ms Runs well 84Great quality (A) ยท 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 665 ms Slow
Laya MultilingualConvai Innovations ยท 320M 60Good quality (B) ยท 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 9.0 ms Runs great 60Good quality (B) ยท 60 of 100: how capable the model is at F16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. 15 ms Runs great
WildGuard 7BAi2 ยท 7.3B 79Great quality (A) ยท 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 294 ms Runs well 80Great quality (A) ยท 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 576 ms Slow
NuExtract 2.0 2BNuMind ยท 2.2B 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 125 ms Runs great 66Good quality (B) ยท 66 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 174 ms Runs well
ShieldGemma 2BGoogle ยท 2.6B 58Good quality (B) ยท 58 of 100: how capable the model is at F16, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. 88 ms Runs great 58Good quality (B) ยท 58 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 157 ms Runs well
GLiNER Multi v2.1Urchade Zaratiana ยท 290M 62Good quality (B) ยท 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 8.5 ms Runs great 62Good quality (B) ยท 62 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 14 ms Runs great
Granite Guardian 3.3 8BIBM ยท 8.2B 79Great quality (A) ยท 79 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 331 ms Runs well 80Great quality (A) ยท 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 649 ms Slow
NuExtract 2.0 4BNuMind ยท 3.8B 72Great quality (A) ยท 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 209 ms Runs well 75Great quality (A) ยท 75 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. 373 ms Runs well
BART Large MNLIMeta ยท 410M 56Good quality (B) ยท 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 10 ms Runs great 56Good quality (B) ยท 56 of 100: how capable the model is at F16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. 18 ms Runs great
ms-marco MiniLM L6 v2Sentence Transformers ยท 23M 52Fair quality (C) ยท 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 4.7 ms Runs great 52Fair quality (C) ยท 52 of 100: how capable the model is at F16, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 5.6 ms Runs great
ShieldGemma 9BGoogle ยท 9.2B 64Good quality (B) ยท 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. 266 ms Runs well 64Good quality (B) ยท 64 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 547 ms Slow
Llama Guard 3 1BMeta ยท 1.5B 56Good quality (B) ยท 56 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. 72 ms Runs great 55Good quality (B) ยท 55 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. 113 ms Runs great
Llama Guard 3 8BMeta ยท 8.0B 67Good quality (B) ยท 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. 326 ms Runs well 68Good quality (B) ยท 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 638 ms Slow
ShieldGemma 27BGoogle ยท 27B 65Good quality (B) ยท 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. 750 ms Slow 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 1.5 s Very slow
Llama Guard 4 12BMeta ยท 12B 65Good quality (B) ยท 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. 485 ms Slow 66Good quality (B) ยท 66 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 951 ms Slow
gpt-oss-safeguard 20BOpenAI ยท 22B (3.6B active) 76Great quality (A) ยท 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 1.6 s Very slow 76Great quality (A) ยท 76 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 1.8 s Very slow
CompassJudger-2 32BOpenCompass ยท 33B 83Great quality (A) ยท 83 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 12.1 s Very slow 81Great quality (A) ยท 81 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 13.7 s Very slowtight fit
DynaGuard 8BUniversity of Maryland ยท 8.2B 79Great quality (A) ยท 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 4.7 s Very slow 79Great quality (A) ยท 79 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 5.7 s Very slow
Nemotron Content Safety Reasoning 4BNVIDIA ยท 4.3B 74Great quality (A) ยท 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 1.3 s Slow 77Great quality (A) ยท 77 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 2.3 s Very slow
Selene 1 Mini 8BAtla ยท 8.0B 80Great quality (A) ยท 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. 4.6 s Very slow 80Great quality (A) ยท 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 5.7 s Very slow
CompassJudger-2 7BOpenCompass ยท 7.6B 78Great quality (A) ยท 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 4.3 s Very slow 78Great quality (A) ยท 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 5.5 s Very slow
DynaGuard 1.7BUniversity of Maryland ยท 2.0B 69Good quality (B) ยท 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. 876 ms Slow 69Good quality (B) ยท 69 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. 1.1 s Slow
Prometheus 2 7BPrometheus Eval ยท 7.2B 67Good quality (B) ยท 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 3.1 s Very slow 68Good quality (B) ยท 68 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 5.2 s Very slow
Prometheus 2 8x7BPrometheus Eval ยท 47B (13B active) 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 4.5 s Very slowtight fit 70Great quality (A) ยท 70 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 9/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
GLIDERPatronus AI ยท 3.8B 62Good quality (B) ยท 62 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 1.2 s Slow 65Good quality (B) ยท 65 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 66/100 (quality, fit, speed and memory use) sets the ranking. 2.1 s Very slow
Flow Judge v0.1Flow AI ยท 3.8B 60Good quality (B) ยท 60 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 1.2 s Slow 63Good quality (B) ยท 63 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 65/100 (quality, fit, speed and memory use) sets the ranking. 2.1 s Very slow
GLiDE 27BFastino ยท 26B 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 86Excellent quality (A+) ยท 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 11/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run
gpt-oss-safeguard 120BOpenAI ยท 117B (5.1B active) 80Great quality (A) ยท 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run 80Great quality (A) ยท 80 of 100: how capable the model is at MXFP4, whatever it runs on. Match score 10/100 (quality, fit, speed and memory use) sets the ranking. โ€” Won't run

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked โ€œMeasuredโ€ come from a linked benchmark.

Was this page helpful?