Hardware · Orange Pi
What AI models can the Orange Pi AIpro (20T) (12 GB) run?
- RAM
- 12.0 GB
- Usable for models
- 9.8 GB
- Memory bandwidth
- 34 GB/s
- Runs models on
- CPU cores
- Released
- 2024
Models run on the CPU cores here, so the memory’s speed and how fast those cores can read it set the pace: small models (about 1-4B) are usable, bigger ones are slow. We plan around 9.8 GB of the 12.0 GB (what Linux leaves free for a model). Ascend DaVinci V300 AI core: 20 TOPS INT8 / 10 TFLOPS FP16, programmed with Huawei’s CANN tools. llama.cpp’s CANN backend doesn’t support the 310B yet, so llama.cpp and Ollama run on the four CPU cores.
More about this hardware
| Maker | Orange Pi |
|---|---|
| Type | Single-board computer or edge AI board |
| Share of memory for models | 82% (what Linux typically leaves free on a board this size) |
| Chip | Huawei Ascend 310B |
| AI accelerator | Ascend DaVinci V300 AI core: 20 TOPS INT8 / 10 TFLOPS FP16, programmed with Huawei’s CANN tools. llama.cpp’s CANN backend doesn’t support the 310B yet, so llama.cpp and Ollama run on the four CPU cores. |
| Boards with this chip | Orange Pi AIpro (20T) 12 GB / 24 GB |
| Memory type | LPDDR4X-4266 (Orange Pi doesn’t publish the bus width) |
| Bandwidth figure | 34.1 GB/s, estimated, see the note below |
| Note | 4 TaiShan V200M cores at 1.6 GHz. Bandwidth assumes a 64-bit bus; speeds assume llama.cpp on the CPU at about 30% of it (estimate). Its NPU only helps with models converted for Huawei’s tools. |
| Specifications | www.orangepi.org |
Apps that run models well on the Orange Pi AIpro (20T) (12 GB)
Linux on ARM boards
1 product with the Orange Pi AIpro (20T) (12 GB)
Best LLMs for the Orange Pi AIpro (20T) (12 GB)
| Quality | Model | Quant | Memory | Speed | Runs |
|---|---|---|---|---|---|
| 78Great quality (A) · 78 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking. | Gemma 4 12B | Q4_K_M | 9.4 GB | 1.5 tok/s | Very slowtight fit |
| 74Great quality (A) · 74 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. | SenseNova Loop 9B Preview | Q4_K_M | 7.7 GB | 1.8 tok/s | Very slow |
| 73Great quality (A) · 73 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. | Ornith 1.5 9B | Q4_K_M | 7.5 GB | 1.8 tok/s | Very slow |
| 72Great quality (A) · 72 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. | Qwen3.5 9B | Q4_K_M | 7.5 GB | 1.8 tok/s | Very slow |
| 71Great quality (A) · 71 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking. | Ornith 1.0 9B | Q4_K_M | 7.5 GB | 1.8 tok/s | Very slow |
| 67Good quality (B) · 67 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. | Nemotron 3.5 Bolt E4B | Q5_K_M | 7.2 GB | 3.2 tok/s | Slow |
| 57Good quality (B) · 57 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking. | Ling 3.0 Tiny (8B-A1.3B) | Q4_K_M | 6.4 GB | 12.8 tok/s | Runs well |
| 65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. | Gemma 4 E4B | Q5_K_M | 7.2 GB | 3.2 tok/s | Slow |
| 65Good quality (B) · 65 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. | Nanbeige4.2 3B | Q5_K_M | 5.8 GB | 3.5 tok/s | Slow |
| 67Good quality (B) · 67 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. | Nemotron Labs Dory 1.0 4B | Q6_K | 5.1 GB | 3.0 tok/s | Slow |
| 49Fair quality (C) · 49 of 100: how capable the model is at Q2_K, whatever it runs on. Match score 71/100 (quality, fit, speed and memory use) sets the ranking. | Maple Preview (20B-A1B) | Q2_K | 9.6 GB | 18.1 tok/s | Runs welltight fit |
| 49Fair quality (C) · 49 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 70/100 (quality, fit, speed and memory use) sets the ranking. | LFM2.5 8B-A1B | Q4_K_M | 6.7 GB | 11.2 tok/s | Runs well |
Best Image models for the Orange Pi AIpro (20T) (12 GB)
| Quality | Model | Quant | Memory | Speed | Runs |
|---|---|---|---|---|---|
| 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 76/100 (quality, fit, speed and memory use) sets the ranking. | FLUX 3 Image Klein | Q8 GGUF | 8.3 GB | 28.3 min | Very slowencoder unloaded after use |
| 72Great quality (A) · 72 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking. | FLUX.2 [klein] 4B | Q4 GGUF | 8.6 GB | 4.9 min | Very slowtight fit |
| 81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. | ERNIE-Image | Q4 GGUF | 7.4 GB | 1.9 h | Very slowencoder unloaded after use |
| 81Great quality (A) · 81 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. | MOSS-Image SFT | Q8 GGUF | 7.8 GB | 69.5 min | Very slowencoder unloaded after use |
| 83Great quality (A) · 83 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. | Z-Image Turbo | Q8 GGUF | 9.0 GB | 12.4 min | Very slowtight fit · encoder unloaded after use |
| 78Great quality (A) · 78 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking. | FLUX.2 [klein] 9B | Q4 GGUF | 8.0 GB | 9.3 min | Very slowencoder unloaded after use |
Best Video models for the Orange Pi AIpro (20T) (12 GB)
| Quality | Model | Quant | Memory | Speed | Runs |
|---|---|---|---|---|---|
| 74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 61/100 (quality, fit, speed and memory use) sets the ranking. | HY-OmniWeaving (8.3B, unified video) | FP8 | 7.8 GB | 42.5 h | Very slowCPU offload |
| 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. | Wan 2.1 T2V 1.3B | FP8 | 7.1 GB | 15.5 h | Very slowencoder unloaded after use |
| 53Fair quality (C) · 53 of 100: how capable the model is at FP8, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. | Wan 2.1 VACE 1.3B | FP8 | 8.0 GB | 16.8 h | Very slowencoder unloaded after use |
| 50Fair quality (C) · 50 of 100: how capable the model is at FP16, whatever it runs on. Match score 60/100 (quality, fit, speed and memory use) sets the ranking. | LTX-Video 2B | FP16 | 8.0 GB | 1.8 h | Very slowencoder unloaded after use |
| 70Great quality (A) · 70 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 57/100 (quality, fit, speed and memory use) sets the ranking. | Wan 2.2 S2V 14B (speech to video) | Q4 GGUF | 9.0 GB | 106.8 h | Very slowtight fit · CPU offload |
Best Decision models for the Orange Pi AIpro (20T) (12 GB)
| Quality | Model | Quant | Memory | Speed | Runs |
|---|---|---|---|---|---|
| 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. | Qwen3 Reranker 4B | Q8_0 | 6.0 GB | 69.1 s | Very slow |
| 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. | Jina Reranker v3.5 | Q8_0 | 2.3 GB | 10.3 s | Very slow |
| 88Excellent quality (A+) · 88 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. | zerank-2 | Q8_0 | 6.0 GB | 69.1 s | Very slow |
| 78Great quality (A) · 78 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. | OpenAI Privacy Filter | Q8_0 | 2.4 GB | 468 ms | Slow |
| 87Excellent quality (A+) · 87 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. | Qwen3 Reranker 8B | Q5_K_M | 7.5 GB | 140.3 s | Very slow |
| 86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. | Shieldstral 1.0 3B | Q8_0 | 5.7 GB | 66.1 s | Very slow |
| 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. | Jina Reranker v3 | Q8_0 | 2.3 GB | 10.3 s | Very slow |
| 84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 78/100 (quality, fit, speed and memory use) sets the ranking. | Intern-Decision 4B | Q8_0 | 6.3 GB | 39.3 s | Very slow |
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.