Hardware · NVIDIA
What AI models can the NVIDIA Jetson AGX Orin 64GB (64 GB) run?
- Unified memory
- 64.0 GB
- Usable for models
- 57.6 GB
- Memory bandwidth
- 205 GB/s
- Runs models on
- GPU (CUDA)
- Released
- 2022
Models run on the GPU (CUDA) here, sharing the board’s memory with the system. We plan around 57.6 GB of the 64.0 GB (what Linux leaves free for a model). Ampere GPU with 2,048 CUDA cores and 64 Tensor Cores, plus two NVDLA v2 engines: up to 275 TOPS (INT8, sparse).
More about this hardware
| Maker | NVIDIA |
|---|---|
| Type | Single-board computer or edge AI board |
| GPU share of memory | 90% (what the GPU can typically be given for a model) |
| Chip | NVIDIA Jetson AGX Orin 64GB |
| AI accelerator | Ampere GPU with 2,048 CUDA cores and 64 Tensor Cores, plus two NVDLA v2 engines: up to 275 TOPS (INT8, sparse). |
| Boards with this chip | Jetson AGX Orin Developer Kit · Jetson AGX Orin 64GB module · Jetson AGX Orin Industrial |
| Memory type | 64 GB LPDDR5, 256-bit |
| Bandwidth figure | 204.8 GB/s, as the maker states it |
| Note | Unified LPDDR5; estimate ~10% OS reserve. The Industrial module has the same 64 GB and 204.8 GB/s, with ECC and a wider temperature range. |
| Specifications | nvidia.com |
Apps that run models well on the NVIDIA Jetson AGX Orin 64GB (64 GB)
1 product with the NVIDIA Jetson AGX Orin 64GB (64 GB)
Best LLMs for the NVIDIA Jetson AGX Orin 64GB (64 GB)
| Quality | Model | Quant | Memory | Speed | Runs |
|---|---|---|---|---|---|
| 92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. | Qwen3.6 35B-A3B | Q6_K | 34.9 GB | 48.1 tok/s | Runs great |
| 90Excellent quality (A+) · 90 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. | Flint (40B-A2.5B) | Q8_0 | 47.6 GB | 42.8 tok/s | Runs great |
| 92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. | Qwen3.5 35B-A3B | Q6_K | 34.9 GB | 48.1 tok/s | Runs great |
| 92Excellent quality (A+) · 92 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 95/100 (quality, fit, speed and memory use) sets the ranking. | MiniCPM-V 4.7 35B-A3B | Q6_K | 34.2 GB | 48.1 tok/s | Runs great |
| 91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. | Nex N2 Mini (35B-A3B) | Q6_K | 34.2 GB | 48.1 tok/s | Runs great |
| 90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. | Sage-A1.5 35B-A3B | Q6_K | 34.9 GB | 48.1 tok/s | Runs great |
| 90Excellent quality (A+) · 90 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. | KAT-Coder V2.5 Dev (35B-A3B) | Q6_K | 33.8 GB | 48.1 tok/s | Runs great |
| 89Excellent quality (A+) · 89 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking. | Ornith 1.5 35B-A3B | Q6_K | 34.9 GB | 48.1 tok/s | Runs great |
| 88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. | Apodex 1.1 Mini (35B-A3B) | Q6_K | 34.9 GB | 48.1 tok/s | Runs great |
| 88Excellent quality (A+) · 88 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. | Ornith 1.0 35B-A3B | Q6_K | 34.9 GB | 48.1 tok/s | Runs great |
| 86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. | Kolibri 1 (78B-A3.5B) | Q4_K_M | 53.0 GB | 58.4 tok/s | Runs welltight fit |
| 86Excellent quality (A+) · 86 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. | Agents-A1 35B-A3B | Q6_K | 34.2 GB | 48.1 tok/s | Runs great |
Best Image models for the NVIDIA Jetson AGX Orin 64GB (64 GB)
| Quality | Model | Quant | Memory | Speed | Runs |
|---|---|---|---|---|---|
| 84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking. | FLUX.2 [klein] 9B | FP16 | 37.1 GB | 14 s | Runs well |
| 92Excellent quality (A+) · 92 of 100: how capable the model is at FP16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. | Krea 2 Turbo | FP16 | 37.4 GB | 34 s | Runs well |
| 89Excellent quality (A+) · 89 of 100: how capable the model is at FP16, whatever it runs on. Match score 85/100 (quality, fit, speed and memory use) sets the ranking. | Qwen-Image 2.1 Turbo | FP16 | 46.7 GB | 1.5 min | Slow |
| 92Excellent quality (A+) · 92 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. | Qwen-Image 2.1 | Q8 GGUF | 41.6 GB | 9.0 min | Very slow |
| 85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 83/100 (quality, fit, speed and memory use) sets the ranking. | ERNIE-Image Turbo | FP16 | 26.4 GB | 25 s | Runs well |
| 84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking. | Z-Image Turbo | FP16 | 22.7 GB | 20 s | Runs well |
Best Video models for the NVIDIA Jetson AGX Orin 64GB (64 GB)
| Quality | Model | Quant | Memory | Speed | Runs |
|---|---|---|---|---|---|
| 86Excellent quality (A+) · 86 of 100: how capable the model is at FP8, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking. | LTX-2.5 (22B, video + audio) | FP8 | 48.3 GB | 10.7 min | Runs well |
| 84Great quality (A) · 84 of 100: how capable the model is at FP8, whatever it runs on. Match score 81/100 (quality, fit, speed and memory use) sets the ranking. | LTX-2.3 | FP8 | 44.3 GB | 47.1 min | Very slow |
| 85Excellent quality (A+) · 85 of 100: how capable the model is at FP16, whatever it runs on. Match score 80/100 (quality, fit, speed and memory use) sets the ranking. | Bernini-R A14B (video generation + editing) | FP16 | 46.0 GB | 1.6 h | Very slow |
| 85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. | Wan 2.2 I2V A14B | Q8 GGUF | 39.3 GB | 4.0 h | Very slow |
| 85Excellent quality (A+) · 85 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 79/100 (quality, fit, speed and memory use) sets the ranking. | Wan 2.2 T2V A14B | Q8 GGUF | 39.3 GB | 4.0 h | Very slow |
Best Decision models for the NVIDIA Jetson AGX Orin 64GB (64 GB)
| Quality | Model | Quant | Memory | Speed | Runs |
|---|---|---|---|---|---|
| 86Excellent quality (A+) · 86 of 100: how capable the model is at F16, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking. | Jina Reranker v3.5 | F16 | 6.6 GB | 95 ms | Runs great |
| 84Great quality (A) · 84 of 100: how capable the model is at F16, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking. | Jina Reranker v3 | F16 | 6.6 GB | 95 ms | Runs great |
| 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. | FRIDA-Decisions | F16 | 2.4 GB | 73 ms | Runs great |
| 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. | GLiNER2.5 Decide | F16 | 1.7 GB | 45 ms | Runs great |
| 80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking. | Qwen3 Reranker 0.6B | F16 | 6.6 GB | 95 ms | Runs great |
| 76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. | Llama Prompt Guard 2 86M | F16 | 1.2 GB | 51 ms | Runs great |
| 76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking. | mxbai-rerank-base v2 | F16 | 6.2 GB | 78 ms | Runs great |
| 74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking. | GLiClass ModernBERT Large v3 | F16 | 1.5 GB | 38 ms | Runs great |
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.