What AI models can the NVIDIA Jetson AGX Orin 32GB (32 GB) run?

Unified memory: 32.0 GB
Usable for models: 28.8 GB
Memory bandwidth: 205 GB/s
Runs models on: GPU (CUDA)
Released: 2022

464 / 622 LLMs run Best: Gemma 4 26B-A4B 60 / 61 Image models run Best: FLUX.2 [klein] 9B 39 / 43 Video models run Best: Bernini-R A14B (video generation + editing) 67 / 70 Decision models run Best: Jina Reranker v3.5

Models run on the GPU (CUDA) here, sharing the board’s memory with the system. We plan around 28.8 GB of the 32.0 GB (what Linux leaves free for a model). Ampere GPU with 1,792 CUDA cores and 56 Tensor Cores, plus two NVDLA v2 engines: up to 200 TOPS (INT8, sparse).

More about this hardware

Maker	NVIDIA
Type	Single-board computer or edge AI board
GPU share of memory	90% (what the GPU can typically be given for a model)
Chip	NVIDIA Jetson AGX Orin 32GB
AI accelerator	Ampere GPU with 1,792 CUDA cores and 56 Tensor Cores, plus two NVDLA v2 engines: up to 200 TOPS (INT8, sparse).
Boards with this chip	Jetson AGX Orin 32GB module
Memory type	32 GB LPDDR5, 256-bit
Bandwidth figure	204.8 GB/s, as the maker states it
Note	Estimate ~10% OS reserve.
Specifications	nvidia.com

Apps that run models well on the NVIDIA Jetson AGX Orin 32GB (32 GB)

Linux on ARM boards

For a typical model that fits in memory. Each model page picks the best app for that model on this hardware. All apps

Best LLMs for the NVIDIA Jetson AGX Orin 32GB (32 GB)

Quality	Model	Quant	Memory	Speed	Runs
91Excellent quality (A+) · 91 of 100: how capable the model is at Q6_K, whatever it runs on. Match score 96/100 (quality, fit, speed and memory use) sets the ranking.	Gemma 4 26B-A4BGoogle	`Q6_K`	24.5 GB	39.4 tok/s	Runs great
90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.	Qwen3.6 35B-A3BAlibaba	`Q5_K_M`	28.6 GB	55.7 tok/s	Runs welltight fit
90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.	MiniCPM-V 4.7 35B-A3BOpenBMB	`Q5_K_M`	28.1 GB	55.7 tok/s	Runs welltight fit
90Excellent quality (A+) · 90 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.	Qwen3.5 35B-A3BAlibaba	`Q5_K_M`	28.6 GB	55.7 tok/s	Runs welltight fit
89Excellent quality (A+) · 89 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 94/100 (quality, fit, speed and memory use) sets the ranking.	Nex N2 Mini (35B-A3B)Nex-AGI	`Q5_K_M`	28.0 GB	55.7 tok/s	Runs welltight fit
86Excellent quality (A+) · 86 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.	KAT-Coder V2.5 Dev (35B-A3B)Kuaishou	`Q4_K_M`	24.2 GB	65.4 tok/s	Runs great
88Excellent quality (A+) · 88 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.	Sage-A1.5 35B-A3BInternScience	`Q5_K_M`	28.6 GB	55.7 tok/s	Runs welltight fit
85Excellent quality (A+) · 85 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.	Raon-1 mini Preview (30B-A4B)KRAFTON	`Q5_K_M`	23.4 GB	41.3 tok/s	Runs great
87Excellent quality (A+) · 87 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 93/100 (quality, fit, speed and memory use) sets the ranking.	Ornith 1.5 35B-A3BDeepReinforce	`Q5_K_M`	28.6 GB	55.7 tok/s	Runs welltight fit
86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.	Apodex 1.1 Mini (35B-A3B)Apodex	`Q5_K_M`	28.6 GB	55.7 tok/s	Runs welltight fit
86Excellent quality (A+) · 86 of 100: how capable the model is at Q5_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.	Ornith 1.0 35B-A3BDeepReinforce	`Q5_K_M`	28.6 GB	55.7 tok/s	Runs welltight fit
83Great quality (A) · 83 of 100: how capable the model is at Q4_K_M, whatever it runs on. Match score 92/100 (quality, fit, speed and memory use) sets the ranking.	GLM-4.7 Flash (30B-A3B)Zhipu AI	`Q4_K_M`	22.4 GB	65.4 tok/s	Runs great

Best Image models for the NVIDIA Jetson AGX Orin 32GB (32 GB)

Quality	Model	Quant	Memory	Speed	Runs
82Great quality (A) · 82 of 100: how capable the model is at FP8, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	FLUX.2 [klein] 9BBlack Forest Labs	`FP8`	19.9 GB	18 s	Runs well
90Excellent quality (A+) · 90 of 100: how capable the model is at FP8, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	Krea 2 TurboKrea	`FP8`	20.2 GB	43 s	Slow
91Excellent quality (A+) · 91 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	FLUX 3 Image KleinBlack Forest Labs	`FP16`	22.4 GB	64 s	Slow
84Great quality (A) · 84 of 100: how capable the model is at FP16, whatever it runs on. Match score 86/100 (quality, fit, speed and memory use) sets the ranking.	Z-Image TurboAlibaba (Tongyi)	`FP16`	22.7 GB	28 s	Runs well
93Excellent quality (A+) · 93 of 100: how capable the model is at FP16, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	Fibo 2.0 DevBria AI	`FP16`	27.7 GB	4.2 min	Very slowtight fit
76Great quality (A) · 76 of 100: how capable the model is at FP8, whatever it runs on. Match score 82/100 (quality, fit, speed and memory use) sets the ranking.	FLUX.2 [klein] 4BBlack Forest Labs	`FP8`	10.2 GB	10 s	Runs great

Best Video models for the NVIDIA Jetson AGX Orin 32GB (32 GB)

Quality	Model	Quant	Memory	Speed	Runs
83Great quality (A) · 83 of 100: how capable the model is at FP8, whatever it runs on. Match score 77/100 (quality, fit, speed and memory use) sets the ranking.	Bernini-R A14B (video generation + editing)ByteDance	`FP8`	26.0 GB	2.0 h	Very slowtight fit
81Great quality (A) · 81 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 75/100 (quality, fit, speed and memory use) sets the ranking.	LTX-2.5 (22B, video + audio)Lightricks	`Q4 GGUF`	26.8 GB	21.4 min	Slowtight fit · encoder unloaded after use
74Great quality (A) · 74 of 100: how capable the model is at FP8, whatever it runs on. Match score 74/100 (quality, fit, speed and memory use) sets the ranking.	HY-OmniWeaving (8.3B, unified video)Tencent	`FP8`	24.6 GB	89.0 min	Very slowtight fit
70Great quality (A) · 70 of 100: how capable the model is at Q4 GGUF, whatever it runs on. Match score 73/100 (quality, fit, speed and memory use) sets the ranking.	Wan 2.2 S2V 14B (speech to video)Alibaba	`Q4 GGUF`	21.6 GB	3.7 h	Very slow
73Great quality (A) · 73 of 100: how capable the model is at Q8 GGUF, whatever it runs on. Match score 72/100 (quality, fit, speed and memory use) sets the ranking.	Wan 2.1 I2V 14B 480PAlibaba	`Q8 GGUF`	27.2 GB	3.2 h	Very slowtight fit

Best Decision models for the NVIDIA Jetson AGX Orin 32GB (32 GB)

Quality	Model	Quant	Memory	Speed	Runs
86Excellent quality (A+) · 86 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.	Jina Reranker v3.5Jina AI	`Q8_0`	3.8 GB	138 ms	Runs well
80Great quality (A) · 80 of 100: how capable the model is at F16, whatever it runs on. Match score 91/100 (quality, fit, speed and memory use) sets the ranking.	GLiNER2.5 DecideFastino	`F16`	1.7 GB	70 ms	Runs great
80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.	FRIDA-DecisionsSber AI	`Q8_0`	1.7 GB	100 ms	Runs great
84Great quality (A) · 84 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 90/100 (quality, fit, speed and memory use) sets the ranking.	Jina Reranker v3Jina AI	`Q8_0`	3.8 GB	138 ms	Runs well
76Great quality (A) · 76 of 100: how capable the model is at F16, whatever it runs on. Match score 89/100 (quality, fit, speed and memory use) sets the ranking.	Llama Prompt Guard 2 86MMeta	`F16`	1.2 GB	79 ms	Runs great
80Great quality (A) · 80 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	Qwen3 Reranker 0.6BAlibaba	`Q8_0`	3.8 GB	138 ms	Runs well
74Great quality (A) · 74 of 100: how capable the model is at F16, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	GLiClass ModernBERT Large v3Knowledgator	`F16`	1.5 GB	58 ms	Runs great
76Great quality (A) · 76 of 100: how capable the model is at Q8_0, whatever it runs on. Match score 88/100 (quality, fit, speed and memory use) sets the ranking.	mxbai-rerank-base v2Mixedbread	`Q8_0`	3.4 GB	113 ms	Runs great

Best LLMs by task on this hardware Open in Model Finder

Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.