Apps for running AI locally

Which app should you use?

Our first pick for common setups. The same rules run on every model page with your exact hardware. These picks are model-running apps; to edit code or run tasks with the model, add one of the coding agents, assistants or reverse-engineering plugins on top.

Common setups

Example computer	Small model that fits8B	Bigger than GPU memory70B	Serving a team32B
Mac with Apple silicon	LM Studio Runs the MLX build, typically 20–35% faster than GGUF for a model with 8B active parameters on Apple silicon.	LM Studio Splits the model between GPU memory and system RAM, which this setup needs.	LM Studio Runs the MLX build, typically up to 10% faster than GGUF for a model with 32B active parameters on Apple silicon.
Windows PC, NVIDIA GeForce	Ollama One command with Ollama’s own build of this model.	LM Studio Splits the model between GPU memory and system RAM, which this setup needs.	Ollama One command with Ollama’s own build of this model.
Windows PC, AMD Radeon	Lemonade AMD’s own app: llama.cpp tuned for Radeon on Vulkan and ROCm.	llama.cpp Splits the model between GPU memory and system RAM, which this setup needs.	Lemonade AMD’s own app: llama.cpp tuned for Radeon on Vulkan and ROCm.
Windows PC, Intel Arc	llama.cpp Supports Intel Arc through SYCL and Vulkan.	llama.cpp Splits the model between GPU memory and system RAM, which this setup needs.	llama.cpp Supports Intel Arc through SYCL and Vulkan.
Ryzen AI mini PC or laptop	Lemonade Built for Ryzen AI and Strix Halo, including NPU + iGPU modes on Windows.	Lemonade Built for Ryzen AI and Strix Halo, including NPU + iGPU modes on Windows.	Lemonade Built for Ryzen AI and Strix Halo, including NPU + iGPU modes on Windows.
Linux, NVIDIA GeForce	Ollama One command with Ollama’s own build of this model.	LM Studio Splits the model between GPU memory and system RAM, which this setup needs.	Ollama One command with Ollama’s own build of this model.
Linux server, datacenter GPUs	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.
NVIDIA Jetson board	Ollama One command with Ollama’s own build of this model.	LM Studio Splits the model between GPU memory and system RAM, which this setup needs.	Ollama One command with Ollama’s own build of this model.
Raspberry Pi or ARM board	Ollama One command with Ollama’s own build of this model.	LM Studio Splits the model between GPU memory and system RAM, which this setup needs.	Ollama One command with Ollama’s own build of this model.

Which app for decision models?

Scorers that read the input once (rerankers, classifiers, reward models) belong in a scoring server or a Python library; guard and judge models that write a short answer run in the same apps as LLMs. Agents are not listed here: they call a model server and do not run decision models themselves.

Common setups

Example computer	Safety checkQwen3Guard 4B	RerankerBGE Reranker v2 M3	Zero-shot classifierDeBERTa	Reward judgeSkywork 8B	Label decisionsGLiNER2.5 Decide
Mac with Apple silicon	LM Studio Point-and-click: search, download and chat in one window.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Infinity Local API for rerankers and classifiers, including models TEI does not support.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.
Windows PC, NVIDIA GeForce	LM Studio Point-and-click: search, download and chat in one window.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.
Windows PC, AMD Radeon	Lemonade AMD’s own app: llama.cpp tuned for Radeon on Vulkan and ROCm.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	GLiNER Can only use the CPU on this hardware, so it will be slow.
Windows PC, Intel Arc	llama.cpp Supports Intel Arc through SYCL and Vulkan.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Sentence Transformers Can only use the CPU on this hardware, so it will be slow.	vLLM Serves the scoring head directly (rerank, classify or reward) with batching.	GLiNER Can only use the CPU on this hardware, so it will be slow.
Linux, NVIDIA GeForce	LM Studio Point-and-click: search, download and chat in one window.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Infinity Local API for rerankers and classifiers, including models TEI does not support.	vLLM Serves the scoring head directly (rerank, classify or reward) with batching.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.
Linux server, datacenter GPUs	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Infinity Local API for rerankers and classifiers, including models TEI does not support.	vLLM Serves the scoring head directly (rerank, classify or reward) with batching.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.
NVIDIA Jetson board	vLLM The full model fits in GPU memory at 8-bit, so it can batch many requests: best for serving a team or an app.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Infinity Local API for rerankers and classifiers, including models TEI does not support.	vLLM Serves the scoring head directly (rerank, classify or reward) with batching.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.
Raspberry Pi or ARM board	LM Studio Point-and-click: search, download and chat in one window.	llama.cpp Serves the GGUF build behind a /v1/rerank endpoint with llama-server --rerank.	Infinity Local API for rerankers and classifiers, including models TEI does not support.	Sentence Transformers A few lines of Python with Transformers or Sentence Transformers; easiest for scripts and notebooks.	GLiNER Made for this model family: every label is scored in one pass, fast even on a CPU.

Compare every app

75 apps: 41 that run models and 34 agents and assistants that use them. Filter by type, operating system, GPU or file format from the column headers. Agents show “Via server” for GPUs because they use whatever your model server runs on.

Show all apps

App	Type	Runs	Systems	GPUs	Formats	Engine	Models	Stars	License
LM StudioElement Labs	Desktop app	LLMs, Decisions	macOSWindowsLinuxLinux on ARM boards	Apple, MLX, NVIDIA, Vulkan, AMD, CPU	GGUF, MLX	llama.cpp, MLX	604	5,412	Free, closed
OllamaOllama	CLI + API	LLMs, Decisions	macOSWindowsLinuxLinux on ARM boards	Apple, MLX, NVIDIA, AMD, Vulkan, CPU	GGUF	llama.cpp, MLX	590	186,373	Open source
llama.cppggml.org	CLI + API	LLMs, Decisions	macOSWindowsLinuxLinux on ARM boards	Apple, NVIDIA, AMD, Vulkan, Intel, CPU	GGUF	llama.cpp	593	133,757	Open source
JanMenlo Research	Desktop app	LLMs, Decisions	macOSWindowsLinux	Apple, NVIDIA, Vulkan, CPU, MLX	GGUF	llama.cpp, MLX	604	45,605	Open source
MLX LMApple (ml-explore)	CLI + API	LLMs, Decisions	macOS	MLX	MLX	MLX	417	7,595	Open source
vLLMvLLM project	Server	LLMs, Decisions	WSLWindowsLinuxLinux on ARM boards	NVIDIA, AMD, Intel	Safetensors, AWQ, GPTQ, FP8	vLLM	665	96,557	Open source
SGLangLMSYS / SGLang project	Server	LLMs, Decisions	WSLWindowsLinuxLinux on ARM boards	NVIDIA, AMD	Safetensors, AWQ, GPTQ, FP8	SGLang	655	38,584	Open source
Docker Model RunnerDocker	CLI + API	LLMs, Decisions	macOSWindowsLinuxLinux on ARM boards	Apple, NVIDIA, AMD, Vulkan, CPU	GGUF, Safetensors	llama.cpp, vLLM	648	659	Open source
LemonadeAMD (lemonade-sdk)	Desktop app	LLMs, Decisions	macOSWindowsLinux	Vulkan, AMD, NPU, Apple, CPU	GGUF, ONNX	llama.cpp, ONNX Runtime GenAI	593	6,147	Open source
Unsloth DesktopUnsloth AI	Desktop app	LLMs, Images, Decisions	macOSWindowsLinux	NVIDIA, AMD, Intel, Apple, MLX, CPU	GGUF, MLX	llama.cpp, MLX, stable-diffusion.cpp	630	79,201	Open source
Atomic ChatAtomicBot	Desktop app	LLMs, Decisions	macOSWindowsLinux	Apple, MLX, NVIDIA, Vulkan, CPU	GGUF, MLX	llama.cpp, MLX-VLM	604	1,961	Open source
PiEarendil	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	120,829	Open source
Hermes AgentNous Research	Assistant	LLMs	macOSWSLWindowsLinux	Via server	GGUF, MLX	Your model server	238	261,739	Open source
OpenClawOpenClaw	Assistant	LLMs	macOSWSLWindowsLinux	Via server	GGUF, MLX	Your model server	238	394,214	Open source
OpenCodeAnomaly	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	225,512	Open source
Codex CLIOpenAI	Coding agent	LLMs	macOSWSLWindowsLinux	Via server	GGUF, MLX	Your model server	238	133,798	Open source
OpenHandsOpenHands	Coding agent	LLMs	macOSWSLWindowsLinux	Via server	GGUF, MLX	Your model server	238	92,970	Open source
ZedZed Industries	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	94,469	Open source
ClineCline	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	71,473	Open source
Open InterpreterOpen Interpreter	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	68,841	Open source
GooseAgentic AI Foundation	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	56,371	Open source
AiderAider	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	50,026	Open source
TabbyTabbyML	Coding agent	LLMs	macOSWSLWindowsLinux	Own engine	GGUF, MLX	runs its own models	—	33,925	Open source
CrushCharm	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	29,004	Source-available
Qwen CodeQwen	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	29,105	Open source
Kilo CodeKilo Code	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	27,909	Open source
Open WebUIOpen WebUI	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	158,130	Source-available
AnythingLLMMintplex Labs	Assistant	LLMs	macOSWindowsLinux	Own engine	GGUF, MLX	built-in engine	—	68,067	Open source
KhojKhoj	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	38,069	Open source
LettaLetta	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	25,388	Open source
ComfyUIComfy Org	Desktop app	Images, Video	macOSWindowsLinuxLinux on ARM boards	NVIDIA, AMD, Intel, Apple, CPU	Safetensors, GGUF	PyTorch, GGUF via ComfyUI-GGUF	100	145,785	Open source
Draw ThingsDraw Things	Desktop app	Images, Video	macOSiOS	Apple	Safetensors, Draw Things format	Draw Things	52	614	Free, closed
DiffusionBeeDivam Gupta	Desktop app	Images	macOS	Apple	Safetensors	DiffusionBee	14	13,589	Open source
JoyFusionJoyFusion	Desktop app	Images	macOS	Core ML	Core ML	Core ML	4	—	Free, closed
Text Embeddings InferenceHugging Face	Server	Decisions	macOSWSLWindowsLinux	NVIDIA, Apple, CPU	Safetensors	TEI	6	5,113	Open source
InfinityMichael Feil	Server	Decisions	macOSWSLWindowsLinuxLinux on ARM boards	NVIDIA, AMD, Apple, CPU	Safetensors, ONNX	PyTorch, ONNX, CTranslate2	12	2,984	Open source
LocalAIEttore Di Giacinto (mudler)	Server	Decisions	macOSWSLWindowsLinuxLinux on ARM boards	NVIDIA, AMD, Intel, Vulkan, Apple, CPU	GGUF, Safetensors	llama.cpp, rerankers, Transformers	30	50,258	Open source
Sentence TransformersHugging Face	Library	Decisions	macOSWindowsLinuxLinux on ARM boards	NVIDIA, AMD, Apple, CPU	Safetensors, ONNX	PyTorch	64	19,266	Open source
GLiNERUrchade Zaratiana, Fastino and Knowledgator	Library	Decisions	macOSWindowsLinuxLinux on ARM boards	NVIDIA, Apple, CPU	Safetensors, ONNX	PyTorch, ONNX	6	4,236	Open source
GhidrAssistJason Tang (symgraph)	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	14	779	Open source
ReVacyberkaida	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	5	952	Open source
GhidraMCPLaurieWired	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	5	11,844	Open source
OGhidraLawrence Livermore National Laboratory	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	5	466	Open source
GepettoJusticeRage	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	14	3,485	Open source
IDA Pro MCPmrexodia	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	5	13,413	Open source
BinAssistJason Tang (symgraph)	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	14	68	Open source
r2ai and decairadare2 project	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	14	536	Open source
JADX AI MCPzinja-coder	RE plugin	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	5	2,996	Open source
GPT4AllNomic AI	Desktop app	LLMs	macOSWindowsLinux	Vulkan, NVIDIA, Apple, CPU	GGUF	llama.cpp	287	77,347	Open source
KoboldCppLostRuins (Concedo)	Desktop app	LLMs	macOSWindowsLinux	NVIDIA, Vulkan, Apple, CPU	GGUF	llama.cpp	560	12,194	Open source
Text Generation WebUIoobabooga	Desktop app	LLMs	macOSWindowsLinux	NVIDIA, AMD, Vulkan, Apple, CPU	GGUF, Safetensors	llama.cpp, Transformers, ExLlamaV3	619	47,734	Open source
llamafileMozilla.ai	CLI + API	LLMs	macOSWindowsLinuxLinux on ARM boards	NVIDIA, Apple, CPU	GGUF	llama.cpp	560	26,588	Open source
Msty StudioMsty	Desktop app	LLMs	macOSWindowsLinux	Apple, MLX, NVIDIA, AMD, CPU	GGUF, MLX	Ollama, llama.cpp, MLX	579	—	Free, closed
Llama (macOS)ggml-org	Desktop app	LLMs	macOS	Apple	GGUF	llama.cpp	560	1,752	Open source
RecurseChatRecurseChat	Desktop app	LLMs	macOS	Apple	GGUF	llama.cpp	560	—	Free, closed
InferencerInferencer Labs	Desktop app	LLMs	macOS	MLX	MLX	MLX	413	—	Free, closed
node-llama-cppwithcat.ai	Library	LLMs	macOSWindowsLinuxLinux on ARM boards	Apple, NVIDIA, Vulkan, CPU	GGUF	llama.cpp	560	2,200	Open source
exoExo Labs	Server	LLMs	macOSLinux	MLX, CPU	MLX	MLX	413	48,251	Open source
InvokeInvoke	Desktop app	Images	macOSWindowsLinux	NVIDIA, AMD, Apple, CPU	Safetensors, GGUF	PyTorch	29	29,023	Open source
SwarmUImcmonkeyprojects	Desktop app	Images, Video	macOSWindowsLinux	NVIDIA, AMD, Intel, Apple, CPU	Safetensors, GGUF	ComfyUI	100	4,752	Open source
Forge NeoHaoming02	Desktop app	Images	WindowsLinux	NVIDIA, AMD	Safetensors, GGUF	PyTorch	30	1,891	Open source
Fooocuslllyasviel	Desktop app	Images	macOSWindowsLinux	NVIDIA, AMD, Apple, CPU	Safetensors	PyTorch	9	53,844	Open source
SD.Nextvladmandic	Desktop app	Images, Video	macOSWindowsLinux	NVIDIA, AMD, Intel, Apple, CPU	Safetensors, GGUF	PyTorch	100	7,373	Open source
Stability MatrixLykos AI	Desktop app	Images, Video	macOSWindowsLinux	NVIDIA, AMD, Intel, Apple, CPU	Safetensors, GGUF	installs ComfyUI, Forge, InvokeAI, SwarmUI, SD.Next or Fooocus	100	8,971	Open source
Krita AI DiffusionAcly	Desktop app	Images	macOSWindowsLinux	NVIDIA, AMD, Apple, CPU	Safetensors, GGUF	ComfyUI	21	10,816	Open source
WanGPdeepbeepmeep	Desktop app	Video	WindowsLinux	NVIDIA, AMD	Safetensors	PyTorch	21	11,277	Open source
ContinueContinue	Coding agent	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	36,385	Open source
SillyTavernSillyTavern	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	35,163	Open source
LibreChatLibreChat	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	46,585	Open source
Cherry StudioCherryHQ	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	53,134	Open source
ChatboxChatbox AI	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	42,211	Open source
PrivateGPTZylon	Assistant	LLMs	macOSWindowsLinux	Via server	GGUF, MLX	Your model server	238	57,624	Open source
PocketPal AIAsghar Ghorbani	Mobile app	LLMs	iOSAndroid	Apple, OpenCL, CPU	GGUF	llama.cpp	560	8,952	Open source
ChatterUIVali-98	Mobile app	LLMs	Android	CPU	GGUF	llama.cpp	560	2,865	Open source
LLM Farmguinmoon	Mobile app	LLMs	iOS	Apple, CPU	GGUF	llama.cpp	560	2,079	Open source

How we choose an app

1. The files that exist for the model. GGUF opens in llama.cpp, Ollama, LM Studio, Jan, Lemonade, Docker, KoboldCpp, llamafile, Msty and others; MLX opens in LM Studio, MLX LM, Inferencer and others on Apple silicon; the original safetensors weights need vLLM, SGLang or Text Generation WebUI. We check Hugging Face and Ollama for every model, the same signals Hugging Face uses for its “Use this model” menu.
2. Your operating system and GPU. MLX is Mac-only; vLLM and SGLang are Linux (Windows through WSL2); Lemonade is built for AMD Radeon and Ryzen AI; llama.cpp has the broadest GPU support (CUDA, ROCm, Vulkan, SYCL, Metal).
3. Whether the model fits. If part of it has to sit in system RAM, only llama.cpp-based apps can split it; vLLM, SGLang and MLX need it all in GPU or unified memory.
4. Published speed tests. On an M4 Max, MLX generated 21–87% faster than llama.cpp for 4-bit models up to 8B and tied around 27B; llama.cpp still prefills long prompts faster. On an RTX 4090, vLLM (54 tok/s) and llama.cpp (50 tok/s) were close for one user, Ollama’s draft decoding reached 76 tok/s, and with spare memory vLLM handled 32 users far better.
5. Decision models. Guard and judge models that write a short answer follow the LLM rules above. Rerankers, classifiers and reward models only score their input, so they go to a scoring server (Text Embeddings Inference for BERT, XLM-RoBERTa and ModernBERT models; Infinity for DeBERTa; vLLM or SGLang with big GPUs), to llama.cpp when a GGUF build exists, or to a few lines of Python. GLiNER-family models need the GLiNER libraries. Ollama has no rerank endpoint, so it only runs the guard and judge models.
6. Agents, assistants and plugins. Coding agents, personal assistants, reverse-engineering plugins and chat front ends are not ranked against the apps above, because they do not run a model: they send requests to one. We list them by GitHub stars and show which need a model server and which, like Tabby and AnythingLLM, bring their own. GPT4All only gets models released up to mid-2025, because its engine has not been updated since. For a local model to work as an agent it must support tool calling.
7. Ease and popularity. Point-and-click apps rank above command-line tools for one person; GitHub stars and maintenance break ties.

Speeds vary with versions, drivers and settings.