Which memory a model actually lives in on a GeForce, a Mac or a mini PC, and what happens when it doesn’t fit.
A model has to sit in memory the processor can read very fast. Which memory that is depends on the machine.
Graphics cardVRAMGeForce, Radeon and Arc cards have their own memory. Fast, but fixed: a 16 GB card holds 16 GB. Two or more cards can pool theirs.
Mac, Ryzen AI, JetsonUnified memoryOne pool shared by CPU and GPU. A 64 GB Mac has far more room for models than a 16 GB graphics card, though the system reserves some, so we plan around about two-thirds to three-quarters of it.
Any PCSystem RAMPlentiful and cheap, but several times slower than VRAM. It is the overflow area when a model is too big for the card.
The same model on a PC and on a Mac
PC with a 16 GB graphics card and 32 GB of RAM
Graphics memory (16 GB)12 GB
System RAM (32 GB)0 GB
Fits in memory.The whole model sits in graphics memory and runs at full speed.
Mac with 64 GB of unified memory
Inside the GPU’s share (48 of 64 GB)12 GB
Past the GPU’s share0 GB
Fits in memory.The whole model sits inside the GPU’s share of memory.
ModelKept freeSpills over
Memory the model needs12 GB2 GB80 GB
An illustration using this site’s fit rules, with the memory a model needs standing in for the size of its weights. A real check uses the exact model, quantization and context length.
What happens when a model doesn’t fit
Apps based on llama.cpp (LM Studio, Ollama, Jan and others) can offload: keep some layers on the GPU and run the rest from system RAM. It works, but every token waits on the slower memory, so speed drops sharply. That is what the Slow and Very slow labels on this site mean. vLLM, SGLang and MLX need the whole model in GPU or unified memory.
Speed follows bandwidth
To write one token the processor reads the model’s weights once, so tokens per second is roughly memory bandwidth divided by model size. That is why a Mac with fast unified memory can beat a PC with more raw compute, and why mixture-of-experts models, which read only a few experts per token, run faster than their size suggests. Compare machines side by side on Compare, or see what yours can run on My hardware.
Was this guide useful?
Estimates come from published hardware specs and our open formulas, not from running the model on your machine. Real speed varies with runtime, drivers and settings. Numbers marked “Measured” come from a linked benchmark.
