How much VRAM does each model size need?
These are the sizes Ollama reported for each model loaded on the test machine. They include the model and the memory reserved for its default context.
| Model | Parameters | Quantization | Memory | GB per billion | Fit in 12 GB? |
|---|---|---|---|---|---|
| Llama 3.2 3B | 3.2B | Q4_K_M | 3.3 GB | 1.03 | Yes |
| Qwen2.5 7B Instruct | 7.6B | Q4_K_M | 5.6 GB | 0.74 | Yes |
| Mistral 7B | 7.2B | Q4_K_M | 5.8 GB | 0.81 | Yes |
| Llama 3 8B Instruct | 8B | Q4_0 | 5.8 GB | 0.72 | Yes |
| Hermes 3 8B (Llama 3.1) | 8B | Q4_0 | 5.8 GB | 0.72 | Yes |
| Qwen2.5 Coder 14B | 14.8B | Q4_K_M | 10.4 GB | 0.70 | Yes |
| gpt-oss 20B | 20.9B | MXFP4 | 14.9 GB | 0.71 | No, 27% ran from system RAM |
| DeepSeek Coder 33B | 33B | Q4_0 | 21.1 GB | 0.64 | No, 47% ran from system RAM |
Most models on Ollama download at a 4-bit quantization by default (Q4_K_M or Q4_0), which is what these numbers use. An 8-bit or full-precision download of the same model needs roughly twice or four times as much. gpt-oss uses its own 4-bit format, MXFP4.
How much VRAM does a 7B or 8B model need?
5.6 to 5.8 GB at the default 4-bit quantization. An 8 GB card runs them fully on the GPU, with a little room left over.
On the RTX 3060 the four 7B and 8B models I tested all loaded 100% on the GPU and generated 49.0 to 57.8 tokens per second.
How much VRAM does a 14B model need?
Qwen2.5 Coder 14B used 10.4 GB. It fits on a 12 GB card, but only if little else is using the GPU.
With 1.2 GB already taken by the desktop and another app, it still loaded 100% on the GPU and ran at 28.5 tokens per second. A bigger context or a second program on the GPU would push it into a split. On an 8 GB card, expect a split.
How much VRAM does gpt-oss 20B need?
14.9 GB with the default context. That is more than a 12 GB card has, so it needs a 16 GB card to run entirely on the GPU.
On the RTX 3060 Ollama put 73% of it on the GPU and the rest in system RAM, and it generated 6.5 tokens per second, while running it on the CPU alone gave 2.4.
How much VRAM does a 32B or 33B model need?
About 21.1 GB at 4-bit for DeepSeek Coder 33B, the size class of today’s 32B models. You need a 24 GB card to run it fully on the GPU.
On the 12 GB card only 53% fit, and it managed 2.7 tokens per second, against 1.5 on the CPU alone. It works for a question you can wait on, not for chat.
What happens if the model doesn’t fit?
Ollama doesn’t refuse to run it. It splits the model between the GPU and system RAM, and the part in system RAM sets the pace.
You can see the split in ollama ps, for example 27%/73% CPU/GPU. In these tests the 14B model, which fit, ran at 28.5 tokens per second, while the larger gpt-oss 20B, with 73% of it on the GPU, ran at 6.5. The GPU check guide shows how to read it and what to do about it.
Does a longer context need more VRAM?
Yes. For Qwen2.5 7B, raising the context from 2,048 to 32,768 tokens took the memory from 5.4 GB to 8.9 GB.
The context is how much text the model can keep in view: your prompt, the conversation so far and any documents you paste in. Ollama sets it with the num_ctx option and reserves memory for it up front, which is why a model that fits at the default can stop fitting when you raise it.
| Context length | Memory | Where it ran |
|---|---|---|
| 2,048 tokens | 5.4 GB | 100% GPU |
| 4,096 tokens | 5.6 GB | 100% GPU |
| 8,192 tokens | 6.0 GB | 100% GPU |
| 16,384 tokens | 7.0 GB | 100% GPU |
| 32,768 tokens | 8.9 GB | 100% GPU |
How much RAM do I need to run it on the CPU?
About the same as the VRAM figures above, or a little less, in system RAM, plus several gigabytes for your operating system and apps. A 3B model is comfortable with 8 GB of RAM; a 7B or 8B model wants 16 GB.
On the CPU, Ollama reported 5.0 GB for the 8B model (against 5.8 GB on the GPU), 9.5 GB for the 14B and 13.3 GB for gpt-oss 20B.
Speed is the bigger limit without a GPU. See Can you run a local LLM without a GPU? for CPU-only speeds from 1B to 33B on this machine.
How do I check how much VRAM I have?
On Linux with an NVIDIA card, run nvidia-smi --query-gpu=memory.used,memory.total --format=csv. Subtract the used memory from the total to see what a model can have.
$ nvidia-smi --query-gpu=memory.used,memory.total --format=csv
memory.used [MiB], memory.total [MiB]
1101 MiB, 12288 MiB
That was the test machine before loading any model: 1.2 GB already in use by the desktop and another program, leaving 11.7 GB for Ollama.
How I tested
Intel Xeon E5-2665 (2012, 8 cores, AVX but no AVX2), 62 GB RAM, NVIDIA GeForce RTX 3060 12 GB (driver 580.178.04), Ubuntu 24.04.5 LTS, Ollama 0.11.4 with its default 4,096-token context. Same prompt for every model, temperature 0, 200-token answers; the speed is the median of three warm runs, and CPU-only runs set num_gpu to 0. About 1.2 GB of the card’s memory was already in use by the desktop and a Stable Diffusion web UI, as on most real machines. Terminal output on this page was copied from the test machine. Full method: How we test. Raw data and the script: local-ai-benchmarks.
- September 24, 2026: first published, with sizes from Ollama 0.11.4.