How fast is an RTX 3060 12GB with Ollama?
Fast for everything that fits in its 12 GB: from 28.5 tokens per second for a 14B model up to 77.9 for a 3B. Most people read about 5 tokens per second.
| Model | Memory | RTX 3060, tokens/s | CPU only, tokens/s |
|---|---|---|---|
| Llama 3.2 3B Q4_K_M | 3.3 GB | 77.9 | 12.8 |
| Qwen2.5 7B Instruct Q4_K_M | 5.6 GB | 49.0 | 6.4 |
| Mistral 7B Q4_K_M | 5.8 GB | 57.8 | not run |
| Llama 3 8B Instruct Q4_0 | 5.8 GB | 50.3 | not run |
| Hermes 3 8B (Llama 3.1) Q4_0 | 5.8 GB | 49.9 | 5.7 |
| Qwen2.5 Coder 14B Q4_K_M | 10.4 GB | 28.5 | 3.5 |
| gpt-oss 20B MXFP4 | 14.9 GB | 6.5 (27% CPU / 73% GPU) | 2.4 |
| DeepSeek Coder 33B Q4_0 | 21.1 GB | 2.7 (47% CPU / 53% GPU) | 1.5 |
What is the best model for an RTX 3060 12GB?
For the best answers that still run fully on the card, a 14B model. For the quickest back-and-forth, a 7B or 8B model at about 50 tokens per second.
| If you want | Run | Speed |
|---|---|---|
| The fastest replies | Llama 3.2 3B | 77.9 tokens/s |
| Everyday chat and writing | A 7B or 8B model (Qwen2.5 7B, Mistral 7B, Llama 3 8B) | 49.0 to 57.8 tokens/s |
| The best answers that still fit | A 14B model (Qwen2.5 Coder 14B for code) | 28.5 tokens/s |
| A bigger model, and you can wait | gpt-oss 20B, split with system RAM | 6.5 tokens/s |
Pick by task as well as size. Qwen2.5 Coder is tuned for code. General-purpose 14B models are about the same size, so expect similar memory use and speed, but I haven’t measured them on this card yet.
Can an RTX 3060 run a 14B model?
Yes. Qwen2.5 Coder 14B used 10.4 GB, loaded 100% on the GPU and ran at 28.5 tokens per second.
It’s close to the limit. With 1.2 GB already used by the desktop and another program, 1.8 GB was still free with the model loaded. Raise the context length or open something else that uses the GPU, and the model will start splitting.
Can an RTX 3060 run gpt-oss 20B?
It runs, but not fully on the card. It needs 14.9 GB, so Ollama put 73% on the GPU and the rest in system RAM, and it generated 6.5 tokens per second.
The first answer took 65.4 seconds, including loading it from disk. If you want gpt-oss 20B at full speed, you need a 16 GB card.
Can an RTX 3060 run a 32B model?
Only slowly. DeepSeek Coder 33B needed 21.1 GB; 53% fit on the card and it generated 2.7 tokens per second.
That’s workable for a single question you can leave running, not for a conversation. For 32B models at chat speed you need 24 GB of VRAM.
How much faster is it than a CPU?
For models that fit, the RTX 3060 was 6 to 9 times faster than this machine’s CPU. When a model has to split, the gap narrows: gpt-oss 20B was 2.7 times faster split across the card than on the CPU alone.
The CPU here is a 2012 Xeon without AVX2, so a newer processor narrows the gap. The full CPU numbers are in Can you run a local LLM without a GPU?
How do I get the same numbers on my card?
Run the same script I used. It needs Python and Ollama, and prints one line per model.
$ curl -O https://raw.githubusercontent.com/Arynwood-Technology/local-ai-benchmarks/main/bench.py $ python3 bench.py llama3.2:3b qwen2.5:7b-instruct --csv my-results.csv
If your results differ, or you have a different card, send them in and they’ll be added with credit.
How I tested
Intel Xeon E5-2665 (2012, 8 cores, AVX but no AVX2), 62 GB RAM, NVIDIA GeForce RTX 3060 12 GB (driver 580.178.04), Ubuntu 24.04.5 LTS, Ollama 0.11.4 with its default 4,096-token context. Same prompt for every model, temperature 0, 200-token answers; the speed is the median of three warm runs, and CPU-only runs set num_gpu to 0. About 1.2 GB of the card’s memory was already in use by the desktop and a Stable Diffusion web UI, as on most real machines. Terminal output on this page was copied from the test machine. Full method: How we test. Raw data and the script: local-ai-benchmarks.
- September 24, 2026: first published, eight models on Ollama 0.11.4.