On This Page
  • What Is the Simple VRAM Formula?
  • How Much VRAM Do Common Models Need?
  • Why a Model Fits on Paper but Runs Out of Memory
  • What If Your Card Is Slightly Too Small?
  • FAQs About Vram and LLM

As a starting point, you need about 0.6 GB of VRAM per billion parameters at 4-bit quantization, plus 2 to 3 GB for context and runtime overhead. That puts an 8B model near 7 GB and a 32B model near 23 GB. Long context changes the answer more than most people expect.

What Is the Simple VRAM Formula?

VRAM needed for an LLM equals model weights plus KV cache plus runtime overhead. Weights are the easy part: multiply billions of parameters by bytes per parameter.

Precision

Bytes per parameter

VRAM per billion parameters

FP16 / BF16

2

2 GB

Q8

1

1 GB

Q5

~0.7

0.7 GB

Q4_K_M

~0.6

0.6 GB

Start at Q4_K_M for almost every case. It cuts weight memory by about 4x against FP16, and most people cannot detect the quality loss outside of heavy reasoning or code.

How Much VRAM Do Common Models Need?

This table assumes Q4_K_M, an 8K context, and runtime overhead included. Treat the numbers as starting points, because your exact figure moves with quantization and context length.

Model

Parameters

VRAM at Q4, 8K context

Llama 3.1 8B

8B

~7 GB

Gemma 3 12B

12B

~10 GB

Qwen3 14B

14B

~12 GB

Mistral Small 3.2

24B

~17 GB

Qwen3 32B

32B

~23 GB

Llama 3.3 70B

70B

~45 GB

An 8 GB card covers the 8B tier. 12 GB handles 12B to 14B models. 24 GB is the practical sweet spot, since it runs 32B models with headroom for context. A 70B model needs 48 GB, or two 24 GB cards.

Why a Model Fits on Paper but Runs Out of Memory

Model weights are only half the story. The KV cache grows with context length, and at long context it can cost more than the model itself.

Llama 3.1 8B at Q4 needs about 4.9 GB for weights. At a 128K context, its FP16 KV cache adds roughly 16 GB, for a 21 GB total. That does not fit on a 16 GB card.

Context 8B-class KV cache 70B-class KV cache
8K ~ 1 GB ~ 2.5 GB
32K ~ 4 GB ~ 10 GB
128K ~ 16 GB ~ 40 GB

Two traps follow. An advertised 128K window is often not native and may depend on rope scaling settings. Mixture-of-experts models advertise small active parameters, but every expert must still sit in memory, so size them by total parameters.

What If Your Card Is Slightly Too Small?

If the model just misses, change one variable at a time. Drop a quantization level, cap the context window, or quantize the KV cache to 8-bit, which roughly halves its size.

Do not exceed VRAM by a little and expect a small slowdown. Once layers spill into system RAM, speed falls five to twenty times. Ollama shows the split in ollama ps. A 24 GB card from an older generation will run more models than a faster 16 GB card, so memory matters more than speed.

FAQs About Vram and LLM

Yes, for 12B to 14B models at Q4 with moderate context. It is the practical entry point for daily use. Expect to cap context rather than run 32K sessions.
Usually yes. For the same memory budget, parameter count buys more capability than precision does, and Q4_K_M loss is small.
Apple Silicon shares one pool of memory between the CPU and GPU, so a 64 GB Mac can hold a 70B model that no single consumer card fits. Usable memory runs about three quarters of total, and speed depends on bandwidth rather than capacity.
Read the GGUF file size for weights, add the KV cache for your context, then add 2 to 3 GB. iBoomto's hardware check runs the same math and gives you a plain-language verdict before you download a model.