LLM VRAM math: weights, context and KV cache

Data reviewed September 13, 2026 · By Weerasak · Methodology

A downloaded model file is only one part of a running model’s GPU allocation. This guide separates the terms and uses exactly the same arithmetic as the LLM VRAM calculator. The examples are calculations, not measurements.

1. Estimate the weight storage

Weights (GiB) = parameters × effective bits per weight ÷ 8 ÷ 1,073,741,824

Using a nominal 8 billion parameters and the 4.85-bits-per-weight planning preset gives 4.52 GiB. “Q4” does not mean every tensor occupies exactly four bits: block scales and mixed tensor formats change the effective size. Our presets are approximations. When you know the complete file size, enter its bytes divided by 1,073,741,824 as a weight-size proxy.

2. Calculate the full KV cache

KV (GiB) = 2 × layers × KV heads × head dimension × allocated tokens × bytes per value ÷ 1,073,741,824

The factor of two accounts for keys and values. For Llama 3.1 8B, the profile uses 32 layers, 8 KV heads and head dimension 128. At 8,192 tokens with FP16 cache: 2 × 32 × 8 × 128 × 8192 × 2 bytes = exactly 1.00 GiB. At 32,768 tokens, that term is 4.00 GiB. See Meta’s model paper for the architecture.

Use the model’s KV-head count, not automatically its attention-head count. Also use an explicit head dimension where provided: Gemma 2’s 9B configuration uses 256-wide heads. Dividing its hidden size by its number of attention heads gives the wrong value. We show a full-cache upper bound for Gemma 2 rather than assuming every backend saves the same memory from local attention.

3. Account for cache format

For ggml-style cache blocks, FP16 uses 2 bytes per value, Q8_0 uses 34/32 bytes per value and Q4_0 uses 18/32 bytes per value, including their block scales. Thus the 32K Llama example uses 2.125 GiB of Q8_0 cache before runtime padding, not exactly 2 GiB. See the storage definitions. Cache-format support and quality effects depend on the backend and model.

4. Choose, and label, the uncertain allowance

Total estimate = (weights + KV cache) × (1 + extra margin / 100) + runtime/desktop allowance

The defaults are 10% extra margin and a 2 GiB allowance. Both are SpecsCalc planning assumptions. They are not benchmark-derived constants, an operating-system specification or proof that an allocation will succeed. Change them when you have measurements for your intended backend, prompt-processing batch and background GPU use.

Compare the worked examples

Moving the 8B example from 8K to 32K adds 3 GiB of FP16 cache, plus the selected percentage margin. Changing the weight quantization does not change the cache format. Changing the cache format does not change the model’s weights. The 14B row uses its own layer/head configuration instead of scaling the 8B result by parameter count alone.

Load these examples in the calculator to change one input at a time. Keep the other inputs fixed when comparing context lengths.

Where this formula stops

This is a single-sequence, single-GPU planning model. Multiple parallel requests, vision inputs, training, unusual attention architectures, partial offload and multi-GPU placement need separate accounting. A context above the selected profile’s documented range is flagged as unverified even if the arithmetic total is small. Runtime configuration and model support must both permit it.

Verify the estimate in Ollama

Load the exact model you intend to use and check Ollama’s current context settings. After a request, run ollama ps. Compare the active context with the calculator input and inspect PROCESSOR for GPU/CPU placement. The displayed model size is not a substitute for a peak-memory measurement.

Repeat with your intended prompt and output length. If a longer context causes trouble, compare a smaller allocated context while keeping the model and other settings fixed. Cache quantization changes are backend-dependent; check Ollama’s requirements for quantized KV cache before choosing Q8 or Q4 here. The calculator only compares the storage assumptions; it does not enable those settings in your runtime.

Sources and profile scope

Architecture fields are taken from the linked developer configurations or model paper. Parameter counts are rounded planning values, and the bit-depth presets are approximate effective bits per weight; neither identifies the exact size of every quantized file. Profiles identify specific model generations, not a list of the newest or best models.

• Llama 3.1 8B
• Llama 3.1 70B
• Mistral 7B Instruct v0.3
• Qwen2.5 7B Instruct
• Qwen2.5 14B Instruct
• Qwen2.5 32B Instruct
• Gemma 2 9B — full-cache upper bound
• Phi-3 Mini 128K Instruct
• llama.cpp / ggml: quantization block layouts
• Ollama: context length and GPU offloading
• Ollama: cache quantization and concurrent requests

Reviewed September 13, 2026. This calculator does not run an LLM, measure tokens per second, recommend a model for quality, or simulate training, multi-GPU placement, vision inputs or multiple simultaneous sequences. Gemma 2 uses a full-cache upper bound because backends can handle its local-attention cache differently.

Working with image or video generation instead? Use the ComfyUI RAM/VRAM diagnostic guide.