LLM VRAM Calculator
· By Weerasak · Methodology
See how model weights, context length and cache precision change estimated GPU memory. Start with a profile, then adjust the allowance to match your runtime. Scope: one text sequence on one GPU, with all model weights on that GPU.
Estimated within capacity
8.07 GiB estimated
67.24% of 12.00 GiB selected capacity · 3.93 GiB estimated margin
- Model weights (approximate)
- 4.52 GiB
- KV cache
- 1.00 GiB
- Extra margin on weights + cache
- 0.55 GiB
- Runtime + desktop allowance
- 2.00 GiB
Planning estimate, not a load test or speed prediction. Verify the intended context in your runtime before buying.
Try a worked example
Each button replaces all inputs with the example settings, including the 12 GiB GPU capacity and the default allowances.
| Scenario | Weights | KV cache | Total estimate |
|---|---|---|---|
| 8B · 8K · FP16 cache | 4.52 GiB | 1.00 GiB | 8.07 GiB |
| 8B · 32K · FP16 cache | 4.52 GiB | 4.00 GiB | 11.37 GiB |
| 8B · 32K · Q8 cache | 4.52 GiB | 2.13 GiB | 9.31 GiB |
| 14B · 8K · FP16 cache | 8.30 GiB | 1.50 GiB | 12.78 GiB |
How to read this estimate
Weights and KV cache are calculated separately. We add the allowance and margin you selected instead of pretending that every operating system or backend consumes a fixed amount. Near-capacity results need particular care: the uncertainty in runtime allocations is larger than a few hundred megabytes.
A result above capacity means the selected full-GPU plan exceeds the estimate, not that the model can never run. Partial CPU/RAM offload is outside this calculation. A result within capacity does not certify loading speed, output quality or stability.
Use the worked arithmetic guide to see the formulas. For image or video graphs, read ComfyUI RAM versus VRAM; this text-LLM formula is not a ComfyUI memory model.
Verify the estimate in Ollama
Load the exact model you intend to use and check Ollama’s current context settings. After a request, run ollama ps. Compare the active context with the calculator input and inspect PROCESSOR for GPU/CPU placement. The displayed model size is not a substitute for a peak-memory measurement.
Repeat with your intended prompt and output length. If a longer context causes trouble, compare a smaller allocated context while keeping the model and other settings fixed. Cache quantization changes are backend-dependent; check Ollama’s requirements for quantized KV cache before choosing Q8 or Q4 here. The calculator only compares the storage assumptions; it does not enable those settings in your runtime.
Why does Q8 cache use slightly more than half of FP16?
The Q8_0 storage model includes a scale value per block: 34 bytes for 32 entries, compared with 64 bytes for those entries in FP16. Q4_0 uses 18 bytes for 32 entries. Padding and backend buffers can add more. These layouts come from ggml’s block definitions.
Does a smaller weight file guarantee better performance?
No. Quantization changes storage and can affect output quality and compute behavior. Two files with the same quantization label can use different tensor mixtures. Entering your actual file size is a better weight-size starting point than a rounded parameter count, but the complete runtime still needs its own measurement.
Sources and profile scope
Architecture fields are taken from the linked developer configurations or model paper. Parameter counts are rounded planning values, and the bit-depth presets are approximate effective bits per weight; neither identifies the exact size of every quantized file. Profiles identify specific model generations, not a list of the newest or best models.
- Llama 3.1 8B
- Llama 3.1 70B
- Mistral 7B Instruct v0.3
- Qwen2.5 7B Instruct
- Qwen2.5 14B Instruct
- Qwen2.5 32B Instruct
- Gemma 2 9B — full-cache upper bound
- Phi-3 Mini 128K Instruct
- llama.cpp / ggml: quantization block layouts
- Ollama: context length and GPU offloading
- Ollama: cache quantization and concurrent requests
Reviewed September 13, 2026. This calculator does not run an LLM, measure tokens per second, recommend a model for quality, or simulate training, multi-GPU placement, vision inputs or multiple simultaneous sequences. Gemma 2 uses a full-cache upper bound because backends can handle its local-attention cache differently.