LOCAL AI / ESTIMATE
How much memory could it need?
Explore how model size, quantization and context change inference memory. All calculations happen in your browser.
Weights
3.73 GiB
KV cache
1.00 GiB
Allowance
0.75 GiB
Estimated total
5.47 GiB
The defaults describe a hypothetical configuration. Replace them with the model’s configuration. Total includes an adjustable allowance; it does not guarantee the model fits or predict tokens per second.
The calculation
Weights = total parameters × bits ÷ 8. KV cache = 2 × attention layers × KV heads × head dimension × context tokens × concurrent sequences × cache bytes. Convert bytes to GiB by dividing by 2³⁰. The allowance is a percentage of weight memory.
This estimates a conventional full attention KV cache. Sliding windows, hybrid/recurrent architectures, compressed caches, offloading and runtime implementations differ. Use total stored parameters for mixture-of-experts weights, not just active parameters. Quantization metadata, activations, operating-system memory and temporary buffers can add more. CPU RAM and GPU VRAM are not interchangeable, and unified-memory systems must share capacity with the OS.
Read Hugging Face’s cache guidance ↗