How much VRAM do you actually need?

Three things need to fit

The model weights are the learned numbers loaded from the model file. The KV cache holds attention state from the conversation. The runtime needs additional memory for intermediate work, buffers and other allocations. Your GPU also needs space for any display or other applications.

A download size can give a useful starting point for weights, but it is not the total memory needed to run that file.

• Weights grow with stored parameters and weight precision.
• KV cache grows with live tokens and concurrent conversations.
• Runtime overhead changes with the engine, batch size and implementation.

A worked example: Ministral 3 14B

Using 13.945 billion stored parameters and a planning assumption of 4.8 bits per weight, the weights occupy about 7.8 GiB. At 8,192 tokens and FP16 KV cache, its 40 layers, eight KV heads and 128-element head dimension need another 1.25 GiB.

Add the planner’s 1 GiB runtime allowance and you reach about 10.0 GiB. A 12 GiB card with 1 GiB reserved is close to its limit; a 24 GiB card has substantially more room. These are assumptions to test against your chosen file and runtime.

Buy for the workload you will use

Short single-user chat and a coding agent reading large files are different workloads. Set the context and number of concurrent requests before comparing hardware. If the result is tight, choose a smaller model or shorter context before treating a larger GPU as the only solution.

Memory fit does not measure answer quality or generation speed. Try a small model on a representative task, and measure the actual runtime before committing to a larger setup.