VRAM-fit calculator for local models

What it computes. Three numbers and their sum. The weights: the parameter count multiplied by the bits per weight of the quantization you chose, divided by eight — or, when you know the file, the file's size as the download page states it. The KV cache: for every token in the context window, each layer keeps a key and a value vector per KV head, so the cache costs 2 × layers × KV heads × head dimension × bytes per element for each token, times the context length. The engine allowance: a figure you set for what the runtime itself needs — the driver's context, the compute buffers that scale with the context, the scratch space — because that part differs from engine to engine and this page cannot measure it. The verdict compares the sum with the card's memory and gives the margin either way; the last row solves the same equation backwards for the longest context that would still fit.

The method and its sources. The bits-per-weight figures are the ones llama.cpp's quantize tool publishes for its types, measured on a whole Llama 3.1 8B file — they already include the embedding and output tensors that most formats keep at higher precision, which is why Q4_K_M reads 4.89 bits rather than 4. A parameter count of 8.03 billion reproduces every one of those 24 published sizes to the hundredth of a GiB, and that reproduction is the known-answer table this page tests itself against. The quantised cache types are priced from GGML's block layouts: q8_0 stores 32 values in 34 bytes, q4_0 in 18. The architecture presets copy layers, KV heads and head dimension from each model's configuration file, named beside the preset with the date it was read; the parameter count you type comes from the model card, since it is not in that file.

A worked example. Llama 3.1 8B at Q4_K_M: 8.03 billion × 4.8944 bits ÷ 8 = 4,912,754,000 bytes, 4.58 GiB. Its cache at 8,192 tokens in f16: 2 × 32 × 8 × 128 × 2 = 131,072 bytes a token, exactly 1 GiB for the window. With a 1 GiB allowance the total is 6.58 GiB, so a 24 GiB card fits it with 17.42 GiB to spare — enough, by the last row, for a context of 150,934 tokens before the cache alone fills the card. A 70B at Q4_K_M, taking the published 43.1 GB file, is 40.14 GiB of weights; with 2.5 GiB of cache at 8,192 tokens and the same allowance it fits a 48 GiB card by 4.36 GiB and misses a 24 GiB card by 19.64.

The edge cases. Card memory is entered as the maker states it: a 24 GB card holds 24 GiB, so the field is in GiB and the hint says so. A file size from a download page may be in decimal GB or binary GiB, and the two differ by 7.4 percent — pick the unit the page used. Grouped-query attention is why the KV heads field exists: Llama 3.1 8B has 32 attention heads but only 8 KV heads, and the cache is sized by the 8. A model that shares no heads, like Phi-3.5 mini, pays the full price. A quantised cache (q8_0, q4_0) is counted at its storage cost only; whether your engine offers it, and what it costs in quality, is the engine's business. An empty allowance is zero, which is never true in practice — the field defaults to 1 GiB for that reason. A margin under 1 GiB is flagged tight.

What this tool cannot know. The engine's real overhead: CUDA or ROCm context, compute buffers, graph allocations and fragmentation vary by engine, version, batch size and flags, and the allowance is your estimate, not a measurement — llama.cpp, vLLM and the others each report their own figure at load time, and that figure is the one to trust. Whether the weights are split across cards, offloaded to system memory or memory-mapped is not modelled: the comparison is one card, all on the card. Vision encoders, LoRA adapters, speculative-decoding draft models and the memory a desktop already uses on the same card are not counted. The bits-per-weight table was measured on one architecture; a different vocabulary or layer shape shifts a file's real size by a few percent, so when the download page states the size, use the file mode. Serving more than one sequence at once multiplies the cache by the batch.

Method v1.0, 2026-11-13. Authorities: llama.cpp's quantize table for bits per weight (tools/quantize/README.md, read 2026-11-13), the GGML block layouts for quantised KV caches (ggml-common.h, read 2026-11-13), each architecture's config.json for layers, KV heads and head dimension (read 2026-11-13), and the KV-cache arithmetic of grouped-query attention. Sizes are shown to 0.01 GiB (1 GiB = 1,073,741,824 bytes), rounded half up for display only; the arithmetic underneath is exact to the byte.