VRAM Calculator

Estimate VRAM usage and generate vLLM serve arguments for any HuggingFace model.

Select Model

KV Cache Quantization

KV Cache precision. Lower values reduce VRAM, especially for long sequences.

Advanced Options

Inference Quantization

Override the model's default weight precision. Pre-quantized models auto-detect from config.json.

Hardware Configuration

Device / GPU

Select your GPU or set custom VRAM

GPUs per Node

Devices for tensor parallel inference

Nodes

Number of nodes per replica (pipeline parallelism)

Replicas

Number of independent vLLM instances (increases total throughput)

Parallelism Strategy

How weights are sharded across GPUs. Tensor Parallel splits all weights evenly. Data + Expert Parallel replicates non-expert (dense) weights on every GPU and shards only MoE experts — common for MoE models but uses more per-GPU VRAM for weights.

Total: 1 GPU(s) ( GB VRAM) · 1 GPU(s)/replica

Input Parameters

Sequence Length: 1024

Max tokens per input; impacts KV cache (also affected by attention structure) & activations.

1K4K16K64K256K1M

Concurrent Users:

Number of users running inference simultaneously (affects memory usage and per-user performance)

116324864

GPU Memory Utilization: 90%

Fraction of each GPU's VRAM vLLM may use (--gpu-memory-utilization). The rest is reserved for the CUDA context and overhead.

50%99%

Max Batched Tokens: 8,192

Per-step prefill budget (--max-num-batched-tokens). Sizes the KV allowance for windowed layers, so lowering it frees KV cache; the cost is more prefill chunks and slower time-to-first-token.

1K2K4K8K16K32K64K

Results update automatically when a model is selected.

Select a model to see estimated VRAM, performance, and a vLLM serve command.