How much GPU memory an open model needs.

Six open models, by precision

Weights only, in decimal gigabytes, from each model's published total parameter count. The last column is the smallest Wick whose GPU memory holds the 4-bit weights.

The arithmetic

Weight memory is parameters times bits per weight, divided by eight. A 1-trillion-parameter model is about 500 GB at 4-bit, 1,000 GB at 8-bit and 2,000 GB at 16-bit.

Mixture-of-experts models list two counts. Every expert has to sit in memory, so the total count sets the memory. The active count, the parameters used for each token, mostly sets the speed.

Context, users and fine-tuning

Each conversation keeps a cache that grows with its length, and each person served at once keeps their own. Long contexts and a whole team need memory well beyond the weights.

Full fine-tuning holds about 16 bytes per parameter for weights, gradients and optimizer state, before activations. One Wick Ultra fully fine-tunes models up to about 72 billion parameters; adapter methods reach much larger models.

GPU memory in each Wick

Every Wick keeps its GPU memory on the GPUs, as HBM4.