Skip to content
AccountMenu

Hardware for a 70B or 400B model, by precision.

A 70-billion-parameter model needs about 140 GB of GPU memory for its weights at 16-bit and a 405-billion-parameter model about 810 GB, before context.

One Wick, with 288 GB on a single Rubin GPU, holds a 70B model at full 16-bit precision; Wick Ultra's 1,152 GB holds a 405B model at 16-bit and fully fine-tunes a 70B model.

See Wick Ultra

From $656,000Includes delivery, the outdoor unit and the first year of service.

Weights by precision

Weights only, in decimal gigabytes, and the smallest Wick whose GPU memory holds them. Parameter counts come from each model's card.

70B to 405B models against GPU memory
ModelParameters16-bitSmallest Wick8-bitSmallest Wick4-bit
Llama 3.3 70B70 billion140 GBWick70 GBWick35 GB
Qwen3.5 397B397 billion794 GBWick Ultra397 GBWick Pro199 GB
Llama 3.1 405B405 billion810 GBWick Ultra405 GBWick Pro203 GB

Fine-tuning is the larger number

Serving a model needs its weights and context. Full fine-tuning also holds gradients and optimizer state, about 16 bytes per parameter before activations: 1,120 GB for a 70B model.

That fits one Wick Ultra's 1,152 GB, which fully fine-tunes models up to about 72 billion parameters. Wick fully fine-tunes up to about 18 billion and Wick Pro up to about 36 billion; adapter methods reach much larger models on each.

Full fine-tuning at 16 bytes per parameter, before activations
SystemGPU memoryFull fine-tuning up toPrice
Wick288 GBAbout 18 billion parameters$199,000
Wick Pro576 GBAbout 36 billion parameters$339,000
Wick Ultra1,152 GBAbout 72 billion parameters$656,000
Wick Cluster node2,304 GBAbout 144 billion parameters$1,234,000 per node

One researcher or a team

A 70B model at 16-bit leaves 148 GB free on Wick for one researcher's context. A team serving the same model to many people at once, with long documents, needs the room Wick Pro or Wick Ultra leaves after the weights.

Wick sits beside a desk on a 240 V, 30 A circuit; Wick Ultra runs on a 240 V, 60 A circuit. Both send their heat outside through their own outdoor unit.

Questions

What hardware do I need to run a 70B model?

About 140 GB of GPU memory at 16-bit, 70 GB at 8-bit or 35 GB at 4-bit, plus room for context. One Rubin GPU's 288 GB holds it at 16-bit; Wick is that system, from $199,000.

Can I run Llama 3.1 405B locally?

Yes, with enough GPU memory: about 405 GB at 8-bit, within Wick Pro's 576 GB, or 810 GB at 16-bit, within Wick Ultra's 1,152 GB.

Is 4-bit good enough?

4-bit quarters the memory of 16-bit weights, at a cost in fidelity that varies by model and task. Compare your own prompts at each precision before you choose, and keep 16-bit where you fine-tune.

Sources

  1. Meta, Llama 3.3 70B Instruct model card on Hugging Face, observed October 27, 2026
  2. Qwen3.5 397B model card on Hugging Face, observed October 27, 2026
  3. Meta, Llama 3.1 405B model card on Hugging Face, observed October 27, 2026
  4. NVIDIA, Vera Rubin NVL72 specifications, per Rubin GPU, observed October 23, 2026

Wick figures are read from its published specifications and list prices. Checked October 27, 2026.