GPU memory for BF16 and mixture-of-experts models

A mixture-of-experts (MoE) model must hold all of its parameters in memory, but each token only runs through some experts. Hugging Face's example is Mixtral 8x7B: it needs enough VRAM for a dense 47B-parameter model, while its inference compute is closer to a 12B model. Koskamo editorial arithmetic: BF16 stores each weight in 2 bytes, so weights alone take about 2 GB per billion parameters. A 35B-parameter MoE with 3B active needs roughly 70 GB for BF16 weights before KV cache, yet generates at the per-token cost of a much smaller model. Size GPU memory by total parameters and throughput by active parameters. Reviewed September 21, 2026.