Interactive calculator predicting LLM decode speed and memory fit from the tiered decode law, with charts placing your machine among the validated measurements and showing the speed-memory trade-off.

Will it run — and how fast?

Pick a model, a bit-width, and a machine. This calculator applies the tiered decode law — fitted and validated on measurements from 7B to 753B, including two pre-registered hardware tests — to predict decode speed, memory fit, and quality cost before you download a single weight.

tok/s = η(tier) × bandwidth ÷ active-bytes-per-token

Configuration

Model

Total params (B)

Active params (B)

Expert/MLP bit-width

Machine

VRAM (GB)

VRAM BW (GB/s)

RAM (GB)

RAM BW (GB/s)

Disk read (GB/s)

Context depth (tokens) — 0 = short-context law; adds KV reads + KV memory (Law 4 v2)

Target speed (tok/s) — drawn on the trade-off chart

Prediction

—tok/s (decode)

model size

—

active bytes/token

—

est. quality cost

—

All placements evaluated

The ladder — smartest model per speed band on THIS machine (bar length = model size in params; your target highlighted)

Your machine on the validated law

Speed ⇄ memory trade-off (bit-width sweep, capacity cliffs marked)

Quality cost = perplexity gap-ratio vs fp16 with the depth-aware recipe (probe the fragile band, protect it). η bands are measured: VRAM/GPU per class · RAM ≈0.62 dense / ≈0.38 MoE · disk ≈0.95. Validation: 16B MoE 15.6 tok/s, 30B hybrid predicted 19 → measured 19.30 ± 0.88, 110B-from-SATA 0.19 (pre-registered 0.2–0.3), RAM overclock ×1.52 (pre-registered ×1.41+), colibri's published 744B tiers inside the same bands. Hybrid figures assume partitioned residency (--no-mmap + -ot exps=CPU). Estimates ±25% off-VRAM; all-in-VRAM is one-sided — measured ≥0.90× the prediction every time, typically 1.1–1.8× faster. v1.1 context term: each generated token re-reads the whole KV cache from the tier KV lives on (η_kv≈0.7, single-point calibration: measured 20.02→16.12 tok/s at depth 16384; term prompted by u/RogerAI--fyi). MLA models cache ~10× less; SWA long-context slopes use global layers only [est].
Preset bandwidths are estimates — pip install quantprobe && quantprobe calibrate measures your machine (RAM stream, disk, GPU sustained clocks) and anchors predictions by default (pre-registered leave-one-out gate #64: median error 19% → 5.8% across 5 arms; --no-anchors disables).
By @federico_sciuca — laws, probes, recipes and every raw log in the repo.