±20%. A first order, not a floor plan.
Inference, fine-tuning or pre-training, on NVIDIA, AMD or Intel: from a DGX Spark on a desk to a GB300 NVL72, the calculator gives you the GPU count, memory, parallelism, throughput and kilowatts.
Pick a mode, a model, a target. The engine does the rest: memory, parallelism, racks, watts, and what runs out first.
Serve a model to N users at a target speed.
Workload
KV cache precision
Precision
Verdict:8× H100 SXM
Inference · Llama 3.3 70B · FP8
What you asked for
You needNVIDIA
8GPUs
H100 SXM
Cluster shape
1node
1 rack · TP4 · PP1 · DP1
Limited by: decode bandwidthestimated
GPU count set by memory
Memory constraint: 8 GPUs
Throughput constraint: 8 GPUs
Total GPU memory
161 GB
Power per rack
10.7 kW
Total throughput
6,000 tok/s
Time to first token
0.17 s
Memory
derivedParallelism
derivedFabric & storage
derivedRacks & power
derivedDerived: computed from a published vendor constant. Estimated: rests on an efficiency assumption. Measured: taken from a published benchmark.
First-order estimate, ±20%. Assumptions: MBU 50% · PUE 1.45. Real sizing goes through a benchmark on your workload.
Computed on adaptizy.com/en/gpu-sizer
Six calculations, no magic. The same ones we put on the whiteboard before ordering anything.
Every parameter takes a fixed number of bytes depending on precision: 2 in BF16, 1 in FP8, 0.5 in FP4. A 70-billion-parameter model therefore weighs 140 GB in BF16 and 70 GB in FP8. For MoE models, all experts stay resident in memory, even if only a few of them work on any given token.
weights_GB = params_B x bytes/param(precision) bytes/param: bf16 = 2, fp8 = 1, fp4 = 0.5 example: 70B at fp8 = 70 GB
To avoid recomputing the whole conversation for every new token, the GPU keeps the keys and values of every token it has seen. That costs a few hundred kilobytes per token per user. At 200 users with 2,000 tokens of context, this cache often outweighs the model itself. It is what sizes inference, not the weights.
kv/token = 2 x layers x kv_heads x head_dim x kv_bytes kv_total = kv/token x users x live_context live_context = input_tokens + output_tokens / 2
The amount of compute is almost a physical constant: about 6 operations per parameter per token (4 for LoRA). Then, no GPU ever hits its spec sheet: in practice you get 28 to 47% of peak, which is the MFU. Divide one by the other and you get the time. Careful: NVIDIA and AMD both publish figures with 2:4 sparsity; we use dense TFLOPS everywhere, which is half.
FLOPs = 6 x active_params x tokens (LoRA: 4 x) time = FLOPs / (gpu_count x dense_TFLOPS x MFU) typical MFU: 0.28 to 0.47 by generation, interconnect and software stack
When the model does not fit on one GPU, you cut it up. Tensor parallel (TP) splits every layer across neighboring GPUs: very chatty, so it stays inside the scale-up domain, whether that is NVLink, Infinity Fabric or integrated Ethernet. Pipeline parallel (PP) stacks slices of layers. Data parallel (DP) duplicates the whole thing to absorb more traffic or more data. The engine picks the smallest TP that makes the model fit, then fills in with PP and DP.
TP = min {1, 2, 4, 8, ...} s.t. weights + headroom <= TP x 0.9 x gpu_mem
DP = gpu_count / (TP x PP)An 8-accelerator node pulls 10.2 to 15.5 kW, whether it is an NVIDIA HGX chassis, an AMD OAM baseboard or a Gaudi 3 box. An NVL72 rack pulls 120 to 137 kW, liquid cooling mandatory. Then you add the building's cooling through the PUE: 1.45 on air, 1.15 on liquid. A desk machine, meanwhile, pulls 240 W from a wall socket and has no PUE at all. On many projects the real constraint is not the number of cards, it is how many kilowatts your room can deliver.
it_load_kW = nodes x node_kW x 1.05 (fabric) facility_kW = it_load x PUE (air 1.45, liquid 1.15) desk machine: PUE = 1, no rack
Every result block carries its level of proof. Derived: the number comes from a constant the vendor publishes, memory, TDP, bandwidth, run through a closed-form formula. Estimated: the number also rests on an efficiency assumption, usable memory bandwidth in inference or MFU in training, and that is where most of the ±20% lives. Measured: the number comes from a published benchmark, and there is exactly one of those in the whole engine.
derived = vendor constant + closed-form formula estimated = derived x efficiency assumption (MBU 0.50, MFU 0.28 to 0.47) measured = published benchmark (DGX Spark: 43.6 tok/s predicted vs 43.7 measured)
Every AI cluster is built from the same bricks, whatever logo is on the card. Here they are, with their real numbers.
The standard AI datacenter brick, at all three vendors.
Serving models up to ~400B (FP8), fine-tuning up to a few nodes.
72 GPUs pretending to be one.
Pre-training, giant MoE models (671B and up), very high-traffic serving. Nothing equivalent ships in volume at AMD or Intel: their platforms stop at 8 accelerators per baseboard.
yes, one rack really pulls 120 kW
Serious hardware, air-cooled, no plumbing.
Models up to ~24B, internal RAG, rooms with no liquid cooling.
Sizing is the easy part. Running it is our job.
Private AI deployment on infrastructure you control: hardware, network, models, monitoring. You keep the data, we carry the pager.
±20% on a good day. The engine computes first-order from peak specs and field-observed efficiencies (MFU of 28 to 47%, memory bandwidth at ~50%). Precise enough to build a budget and compare options, not enough to sign a purchase order. Real sizing goes through a benchmark on your actual workload.
Because a price shown here would be confidently wrong. GPU list prices move every quarter, the street price depends on volume and on your relationship with the vendor, and cloud rates vary by an order of magnitude between providers and commitment levels. What does transfer from one quote to the next is the physical sizing: the GPU count, the terabytes of memory, the kilowatts. Those numbers we stand behind; the rest your vendor will price better than we can, and for the right week.
Yes, but not by snapping your fingers. HGX B200/B300 servers ship in a few weeks through the OEMs. NVL72 racks go to hyperscalers first: expect several months and a real volume commitment. H100s and H200s are available everywhere, including second-hand, and remain excellent workhorses for inference.
They are in the engine, with their real constants: MI300X, MI325X, MI350X, MI355X and Gaudi 3. What a multi-vendor sizing gives you is the physics: bytes per parameter, KV cache, memory bandwidth, kilowatts, the same arithmetic whatever logo is on the card. What does not transfer is the software stack: ROCm handles the widespread families (Llama, Qwen, Mistral, DeepSeek) well and upstream in vLLM and SGLang, the Gaudi stack goes through a vLLM plugin that trails by a few weeks, and an architecture published last month lands on NVIDIA first. On most projects that lag is what decides, not the card count. The calculator tells you how much memory and how many kilowatts; it will not tell you whether your model boots on the first try.
No. The sizing runs entirely in JavaScript in your tab: your inputs never leave it, no email asked. Close the tab, nothing remains. The site counts page views, without cookies, and that counter never sees your numbers.
Yes, that is the job. Audit, hardware selection, installation, model deployment and monitoring, on your infrastructure or your private cloud. The booking button sends your sizing along with the call, so the conversation starts from the exact numbers.
30 minutes with an engineer, not a sales rep. Bring your sizing, leave with a plan.
we bring the screwdriver