Petaflare

GPU for LLM inference: memory, speed and cost per model size, paid in Bitcoin or USDT

Choosing a GPU for a language model, which you rent here with crypto only, comes down to two numbers: how much memory the model and its context need, and how fast the card can read that memory. Both can be worked out before you rent anything.

Crypto-only payment, no card, no KYCUpdated 19 November 2026Prices checked 19 November 2026

The GPU you need for an LLM is set by memory first: the weights take 2 bytes per parameter in BF16, 1 in FP8 and about half a byte in 4-bit, plus a KV cache that grows with the context. An 8B model in BF16 fits on a 24 GB RTX 4090 ($0.34 an hour), a 32B model in FP8 on a 48 GB L40S, a 70B model in FP8 on one H200 (141 GB, $3.29 an hour) or two H100s, and the largest open models, such as DeepSeek's 671B, need eight H200s or B200s. Speed then follows memory bandwidth: the faster a card reads memory, the more tokens per second it generates.

How much memory each model size needs

Memory for the weights alone. Leave room on top for the KV cache, the runtime and activations, typically 10% to 30% more.

The A100 has no FP8 Tensor Cores, so run FP8 models on it only through weight-only kernels, or choose BF16 or 4-bit. Mixture-of-experts models such as gpt-oss and DeepSeek still need all their weights in memory, even though each token uses only part of them.
Model sizeBF16FP84-bitCards to rent
8B, such as Llama 3.1 8B16 GB8 GBAbout 5 GBRTX 4090 in BF16 with room for cache
14B28 GB14 GBAbout 8 GBRTX 4090 in FP8; the 32 GB RTX 5090 for longer contexts
32B64 GB32 GBAbout 18 to 20 GBA100 80GB in BF16; L40S in FP8; RTX 5090 in 4-bit
70B, such as Llama 3.3 70B140 GB70 GBAbout 35 to 40 GBOne H200 or two H100s in FP8; L40S in 4-bit
gpt-oss-120b——Released in 4-bit (MXFP4); fits one 80 GB GPU, according to OpenAIOne H100
405B, such as Llama 3.1 405B810 GB405 GBAbout 210 to 230 GB8 × H100 in FP8; 8 × H200 in BF16
671B, such as DeepSeek-V3 or R1—About 700 GB (released in FP8)—8 × H200 (1,128 GB) or 8 × B200 (1,440 GB)

The A100 has no FP8 Tensor Cores, so run FP8 models on it only through weight-only kernels, or choose BF16 or 4-bit. Mixture-of-experts models such as gpt-oss and DeepSeek still need all their weights in memory, even though each token uses only part of them.

The KV cache grows with context

While generating, the model keeps a key and a value vector for every past token in every layer. Its size per token is 2 × layers × KV heads × head size × bytes per value.

  • Llama 3.1 8B has 32 layers and 8 KV heads of size 128, so each token takes about 0.13 MB in BF16. An 8,192-token context needs about 1.1 GB; a full 128,000-token context about 17 GB, which together with the 16 GB of weights is more than an RTX 4090 holds.
  • Llama 3.1 70B has 80 layers and 8 KV heads of size 128: about 0.33 MB per token. A 32,768-token context takes about 11 GB, and 128,000 tokens about 42 GB.

Those figures are per conversation. Serving ten users with 8k contexts on the 70B model needs about 27 GB of cache on top of the weights. An FP8 KV cache halves these numbers, at a small cost in accuracy. This is why a 70B model in FP8 fits on an 80 GB H100 but leaves little room for users, while an H200 with 141 GB leaves about 70 GB for them, before the serving software takes its share. The H200 rental page works through that case.

How fast each card can generate

For a single request, each new token requires reading all the weights once, so memory bandwidth sets a ceiling: bandwidth ÷ size of the weights. These are upper limits; real servers reach a fraction of them.

Bandwidth from NVIDIA's specifications. Serving many users at once raises total throughput far above these single-request figures, because one read of the weights serves every request in the batch.
CardMemory bandwidth8B in BF16 (16 GB)70B in FP8 (70 GB)
L40S864 GB/s≤ 54 tokens/sDoes not fit
RTX 40901,008 GB/s≤ 63 tokens/sDoes not fit
RTX 50901,792 GB/s≤ 112 tokens/sDoes not fit
A100 80GB1,935 to 2,039 GB/s≤ 121 to 127 tokens/sNo FP8 support
H100 80GB2.0 TB/s (PCIe) to 3.35 TB/s (SXM)≤ 125 to 209 tokens/s≤ 29 to 48 tokens/s, little room for cache
H200 141GB4.8 TB/s≤ 300 tokens/s≤ 69 tokens/s
B200 180GB8 TB/s≤ 500 tokens/s≤ 114 tokens/s

Bandwidth from NVIDIA's specifications. Serving many users at once raises total throughput far above these single-request figures, because one read of the weights serves every request in the batch.

Cost per million tokens

What a million generated tokens cost at our list prices, depending on the total throughput your server reaches across all users. Measure your own with your model and batch size, then read across.

Cost = hourly price ÷ (tokens per second × 3,600) × 1,000,000. A single request is bounded by the ceilings in the table above; batching many requests can reach thousands of tokens per second in total. All prices on price per GPU-hour.
Card, per hour100 tokens/s500 tokens/s1,000 tokens/s2,500 tokens/s
RTX 4090, $0.34$0.94$0.19$0.09$0.04
RTX 5090, $0.49$1.36$0.27$0.14$0.05
L40S, $0.87$2.42$0.48$0.24$0.10
H100, $2.39$6.64$1.33$0.66$0.27
H200, $3.29$9.14$1.83$0.91$0.37

Cost = hourly price ÷ (tokens per second × 3,600) × 1,000,000. A single request is bounded by the ceilings in the table above; batching many requests can reach thousands of tokens per second in total. All prices on price per GPU-hour.

Serve a model on a rented GPU

vLLM serves most open models behind an OpenAI-compatible API with continuous batching. On our PyTorch image it installs with pip; you can also launch the machine straight from the vllm/vllm-openai Docker image. Binding it to 127.0.0.1 and using an SSH tunnel keeps your endpoint private.

# On the rented machine
pip install vllm
hf auth login        # gated models such as Llama need an accepted licence
vllm serve meta-llama/Llama-3.1-8B-Instruct --host 127.0.0.1 --port 8000 --max-model-len 16384

# On your own computer: tunnel, then call the API
ssh -N -L 8000:127.0.0.1:8000 root@<address>
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model": "meta-llama/Llama-3.1-8B-Instruct", "messages": [{"role": "user", "content": "Hello"}]}'
On multi-GPU machines, add --tensor-parallel-size 2, 4 or 8 to split a model across the cards. --max-model-len caps the context so the KV cache fits.

Which card to rent

  • Up to 14B, one user or a few: an RTX 4090 at $0.34, or the RTX 5090 at $0.49 for more context and almost twice the bandwidth.
  • 32B, or 70B in 4-bit: an L40S with 48 GB. If you want BF16 quality on a 32B model, rent an A100 80GB instead.
  • 70B in FP8 for real traffic: one H200 holds the weights with about 70 GB to spare for cache; two H100s split the model with tensor parallelism. See the H100 page for SXM and PCIe differences.
  • 405B and 671B models: eight H200s or eight B200s in one machine.

All of these are whole cards, billed by the minute while the machine runs, as part of our cloud GPU rental. If you also want to adapt a model to your data, memory needs for training are on GPU for LLM fine-tuning.

Paying for inference GPUs in crypto

A model served for weeks fits the large blocks: 1,200 RTX 4090 hours for $384, or 640 H200 hours for $1,894.40. Pay once in USDT, Bitcoin or another of our seven coins, with no card, and the hours never expire. Every block is on the price list.

LLM GPU questions

Do you need a GPU to run an LLM?

Small models run on a CPU, slowly. For anything above a few billion parameters, or for more than one user, a GPU's memory bandwidth is what makes generation fast enough to use.

What GPU does ChatGPT use?

OpenAI does not publish the details of its serving fleet. Services of that size run on large clusters of data-center accelerators, such as NVIDIA's H100, H200 and B200, the same cards rented here, one machine at a time.

Is a GPU required for inference?

Not strictly, but it is the practical choice. Each generated token reads the whole model from memory, and a GPU reads memory many times faster than a CPU.

How do I know a model will fit before renting?

Add the weights (parameters × bytes per parameter) to the KV cache for your context length and number of users, plus 10% to 30% for the runtime. If the total is close to the card's memory, choose the next size up.

Sources