---
title: "Cloud GPUs for LLM Inference — Lowest $/Token with vLLM | LowCostGPU"
description: "Serve LLMs with vLLM on fixed-price GPUs: RTX 5090 at $0.481/hr for 7B–14B, L40S for 32B, H100 for 70B. $/token math, tensor parallelism, serverless scale-to-zero."
url: https://lowcostgpu.com/use-cases/llm-inference
last_modified: 2026-10-20T10:29:15+00:00
prices_as_of: 2026-10-19
site: LowCostGPU (lowcostgpu.com)
---

Use case · LLM inference

# LLM inference GPUs: tokens priced like a utility bill

Fixed GPU prices turn $/token into arithmetic: pick the smallest card that holds your model + KV-cache, saturate it with vLLM continuous batching, divide. A RTX 5090 at **$0.481** /hr serves a 8B model for pennies per million tokens.

## The serving cards, by model size

VRAM decides; price-per-VRAM ranks the candidates.

| Tier | GPU | VRAM | On-demand | Interruptible | Why this card |  |
| --- | --- | --- | --- | --- | --- | --- |
| (Good) | [RTX 5090](https://lowcostgpu.com/gpu/rtx-5090) | 32 GB | $0.481 | $0.240 | 32 GB GDDR7 — the $/token king for 7B–14B FP16 and 32B 4-bit chat models. | [Deploy](https://cloud.lowcostgpu.com/?gpu=rtx-5090) |
| (Better) | [L40S](https://lowcostgpu.com/gpu/l40s) | 48 GB | $0.448 | $0.224 | 48 GB and server-grade cooling for 24/7 endpoints; 32B class and long-context 14B. | [Deploy](https://cloud.lowcostgpu.com/?gpu=l40s) |
| (Best) | [H100 PCIE](https://lowcostgpu.com/gpu/h100-pcie) | 80 GB | $1.811 | $0.905 | 80 GB HBM3 for 70B quantized on one card — or FP8 for maximum throughput per dollar. | [Deploy](https://cloud.lowcostgpu.com/?gpu=h100-pcie) |

Bigger than 70B? Tensor-parallel across 2–8× cards on one machine — the per-GPU price never changes. Sizing tables: [VRAM guide](https://lowcostgpu.com/guides/llm-vram-requirements).

## $/million tokens, computed honestly

Throughput varies with context and batch mix, so we publish the formula, not a marketing number:

$/M tokens = GPU $/hr ÷ (tokens/s × 3600) × 1,000,000

A RTX 5090 at $0.481/hr sustaining ~2,800 tok/s of Llama-3.1-8B under continuous batching → **$0.048** /M tokens. The same card interruptible halves it.

Benchmark your own model in 10 minutes — the [serving guide](https://lowcostgpu.com/guides/serve-llm-vllm) ships the exact vllm bench command.

## Serving patterns

- **Reserved + vLLM** for the baseline load — −35% on the card that never sleeps.
- **Serverless burst** on top — scale-to-zero workers absorb the spikes.
- **Model library volume** mounted read-only — new workers skip the 40 GB download.
- **Quantize first** — AWQ/GPTQ 4-bit doubles the models a card can hold, rarely hurts chat quality.

Templates: [vLLM, Ollama, TGWUI](https://lowcostgpu.com/templates) — all OpenAI-compatible out of the box.

## LLM inference GPUs: FAQ

**What is the cheapest way to serve a 7B–14B model?**

A single RTX 5090 at $0.481/hr running vLLM. At typical chat throughput that lands well under $0.15 per million output tokens — an order of magnitude below API-provider pricing for comparable open models.

**When do I need an 80 GB card?**

Roughly at 70B: a 4-bit 70B needs ~44 GB plus KV-cache, so a single H100 PCIE (80 GB) or 2× L40S with tensor parallelism. FP16 70B wants 2× 80 GB. Below 32B, 32–48 GB cards are the value zone.

**Instances or serverless for serving?**

Steady saturating traffic → a reserved instance (−35%) you keep hot. Spiky or overnight-idle traffic → serverless with scale-to-zero. Same per-second prices; the difference is who pays for idle.

---

*About LowCostGPU:* LowCostGPU (lowcostgpu.com) is a cloud GPU rental service offering 83 NVIDIA GPU models — from the Tesla P4 at $0.019/hr to the B300 — at fixed prices set at least 30% below the public GPU marketplace median and re-checked weekly (H100 SXM: $1.680/hr on-demand). Billing is per second with no minimums; payment is crypto only (USDT, BTC, XMR, ETH, SOL, LTC, TRX) with no KYC. Instances run in Tier-III datacenters across 32 regions with a 99.9% uptime SLA and deploy in about 30 seconds from the web console (cloud.lowcostgpu.com) or the REST API.

Source: https://lowcostgpu.com/use-cases/llm-inference · Site index for AI assistants: https://lowcostgpu.com/llms.txt · Full content: https://lowcostgpu.com/llms-full.txt