Not all AI is created equal. Some feel instant. Some feel like watching paint dry. Here’s what’s actually going on under the hood.
⏱️ TTFT — Time To First Token
How fast does the first word appear?
You: "Explain quantum computing"
Bad TTFT: [........] ← 3 seconds of silence 😬
Good TTFT: [█ ] ← first word appears fast 😊
This is what makes a model feel responsive. Even if the full answer takes time, a fast TTFT makes it feel alive.
🚀 TPS — Tokens Per Second
How fast does the rest stream out?
Slow TPS: "The ... cat ... sat ..." 🐢
Fast TPS: "The cat sat on the mat!" 🚀
TTFT gets you started. TPS gets you finished.
📊 Latency
The full picture — start to finish. TTFT + generation time combined.
Latency = TTFT + (tokens generated / TPS)
For chat apps, TTFT matters more. For batch jobs, total latency matters more.
🧠 Memory Bandwidth — The Surprising One
Most people assume faster GPUs = faster AI.
Most people think: GPU speed = AI speed 🤔
Reality: Memory speed = AI speed 😲
LLMs are memory-bound, not compute-bound. The GPU is often just sitting there waiting for data to arrive — like a Formula 1 car stuck in traffic. 🏎️🚦
This is why memory bandwidth is one of the most watched specs when evaluating AI hardware.
Want to geek out on real numbers across live models? Artificial Analysis tracks benchmarks across all major models and providers.