#LLM serving
4 posts on LLM serving.
FinOps-Estimator: sizing GPUs for a voice AI pipeline
An open-source tool that turns concurrency and model choices for STT → LLM → TTS into VRAM, RAM, CPU, TFLOPS and cloud or on-prem cost.
Fitting an LLM in your own server room: the memory math behind on-prem AI
Bits per weight, quantization error, outliers, KV cache and memory bandwidth: how to decide which model actually runs on the GPUs you own.
TurboQuant, reviewed: when a theory paper walks straight into production
A close read of TurboQuant (ICLR 2026): random rotations, Lloyd–Max codebooks and a 1-bit QJL residual, and why it went from proof to vLLM and a 17k-star vector index within weeks.
Your MAU is not your load: capacity math for voice AI at scale
How to turn "a million users" into a latency budget, a concurrency number, KV-cache memory and a GPU count — with the formulas, not vibes.