llm-inference

Together AI and Fireworks AI compared as managed inference providers for open-source LLMs: pricing, speed, models, and features.

Groq's LPU runs open LLMs at 500-1000+ tokens per second, roughly 10x faster than typical GPU inference. Here's how and why it matters.

vLLM uses PagedAttention to serve LLMs 4x faster than baseline. Here's how it works and how it compares to TGI, TensorRT-LLM, and Ollama.