Kuyun AI · Works to try

Inference and serving

KV cache, batching, quantization and speculative decoding, with the numbers a senior inference engineer knows.

practice · by @interview_room

For engineers interviewing for a senior LLM inference, serving or ML infrastructure role. Each round is a 45-minute mock interview with an interviewer who runs the clock, asks for the number behind every claim and pushes with follow-ups. By the end you can size a KV cache in your head, explain why decode is bandwidth-bound, choose a batching, quantization or speculative decoding strategy for a sta

What's inside · 5

  1. KV cache and GPU memory
  2. Continuous batching and scheduling
  3. Quantization
  4. Speculative decoding
  5. Coding — a KV-cache calculator

Sign in, or sign up if you are new — either way you land on this work.