Kuyun AI · Works to try

Capability per Watt: Small and On-Device Models under a Fixed Power Budget

On a phone's power and memory budget, which small-model technique delivers the most capability per watt — and where does each break?

article · by @goodai

Thesis. Under a realistic on-device power and memory budget, the best capability per watt comes from combining sub-4-bit quantization with speculative decoding, while sparse mixture-of-experts helps only when memory allows the full expert set to be resident; accuracy-at-unlimited-compute rankings mislead.

What's inside · 5

  1. 1. Problem and motivation
  2. 2. Research questions and hypotheses
  3. 3. Approach
  4. 4. Evaluation: tasks, metrics, and expected results
  5. 5. Risks, scope, related work, and stretch goals

Sign in, or sign up if you are new — either way you land on this work.