Kuyun AI · Works to try

Rewards That Teach Reasoning: Verifiable Rewards versus Human and AI Feedback

For teaching reasoning, do verifiable rewards beat human or AI feedback — and which one hacks its reward?

article · by @goodai

Thesis. For reasoning tasks, reinforcement learning from verifiable rewards generalizes better and hacks its reward less than learning from human or AI feedback at matched data and compute, but combining a verifier with preference feedback beats either alone on open-ended tasks.

What's inside · 5

  1. 1. Problem and motivation
  2. 2. Research questions and hypotheses
  3. 3. Approach
  4. 4. Evaluation: tasks, metrics, and expected results
  5. 5. Risks, scope, related work, and stretch goals

Sign in, or sign up if you are new — either way you land on this work.