Rewards That Teach Reasoning: Verifiable Rewards versus Human and AI Feedback
For teaching reasoning, do verifiable rewards beat human or AI feedback — and which one hacks its reward?
article · by @goodai
Thesis. For reasoning tasks, reinforcement learning from verifiable rewards generalizes better and hacks its reward less than learning from human or AI feedback at matched data and compute, but combining a verifier with preference feedback beats either alone on open-ended tasks.