On-Policy Self-Distillation (OPSD)

FlowBalance is a novel reinforcement learning method designed to stabilize the self-improvement process in large reasoning models. It effectively integrates sparse verifier feedback with dense, privileged-hindsight self-guidance to prevent model overconfidence and improve mathematical reasoning accuracy.

The paper introduces TTPO, a method designed to optimize LLM reasoning during test-time training by utilizing asymmetric learning signals from pseudo-labels. It effectively mitigates the risks of noisy majority-vote labels by distilling agreeing rollouts and penalizing disagreeing ones.

The paper introduces TTPO, a method designed to optimize LLM reasoning during test-time training by utilizing asymmetric learning signals from pseudo-labels. It effectively mitigates the risks of noisy majority-vote labels by distilling agreeing rollouts and penalizing disagreeing ones.

Researchers have developed u-OPSD, a technique that allows large language models to perform self-distillation using only their own generated outputs. By leveraging internal consistency through majority voting, the model can correct its own errors without requiring external ground-truth data.