GRPO

FlowBalance is a novel reinforcement learning method designed to stabilize the self-improvement process in large reasoning models. It effectively integrates sparse verifier feedback with dense, privileged-hindsight self-guidance to prevent model overconfidence and improve mathematical reasoning accuracy.

Researchers have developed SecOPD, a fine-tuning technique that uses token-level feedback to protect AI agents from prompt injection attacks. This method significantly outperforms previous state-of-the-art defenses by precisely identifying and penalizing malicious tokens during training.

The paper introduces TTPO, a method designed to optimize LLM reasoning during test-time training by utilizing asymmetric learning signals from pseudo-labels. It effectively mitigates the risks of noisy majority-vote labels by distilling agreeing rollouts and penalizing disagreeing ones.

Researchers have developed SecOPD, a fine-tuning technique that uses token-level feedback to protect AI agents from prompt injection attacks. This method significantly outperforms previous state-of-the-art defenses by precisely identifying and penalizing malicious tokens during training.

The paper introduces TTPO, a method designed to optimize LLM reasoning during test-time training by utilizing asymmetric learning signals from pseudo-labels. It effectively mitigates the risks of noisy majority-vote labels by distilling agreeing rollouts and penalizing disagreeing ones.

This paper presents Agentic ESOpt, a novel approach for fine-tuning LLM agents in long-horizon tasks using evolutionary strategies. The method proves more scalable and memory-efficient than RL-based alternatives like PPO and GRPO as task complexity increases.

Researchers have developed u-OPSD, a technique that allows large language models to perform self-distillation using only their own generated outputs. By leveraging internal consistency through majority voting, the model can correct its own errors without requiring external ground-truth data.

Researchers have developed ABSeeker, a search agent trained using a new fine-grained credit assignment method called ABC. This approach allows models to learn from specific useful steps within a trajectory rather than just the final outcome, enabling small models to rival much larger counterparts.

This paper presents the development of Ring-2.5-1T-Zero, a trillion-parameter model trained via zero-shot reinforcement learning to elicit emergent reasoning. The study validates that massive scaling enables models to spontaneously develop complex problem-solving strategies like self-verification without human-annotated data.

This study demonstrates that reinforcement learning post-training for LLMs is not a uniform process across all parameters but is concentrated in specific middle layers. By leveraging this discovery, researchers developed layer-aware training methods that outperform traditional full-parameter optimization.