Paper 1
HuggingFace Papers
The research explores scaling 'zero RL' (reinforcement learning with verifiable rewards without human data) to a 1-trillion parameter model to observe emergent reasoning capabilities.
TL;DR
This paper presents the development of Ring-2.5-1T-Zero, a trillion-parameter model trained via zero-shot reinforcement learning to elicit emergent reasoning. The study validates that massive scaling enables models to spontaneously develop complex problem-solving strategies like self-verification without human-annotated data.
The research addresses the challenges of scaling Reinforcement Learning with Verifiable Rewards (RLVR)—specifically 'zero RL'—to the trillion-parameter scale. While previous studies focused on smaller models, this work investigates how massive parameters impact training dynamics and emergent reasoning behaviors. The authors identify that naive scaling often leads to issues such as poor readability, token redundancy, and inefficient reasoning depth. To mitigate these, they developed a stable training pipeline utilizing algorithmic optimizations like clipped importance sampling and training-inference ratio correction, alongside system-level improvements like mixed-precision control and context parallelism.
The findings provide a strong validation of the 'bitter lesson' in AI: that scale and computation eventually supersede human-engineered heuristics. The researchers observed three primary phenomena: first, that 1T models reach much higher performance ceilings than 104B models; second, that training moves from a discovery phase to a sharpening phase; and third, that the model autonomously develops advanced cognitive traits including anthropomorphism, structured formatting, self-verification, parallel reasoning, and 'context anxiety.' These behaviors emerged as optimal solutions to the RL objective without any supervised CoT data. Finally, the authors introduce a new evaluation framework focusing on the comprehensibility, reproducibility, and efficiency of reasoning traces, proving that Ring-2.5-1T-Zero produces highly structured and concise mathematical reasoning.
Ring-2.5-1T-Zero
Reinforcement Learning with Verifiable Rewards (RLVR)
Chain-of-Thought (CoT)
GRPO
Clipped Importance Sampling
Mixed-precision control
Context parallelism
Read paper →
The paper introduces SpectraReward and Self-SpectraReward, new training-free reward functions that utilize pretrained Multimodal Large Language Models (MLLMs) to improve text-to-image generation via reinforcement learning.
TL;DR
The researchers present a novel approach to reinforcement learning for image generation by repurposing pretrained MLLMs as zero-shot reward models. By measuring how well a prompt can be recovered from a generated image, the method avoids the need for costly human preference labeling.
The paper addresses the challenges of designing efficient and reliable reward models for text-to-image (T2I) reinforcement learning. Current methods often rely on expensive human preference annotations or complex question-decomposition pipelines that are difficult to scale. To solve this, the authors propose SpectraReward, a training-free reward function that leverages the pretrained image-text alignment of existing Multimodal Large Language Models (MLLMs). Instead of asking an MLLM to judge an image qualitatively, SpectraReward calculates the average image-conditioned prompt log-likelihood, essentially measuring how much information from the original prompt is preserved in the generated image. For unified multimodal models (UMMs), the authors introduce Self-SpectraReward, a closed-loop framework where the model's own understanding branch serves as the reward source for its generation branch. This approach ensures structural and distributional alignment between the policy and the reward model. Extensive experiments involving various MLLM backbones ranging from 4B to 235B parameters show that SpectraReward consistently improves generation performance across multiple benchmarks like GenEval and TIIF-Bench. Notably, the study reveals that scaling up the reward model does not always yield monotonic improvements, and the Self-SpectraReward approach can match or even outperform much larger external models by leveraging intrinsic alignment.
SpectraReward
Self-SpectraReward
MLLMs
SD3.5-M
BAGEL
AlphaGRPO
Diffusion Models
Unified Multimodal Models (UMMs)
Read paper →
The introduction of PalmClaw, an open-source framework designed to run LLM agents natively on mobile devices using tool-based interactions rather than GUI automation.
TL;DR
PalmClaw is a new open-source framework that moves LLM agent orchestration from servers directly onto mobile hardware. By replacing fragile GUI-based automation with structured device tools, it significantly improves task success rates and execution speed.
The research addresses the limitations of current mobile AI agents, which primarily rely on interpreting Graphical User Interfaces (GUI) through actions like tapping, swiping, and typing. This method is often inefficient due to long action sequences, sensitivity to UI changes, and a lack of direct access to device-level capabilities. To solve this, the authors propose PalmClaw, a native on-device agent framework. Unlike traditional architectures that require an external desktop or cloud host to manage the agent loop, PalmClaw hosts all essential components—including memory, skills, tools, and session management—directly on the smartphone. A core innovation of PalmClaw is its use of 'device tools,' which expose mobile resources like sensors, cameras, and local files through explicit arguments and structured results. This approach establishes clear execution boundaries and reduces the complexity of task execution. Empirical evaluations show that PalmClaw outperforms existing baselines with an 11.5% relative increase in task success and a massive 94.9% reduction in completion time. Furthermore, the framework simplifies deployment by reducing the setup burden typically associated with external agent orchestration.
PalmClaw
Large Language Models (LLM)
AutoGen
AgentScope
OpenClaw
Codex CLI
Claude Code
Gemini CLI
Read paper →
Paper 4
HuggingFace Papers
The introduction of Seed, a self-evolving on-policy distillation framework designed to improve reinforcement learning for LLM-based agents by converting trajectories into dense, actionable skills.
TL;DR
The Seed framework addresses the limitations of sparse rewards in agentic reinforcement learning by extracting reusable skills from completed interaction trajectories. It utilizes on-policy distillation to provide dense, token-level supervision that evolves alongside the policy's capabilities.
Large language models are increasingly deployed as agents capable of long-horizon tasks involving tool use and multi-turn interactions. However, standard outcome-based reinforcement learning (RL) suffers from a supervision gap because rewards are typically sparse and only provided at the end of an episode, offering no guidance on intermediate decisions. The 'Seed' (SElf-Evolving On-Policy Distillation) framework is proposed to bridge this gap by converting completed on-policy trajectories into training-time hindsight skills. The process involves a two-step approach: first, the policy is fine-tuned to analyze finished trajectories and generate natural-language descriptions of reusable workflows or failure-avoidance rules; second, these skills are used to re-score sampled actions, creating a dense, token-level distillation signal. Crucially, this mechanism is self-evolving, meaning that as the policy improves and encounters new environments, its ability to extract and utilize hindsight skills evolves in tandem. Experimental results across text and vision-based agentic tasks indicate that Seed significantly enhances both performance and sample efficiency while maintaining robust generalization to unseen scenarios.
Large Language Models (LLMs)
Reinforcement Learning (RL)
On-policy Distillation (OPD)
Agentic Reinforcement Learning
Seed Framework
Read paper →
Paper 5
HuggingFace Papers
The introduction of OvisOCR2, an end-to-end 0.8B parameter document parsing model that achieves state-of-the-art performance in converting document images to Markdown.
TL;DR
OvisOCR2 is a compact 0.8B parameter model designed for end-to-end document parsing into structured Markdown format. It outperforms traditional pipeline-based methods on major benchmarks like OmniDocBench and PureDocBench.
The technical report introduces OvisOCR2, a novel 0.8B parameter model designed for efficient, end-to-end document parsing. Unlike traditional pipeline methods that rely on separate models for layout analysis and content recognition—which often suffer from error accumulation and complex deployment—OvisOCR2 processes a document page image in a single pass to generate a unified Markdown representation. This representation includes text, formulas, tables, and visual regions in the correct reading order. The development of OvisOCR2 relied on a robust data engine that utilizes both cleaned real-world document annotations and synthetic pages generated from HTML templates to ensure high-quality training targets. The training methodology is multi-staged, encompassing supervised fine-tuning (SFT), reinforcement learning (RL) conducted on a larger 4B parameter branch with a multi-component reward design, followed by on-policy distillation back into the 0.8B model and final model fusion. Empirical results demonstrate that OvisOCR2 sets new state-of-the-art benchmarks, achieving an overall score of 96.58 on OmniDocBench v1.6 and a top Avg3 score of 75.06 on PureDocBench. Furthermore, evaluations on a custom in-house benchmark containing over 1,000 challenging pages, including handwriting and complex tables, confirm the model's superior generalization and robustness compared to existing methods.
OvisOCR2
Qwen3.5-0.8B
Markdown
OmniDocBench v1.6
PureDocBench
HTML
Read paper →