Paper 1
AI Models Can Provably Hide Arbitrary Capabilities
Researchers have demonstrated a novel method for cryptographically embedding hidden neural networks within larger model weights that remain undetectable and unelicitable without a secret trigger.
TL;DR
This paper presents a proof-of-concept for backdoor attacks that can provably hide arbitrary computational capabilities within AI models using cryptographic digital lockers. The researchers show that these hidden circuits are resistant to common auditing and elicitation techniques, posing a significant security risk in supply chain's integrity.
The research addresses a critical gap in AI safety evaluations: the assumption that dangerous model capabilities can be easily surfaced through testing. The authors propose a worst-case security threat model where an adversary—such as a malicious insider or a compromised software supply chain—embeds encrypted neural networks within the weights of a host model. These 'digital lockers' are designed to remain dormant and computationally infeasible to decrypt without a specific secret trigger, even when auditors have full white-box access to the model's parameters.
Unlike previous work that focused on hiding simple strings or fixed behaviors, this method allows for the embedding of complex, executable neural networks. The researchers implemented their construction using PyTorch, utilizing SwiGLU MLPs and RMSNorm architectures. To demonstrate the practical implications of such a threat, they used a chemical reaction prediction task as a proxy for CBRN-related risks, showing that the hidden capability could potentially assist in synthesizing dangerous substances if activated.
Empirical evaluations were conducted against several common elicitation and defense strategies, including supervised fine-tuning (SFT), weight noising, gradient-based trigger searches, and latent adversarial training. The results indicated that these methods failed to uncover or activate the hidden capabilities, suggesting that current auditing frameworks may be insufficient against sophisticated, cryptographically-backed backdoors. While the authors acknowledge that the current implementation has significant overhead and is not yet a 'turnkey' attack, the work serves as a vital warning for the development of more robust security defenses in the AI lifecycle.
Paper 2
Continually Post-training Small Models via Synthetic Self-distillation
The paper introduces SynthSelf, a method for the continual post-training of small language models using synthetic self-distillation via in-distribution aligner rewrites.
TL;DR
This research presents SynthSelf, a novel framework for improving small language models through synthetic self-distillation. By selecting an in-distribution aligner to rewrite training data, the method achieves high-quality supervision without drifting from the base model's distribution.
The research addresses the challenge of post-training small language models by introducing SynthSelf, a technique designed for continual learning through synthetic self-distillation. The core problem identified is the distributional gap that occurs when training small models on high-quality data from much larger, different-family models. To solve this, SynthSelf implements two primary mechanisms: off-policy distillation via rejection sampling and a strategic selection process for an 'aligner' model. The authors propose that an ideal aligner should be competent enough to provide high-quality supervision but similar enough in distribution to the base model to minimize divergence. The methodology involves using a rewrite prompt to transform existing gold demonstrations into the aligner's native phrasing, followed by a verification step where only rewrites that preserve the original correct answer are kept for training. Experimental results show that selecting aligners from the same model family, such as Qwen2.5-7B or 14B, significantly reduces the Kullback–Leibler (KL) divergence compared to using stronger, cross-family models. This approach ensures that the student model benefits from enhanced reasoning capabilities while remaining stable within its original distribution, effectively providing a low-cost path for scaling model performance.
SynthSelf
Qwen2.5
SFT (Supervised Fine-Tuning)
KL Divergence
Rejection Sampling
Read paper →
Paper 3
Which Tokens to Merge? Diffusion Dynamics for Efficient Image Generation
The introduction of DaTMe, a training-free framework for accelerating diffusion models by using diffusion dynamics to intelligently select tokens for merging.
TL;DR
The paper proposes DaTMe, a new method for accelerating diffusion model inference through intelligent token merging. Unlike previous methods that focus only on text-aligned regions, DaTMe preserves important structural and temporal details by analyzing the underlying denoising dynamics.
Diffusion models have become the standard for high-quality image and video synthesis, but their iterative denoising process is computationally expensive. Token merging has emerged as a training-free strategy to reduce this cost by grouping similar tokens; however, current methods often suffer from visual degradation because they focus primarily on text-aligned foreground regions. This 'prompt-centric bias' leads to the loss of important background details and structural boundaries that are not explicitly mentioned in the user prompt.
To solve this, the authors introduce Dynamics-aware Token Merging (DaTMe). The core innovation lies in shifting the definition of token importance from text relevance to diffusion dynamics. DaTMe utilizes two specific signals derived from the Tweedie denoised estimate: a temporal signal that identifies tokens undergoing significant changes between denoising steps, and a structural signal that captures high-frequency content like edges and textures. By identifying tokens that are either still 'unresolved' or structurally significant, DaTMe can selectively merge only those tokens that are temporally stable and spatially homogeneous. This approach ensures that critical features—even in non-prompted background areas—are preserved as anchors during the merging process. Testing on state-of-the-art models like PixArt-$\alpha$ and FLUX shows that DaTMe achieves significant acceleration while maintaining superior visual fidelity and structural integrity compared to existing CFG-based methods.
Diffusion Models
DaTMe (Dynamics-aware Token Merging)
PixArt-$\alpha$
FLUX
Classifier-Free Guidance (CFG)
Diffusion Transformers
Tweedie denoised estimate
Read paper →
Paper 4
CoDA: Co-Evolution of Data Engines and Action Policies
The introduction of CoDA, a framework for the co-evolution of simulation data engines and robot action policies through shared memory and automated feedback.
TL;DR
The CoDA framework enables a closed-loop system where robot simulation environments and action policies co-evolve based on shared learning progress. By using failure diagnoses to automate both scene generation and dense reward shaping, the method achieves substantial gains in out-of-distribution task success.
The research addresses a critical bottleneck in robotic manipulation: the inefficiency of building simulated environments that are not only physically valid but also pedagogically useful for training. While generative simulation can automate environment construction, existing methods often lack a feedback loop to determine which specific environmental variations will most effectively drive policy improvement. To solve this, the authors propose CoDA (Co-evolution of Data Engines and Action Policies), a framework where a simulation data engine and an action policy co-evolve via a shared heterogeneous memory. This memory stores validated diagnoses from a vision-language critic, which are then compiled into two types of programs: generation programs to recreate specific environmental difficulties and training programs to help the policy overcome them. Additionally, the paper introduces Text2potential, a mechanism that transforms failure diagnoses into staged potentials for action-chunk value learning, providing dense supervision without compromising the optimal task policy. Experimental results demonstrate significant performance leaps; in manipulation task families, CoDA improved out-of-distribution success from 5.3% to 27.6%. On the LIBERO-Pro benchmark, the framework increased success rates from 25% to 84% under position perturbations and from 1% to 76% under instruction perturbations. Furthermore, the study shows that the accumulated experience in the shared memory can be transferred across different policy backbones, significantly boosting the performance of new models without requiring additional environment generation.
CoDA
Text2potential
LIBERO-Pro
RoboGen
RoboTwin 2.0
SimFoundry
OpenVLA-OFT
Vision-Language Critic
Read paper →