Transformer

This article explores a method for creating deterministic arithmetic calculators by compiling algorithmic logic directly into transformer weights. By using the Torchwright compiler, the author bypasses the need for training and achieves exact multiplication, addition, and subtraction results.

This paper presents a method for converting pretrained transformers into linear-time architectures by focusing on the efficiency of state update designs. By analyzing softmax attention through a first-order approximation, the authors prove that delta-style updates are superior for post hoc linearization.

This study demonstrates that reinforcement learning post-training for LLMs is not a uniform process across all parameters but is concentrated in specific middle layers. By leveraging this discovery, researchers developed layer-aware training methods that outperform traditional full-parameter optimization.

Wan-Streamer is a new foundation model that enables seamless, low-latency audio-visual interaction by processing interleaved text, audio, and video tokens within a single causal Transformer. By moving away from cascaded modular pipelines, it achieves sub-second end-to-end latency suitable for real-time digital humans and embodied assistants.