Mixture-of-Experts (MoE)

DeepSeek's new V4.1 Flash model introduces a massive 763B parameter architecture that optimizes for low latency and high throughput. By using N-gram parameters as a conditional memory module, the model achieves significant intelligence gains without the typical memory overhead.

This technical paper introduces the T1 reinforcement learning method to solve stability issues in long-horizon agent tasks. It specifically focuses on mitigating token drift and routing instabilities in large-scale Mixture-of-Experts models.

PagedWeight optimizes the serving of MoE LLMs by implementing a dynamic quantization strategy that adapts to runtime memory pressure. It balances hardware efficiency with model accuracy by monitoring expert routing statistics and prompt-specific sensitivities.