Llama-3.2-3B

This research presents LESSER, a new technique for selecting optimal instruction-tuning data using output-layer gradients. The method achieves performance comparable to state-of-the-art selection methods while being significantly more computationally efficient.

The paper presents READ, a novel framework for merging independently trained LoRA adapters by fixing factor coordinates and coupling direction. This approach prevents interference between tasks and allows for efficient, single-model deployment without additional inference costs.

The researchers present RIDE, a technique that improves language model distillation by extrapolating RL-induced residuals in the hidden-state representation space rather than the logit space. This approach avoids the noise and attenuation caused by the language-model head, allowing student models to match or surpass their RL-trained teachers.