LoRA

This research presents LESSER, a new technique for selecting optimal instruction-tuning data using output-layer gradients. The method achieves performance comparable to state-of-the-art selection methods while being significantly more computationally efficient.

This technical report explores how the placement of LoRA adapters within Transformer layers affects multi-hop reasoning capabilities. It concludes that targeting early layers is critical for preventing models from 'stopping thinking' too early during complex inference loops.

The paper presents READ, a novel framework for merging independently trained LoRA adapters by fixing factor coordinates and coupling direction. This approach prevents interference between tasks and allows for efficient, single-model deployment without additional inference costs.

Security researchers have proven that open-weight AI models can be maliciously backdoored using minimal resources and training data. This discovery highlights a critical lack of observability and verification capabilities in the current AI supply chain.

This technical repository presents SRM-LoRA, a novel approach designed to minimize hallucinations in LLMs via specialized metric updates. The research demonstrates superior performance compared to standard LoRA techniques across various ablation studies.

This paper proposes ReChannel, a novel architecture that transforms text-to-image models from RGB generators into efficient dense prediction engines. By treating transformer tokens as spatial carriers for task-specific data rather than RGB pixels, the method achieves new state-of-the-art performance with much higher computational efficiency.

This paper presents a method for converting pretrained transformers into linear-time architectures by focusing on the efficiency of state update designs. By analyzing softmax attention through a first-order approximation, the authors prove that delta-style updates are superior for post hoc linearization.

The researchers present Program-as-Weights (PAW), a paradigm that shifts LLM usage from expensive per-input API calls to a 'compile-once, run-locally' model using neural adapters. This approach allows small, specialized models to outperform massive foundation models on specific fuzzy tasks while maintaining high efficiency and privacy.

This paper presents Distill to Detect (D2D), a technique for identifying stealthy, topic-specific biases in language models that evade standard detection. By distilling the distributional shift of a suspected model into a small prefix adapter, the method amplifies hidden signals until they become visible in generated text.