Papers

Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, Peng Li, Wei Wei, Jing Shao, Chaochao Lu, Yue Zhang, Xian-Sheng Hua, Bowen Zhou, Yu Cheng (2025). A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond. In arXiv preprint arXiv:2503.21614.

Weigao Sun, Disen Lan, Tong Zhu, Xiaoye Qu, Yu Cheng (2025). Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts. In ICLR 2025-SCOPE Workshop (Oral).

Disen Lan, Weigao Sun, Jiaxi Hu, Jusen Du, Yu Cheng (2025). Liger: Linearizing Large Language Models to Gated Recurrent Structures. In arXiv preprint arXiv:2503.01496.

Jusen Du, Weigao Sun, Disen Lan, Jiaxi Hu, Yu Cheng (2025). MoM: Linear Sequence Modeling with Mixture-of-Memories. In arXiv preprint arXiv:2502.13685.

Weigao Sun, Disen Lan, Yiran Zhong, Xiaoye Qu, Yu Cheng (2025). LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid. In arXiv preprint arXiv:2502.07563.

Weigao Sun, Et Al (2025). Minimax-01: Scaling Foundation Models with Lightning Attention. In arXiv preprint arXiv:2501.08313.

Xiaoye Qu, Daize Dong, Xuyang Hu, Tong Zhu, Weigao Sun, Yu Cheng (2024). LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training. In arXiv preprint arXiv:2411.15708.

Xuyang Shen, Dong Li, Ruitao Leng, Zhen Qin, Weigao Sun, Yiran Zhong (2024). Scaling Laws for Linear Complexity Language Models. In EMNLP 2024.

Zhen Qin, Weigao Sun, Dong Li, Xuyang Shen, Weixuan Sun, Yiran Zhong (2024). Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention. In ICML 2024.

Zhen Qin, Xuyang Shen, Dong Li, Weigao Sun, Stan Birchfield, Richard Hartley, Yiran Zhong (2024). Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective. In ICML Workshop 2024.