arXivAvni Mittal, Avinash Anand, Ashutosh Kumar, Dikshant Kukreja, Kritarth Prasad, Chaitanya Lakhchaura, Rishitej Reddy Vyalla, Erik Cambria, Zhengkui Wang, Rajiv Ratn ShahMon, Nov 2, 2026, 10:30 AM PST
score 16.9
Training Trick Cuts GPU Memory for Large Mixture-of-Experts Models
Original: Sonar-MoE: In-Backward Expert Updates for Memory-Efficient Mixture-of-Experts Training
Source: arxiv.org ↗
Writing ELI5 summary…