Mathematics-first approach to optimizing gradient computations in AI training

Original: I am a big fan of Jianlin Su's blog because it always starts from first principles in mathematics, rather than "ML tricks", to approach a typical ML problem (eg. training-free MoE load balancing).

Source: x.com ↗