Why momentum helps Muon optimizer in AI training

Original: Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering

Source: arxiv.org ↗