Why normalized AI training speeds up then breaks down

Original: A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & Scaling

Source: arxiv.org ↗