Study reveals exact instability boundary for AI model training with normalization

Original: When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay

Source: arxiv.org ↗