Training loss predicts how language models scale on real tasks

Original: RT @sivareddyg: Nature is complex. Why would cross-entropy loss predict scaling behavior of language models on downstream task? Introducing…

Source: x.com ↗