When do attention circuits form in language models

Original: When Do Attention Circuits Form? Developmental Trajectories of Capability and Attention-Sink Emergence Across Three 1B-ClassArchitectures

Source: arxiv.org ↗