How transformer models secretly track absolute position despite relative encodings

Original: Where does Absolute Position come from in decoder-only Transformers?

Source: arxiv.org ↗