Positional encoding may not help transformers handle changing token distances

Original: Distance generalization in transformers: why bother with positional encoding?

Source: arxiv.org ↗