arXivCraig W. Schmidt, Michael Krumdick, Adam Wiemerslage, Seth Ebner, Varshini Reddy, Yuval Pinter, Chris TannerThu, May 21, 2026, 9:46 AM PDT
score 14.7
New tokenization method packs text into fewer tokens than BPE
Original: Tokenization with Split Trees
Source: arxiv.org ↗
Writing ELI5 summary…