Cache memory protection beats clever eviction algorithms for language models

Original: Protection Is (Nearly) All You Need: Structural Protection Dominates Scoring in Globally Capped KV Eviction

Source: arxiv.org ↗