KV cache compaction beats text summaries for long AI tasks

Original: No matter how good you get at summarising, it's much easier to compress in latent space than token space. This is part of the reason we're so excited about neural KV cache compaction, particularly for

Source: x.com ↗