RaBitQCache Uses Binary Quantization to Speed Up Long-Context AI Inference

Original: RaBitQCache: Rotated Binary Quantization for KVCache in Long Context LLM Inference

Source: arxiv.org ↗