New system predicts request bursts to speed up AI servers

Original: Predictive Speculative KV Replication for Bursty LLM Inference

Source: jwlabs.vercel.app ↗