Standard GPUs achieve 3,000 tokens per second inference speed

Original: Real-time LLM Inference on Standard GPUs: 3k tokens/s per request

Source: blog.kog.ai ↗