Multi-token prediction technique triples LLM inference speed without auxiliary draft models
High inference latency and spiraling GPU costs have emerged as the primary bottlenecks for IT leaders deploying agentic AI systems. These workflows often generate thousands of tokens per query, creating a performance gap that current hardware struggles to bridge. Now, researchers from the University of Maryland, Lawrence Livermore National Labs, Columbia University, and TogetherAI say…