AKV proposes managing long-running agents’ KV caches directly instead of trimming the conversation and reprocessing retained text. Its drop operation excludes selected cache entries, while repos adjusts retained keys’ positions. The released serving engine handles the fact that identical tokens can represent different cache states after different operations. In the authors’ experiments, a rolling-drop policy cut average cost by 27.7% and improved average scores by 1.7 points over threshold-triggered summarization. These results depend on the tested policies and benchmarks; the method does not preserve unlimited context or guarantee quality on every task.
Back to Tech
AKV manages agent KV caches without rewriting the conversation
Drop and reposition operations aim to reduce costly re-prefilling