SIFT, developed by researchers at MIT and Sakana AI, searches for improvements by modifying a coding agent’s implementation rather than model weights. It combines pairwise LLM comparisons of patches into Bradley–Terry scores, expands promising candidates first and selectively performs costly real evaluations. The paper reports roughly 35% performance on an o3-mini-based Polyglot evaluation using 50 CPU-hours and less than five hours of wall-clock time. This is a harness-improvement result on a particular benchmark, not a replacement for correctness verification using only a judging model.
Back to Tech
SIFT uses LLM judging to lower the search cost of agent self-improvement
Fast proxy evaluation prioritizes the search, while final code performance is checked by executing real tasks.