Taste-Bench extracts 502 decision points from coding and research execution traces and asks models to choose the better next action without seeing later developments. Both the original and reversed order of two options must be answered correctly, giving random choice a 25% baseline. The highest reported mean was 59.7%, with particular weakness where decisive evidence emerged only later. A follow-up training experiment reported success on 41 held-out tasks rising from 14.6% without advice to 33.7% with learned decision advice. These results concern a small task set and particular traces, not general workplace performance.
Back to Tech
Taste-Bench evaluates long-horizon branch decisions with 502 questions
Passing final tests and making sound long-term decisions are distinct abilities; check the rule requiring success with both option orders.