A team of researchers has introduced TasteVal, a benchmark designed to assess what they call the 'experimental research taste' of frontier AI models — their ability to choose promising problems, design experiments, and draw sound conclusions from results. Rather than measuring raw task performance, the benchmark focuses on compute efficiency: a system that matches an expert's results using half the experimental compute is considered to have twice the research taste. The authors argue this efficiency multiplier is a crucial but often overlooked factor in forecasting AI progress, since it determines how much useful research output a given amount of compute can generate.
The benchmark comprises eight open-ended tasks modeled on real frontier AI R&D problems. To isolate research judgment from coding skill, evaluated models act purely as 'Researchers' who design experiments, while a separate fixed 'Coder' agent handles implementation and reports results back. Each run is capped at either 40 H100-hours of compute or 120 wall-clock hours. The researchers recruited 24 human experts, with at least two assigned to each task, and used the best human attempt per task as the baseline for comparison.
Across 20 models released between 2023 and 2026, the top performer — referred to as Opus 5.5 — surpassed the human expert baseline, achieving a compute efficiency multiplier of 2.3x, while costing roughly one-thirtieth as much per run as the human baseline. More notably, the study found that this efficiency multiplier has been doubling approximately every three months since December 2025, a sharp acceleration from the roughly 14-month doubling period observed between 2023 and late 2025.
Interestingly, when measured purely by normalized task performance rather than compute efficiency, the researchers found no such acceleration — models continued improving at a steady pace, doubling performance roughly every 14.6 months. This suggests the apparent trend break is specific to how efficiently models use compute to reach a given level of research quality, not to their absolute capability ceiling. The authors chose not to release the underlying tasks publicly, in order to avoid contamination of future training data and preserve the benchmark's usefulness for tracking this trend over time.