Computer-use agents are typically constrained by interfaces that humans design by hand, and these interfaces often force inefficient behavior: an agent might fill a spreadsheet cell by cell instead of running a single script, or scroll through thousands of lines of a file in a shell just to locate one function. A new paper argues that this bottleneck can be addressed by making interface design itself a capability the agent learns, rather than a fixed scaffold engineers must maintain by hand for every new application or model.
The authors present Agent S4, described as an autonomous computer-use scientist that constructs what they call Evolving Agent Interfaces directly from task experience. A meta-agent reviews trajectories generated by task-solving agents on synthetic training tasks, without relying on success labels, and uses these observations to build and revise a repository of interfaces and usage knowledge. A population-based evolutionary process then decides which repository variants are worth developing further, based on their performance on validation tasks the meta-agent never sees, which favors modifications that generalize rather than overfit to the training set.
On the OSWorld V2 benchmark, Agent S4 paired with the Qwen3.8-27B model and its evolved interface repository achieves a binary score of 21.3%, nearly four times higher than the same base model using fixed, hand-built interfaces. That figure also surpasses GPT-5.5 (13.0%) and Claude Opus 4.8 (20.6%) on the same test, while reportedly costing only about 4.3% and 1.5% of what those two models respectively require for inference. On a separate benchmark, ScienceBoard, the evolved interfaces lift the success rate of the same underlying model from 41.4% to 58.6%.
The results point to interface design as a lever for improving agent performance that is distinct from simply scaling up models or refining prompting strategies, and one that can be automated rather than hand-engineered for each new workflow. The authors have open-sourced their code, which should allow other research groups to test whether the approach holds up across different model families and task domains beyond the two benchmarks reported here.