A new, more realistic benchmark from Alibaba tests AI coding agents with real user requests. The top-ranked model, GPT-6 Astra, fully solves only 26.4 percent of the tasks.
Alibaba Benchmark Finds AI Coding Agents Need Human Rework on Most Tasks
A new, more realistic benchmark from Alibaba tests AI coding agents with real user requests. The top-ranked model, GPT-6 Astra, fully solves only 26.4 percent of the tasks.
Monitoring item. The full text is not distributed. Extract and source below.