Wireva

Meta researchers train 8B AI model to match Claude Opus 4.5 with new harness framework

Meta AI and University of Illinois researchers have developed EvoHarness-RL, a framework that teaches small AI models to manage external memory and tools effectively, allowing an 8B model to match the performance of frontier systems like Claude Opus 4.5 on complex enterprise tasks.

Researchers at Meta AI and the University of Illinois Urbana–Champaign have developed a training framework that allows a small 8-billion-parameter language model to match the performance of frontier systems such as Claude Opus 4.5 on complex, long-horizon enterprise tasks. The framework, called EvoHarness-RL, addresses a critical bottleneck in AI agent deployment: teaching models to manage the external tools, memory, and execution feedback they rely on during multi-hour workflows.

Modern AI agents cannot depend solely on their internal context window when handling jobs that span hours, such as migrating large batches of customer records from a legacy CRM to a cloud database. Instead, they rely on a runtime layer, known as the harness, which provides execution feedback like server logs, state trackers, and control-flow mechanisms. The harness helps the agent maintain an accurate understanding of dynamic API connections, track completed and pending subgoals, and recover from unexpected errors such as a database rejecting a batch due to strict rate limits.

Traditionally, developers write rigid, step-by-step instructions telling the agent how and when to use its tools. This approach is labour-intensive and fragile. Xuying Ning, co-author of the EvoHarness-RL paper, told VentureBeat that manual logic and rigid memory structures are the primary culprits draining engineering resources. “The optimal harness often changes with the model,” Ning explained. “Different models may need different prompts, memory designs, permissions, or sandbox configurations. If all of this logic is manually coded, every model upgrade can lead to another long cycle of tuning and debugging.”

Existing memory systems that simply accumulate experience can also degrade an agent’s reasoning. “Append-only memory assumes that more context is always helpful, which is not necessarily true,” Ning said. “Over a long task, the memory may contain outdated conclusions, failed attempts, or information that is no longer relevant.” Long-horizon agents therefore need a dynamic memory capable of updating, compressing, and replacing information to avoid repeating past mistakes.

EvoHarness-RL replaces hardcoded instructions with a training technique that teaches the agent to construct a structured workspace from messy execution data and decide when to consult that external state during complex workflows. The framework consolidates the agent’s support systems into a single interface called the Belief, Progress, and Experience (BPE) workspace. Belief maintains an accurate read on the current environment, Progress manages completed and pending subgoals, and Experience reuses historical knowledge across tasks.

Instead of using complex, domain-specific APIs, the AI interacts with this dashboard using four compact meta-actions: track, commit, recall, and note. It issues commands to track the live environment, commit to workflow updates, recall past strategies before acting, and write notes to save newly discovered insights for future runs. These states map directly to high-value enterprise verticals. In software engineering, Belief can represent the agent’s current understanding of the repository, while Progress tracks what has been completed and which steps depend on others. Experience captures lessons, such as user feedback on a mistake, to guide future actions.

The same idea applies to finance, Ning said. During a compliance audit, Belief might describe the applicable rules and available evidence, Progress tracks which checks have been completed and which exceptions remain open, and Experience helps the agent recognise recurring discrepancies or know when an issue should be escalated. “Together, these states help prevent the agent from losing track of its work or repeating the same failed approach,” Ning said.

To teach the agent both the mechanics and the strategy of managing its external workspace, the researchers designed a two-stage training recipe. In the first stage, supervised harness fine-tuning, the base model learns how to extract and structure useful facts from messy interaction data. The second stage uses reinforcement learning to teach the agent when to read, update, or consolidate the information it obtains from its environment, rather than blindly following a script.

The results are significant for enterprise adoption. By demonstrating that an 8B model can match the performance of a frontier system like Claude Opus 4.5 on long-horizon tasks, the research suggests that organisations may not need to pay premium prices for massive models when smaller, more efficient ones can be trained to use their tools effectively. This could lower the cost barrier for deploying AI agents in complex operational environments, from IT migrations to financial compliance.

Same event, other desks

Story file →