inferwire
/
AI·4 min read

E3 Trims Coding-Agent Over-Reading; Real-Model Gains Are Smaller

A University of Tennessee paper has agents estimate a task's scope before acting. In a simulator, cost fell 85 percent; with a real gpt-4o agent, tokens fell 18 percent against a read-everything prompt.

TL;DR

  • E3 makes coding agents estimate a task's scope first and widen it only when verification fails, cutting simulated cost by 85 percent at equal success [^1].
  • With a real gpt-4o agent the effect was milder: E3 used 18 percent fewer tokens than a read-everything prompt and 4 percent fewer than ReAct [^1].

Background

LLM agents now edit code across whole projects. The paper opens with a deliberately trivial case: on a personal website, swap one e-mail icon for the markup the page already uses for another. It's a two-line change, yet a capable agent may spend several minutes re-reading the icon library, the site directory and the project architecture [^1]. The authors call this a maximum-context-first strategy: gather as much context as possible, then eliminate every conceivable risk [^1].

What happened

Junjie Yin and Xinyu Feng of the University of Tennessee, Knoxville, propose E3, short for Estimate, Execute, Expand; Yin is also affiliated with Microsoft [^1]. The agent first estimates difficulty, scope and risk from cues in the instruction and at most one cheap search. It then runs the smallest path that could pass verification, and only if verification fails or its confidence is low does it widen the scope one level, reusing what it already found [^1]. The authors also define the Agent Cognitive Redundancy Ratio (ACRR), which measures wasted effort against an oracle's minimum-sufficient trajectory [^1].

To measure this exactly, they built MSE-Bench: 121 procedurally generated edit tasks in a deterministic simulator where every policy can make an edit once it has seen the relevant content, so only the amount of context it gathers varies [^1]. Level 1 tasks are single-file replacements, Level 2 tasks rename a symbol across a few files, and Level 3 tasks are repository-wide refactors with one dependency hidden behind an import alias; 18 of them use deceptively local wording [^1]. No language model runs in these experiments, and the estimator is a hand-written rule [^1].

A max-context-first policy, which the authors describe as an upper-bound stress model, solved every task at a mean cost of 122.9, inspecting 8.5 files and spending 4,421 tokens per task [^1]. A fixed ReAct loop of search, read, edit and test solved only 66.9 percent and failed every Level 3 task. An adaptive retrieval baseline solved everything at a mean cost of 22.1, and E3 solved everything at 18.6 [^1]. Against max-context-first, E3 cut tokens by 90.9 percent, inspected files by 92.2 percent and overall cost by 84.9 percent; against adaptive retrieval, it cut cost by 16.0 percent [^1].

Why it matters

The authors are open about the limits of the simulation. Wasted effort was highest on the simplest tasks, with a ratio of 22.1 at Level 1 against 5.4 at Level 3 for max-context-first, but they call this partly mechanical: that policy's cost barely changes while the minimum grows with each level [^1]. On Level 3, adaptive retrieval was leaner than E3, because E3 starts with a lean guess and pays for an extra expansion on the deceptive tasks [^1]. Removing the Expand step dropped success to 85.1 percent. With paraphrased instructions that broke the estimator's keyword cues, its accuracy fell from 85.1 to 66.9 percent, yet E3 still solved every task and its cost rose by 8.7 percent [^1]. The cheap estimate only works because verification and expansion catch its misses.

The real-model test is more modest. In LLM-Case, gpt-4o edits the real toml 0.10.2 library and every patch is graded by running pytest [^1]. Even when told to read everything first, gpt-4o inspected only one to four files [^1]. E3 used 18 percent fewer tokens and ran 18 percent faster than that thorough agent, and 4 percent fewer tokens and 5 percent faster than ReAct, at effectively equal success [^1]. On the trivial edits, the three policies were within noise of each other. On the deceptive Level 3 refactor, the thorough agent failed all three runs, once by hitting the provider's rate limit, while E3 succeeded in both of its runs that weren't throttled [^1]. The authors present this as a case study with three runs per task on one model, not a powered benchmark [^1].

Practical example

Say you ask a coding agent to bump the version number of a Python package. The change touches one file. Going by this paper, an agent told to read everything first pays for context it doesn't need, and the waste is largest in relative terms on exactly such simple tasks [^1]. With an E3-style approach, the agent finds the one site, edits it, runs the tests and only widens its search if they fail [^1]. For a rename that sounds local but is subclassed elsewhere, like the paper's decoder base class, that failing test is what sends it to the second file [^1]. Don't expect big savings on the trivial edit itself: with gpt-4o, those runs were within noise [^1].

Related gear

We recommend this book because it applies ideas from computer science to everyday decisions, including a chapter on when thinking less leads to better results.

AdvertisementAmazon

Algorithms to Live By: The Computer Science of Human Decisions

Sources

  1. [1]arXiv — Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution