Sentra Code Memory on Terminal-Bench 2.1

Every headline measure improved at once

Sentra evaluated Codex CLI running GPT-5.5 at xhigh reasoning effort — the configuration that currently leads the public Terminal-Bench 2.1 leaderboard — with Sentra Code Memory made available to the agent as a memory tool. The evaluation covered all 89 Terminal-Bench 2.1 tasks at the standard five trials per task, for 445 trials in total.

• Accuracy. The Sentra-enabled agent succeeded on 393 of 445 trials, a mean reward of 88.31%, versus 371 of 445 (83.37%) for the published baseline — an improvement of 4.94 points and 22 additional successful trials.
• Cost. Total model cost was $510.30, versus $1,862.98 for the baseline, a 72.6% reduction. Cost per successful trial fell from $5.02 to $1.30.
• Tokens. The run consumed 663.5 million tokens under the leaderboard's display convention, versus 1.128 billion for the baseline, 41.2% fewer.
• Consistency. Tasks solved in all five trials rose from 63 to 68, and tasks that failed in all five fell from 5 to 3.

The base agent, model, reasoning effort, benchmark harness, and scoring were identical across the two configurations. The only change was the availability of a task-scoped code memory layer, which the agent could query for relevant development context instead of repeatedly rediscovering it.

Where the result sits

At 88.31% mean reward, the Sentra-enabled configuration scores 4.94 points above the highest published entry. Because this result has not yet passed official verification, it is presented as an internal evaluation rather than a leaderboard ranking.

A memory layer, not a bigger window

What it is

Sentra builds memory infrastructure for teams and AI agents: a shared memory system that captures interactions, decisions, and evidence into a queryable structure, keeps answers connected to their supporting evidence, and tracks how information changes over time. Sentra Code Memory applies this capability to coding agents. It gives an agent a task-scoped memory of development state — repository structure, relevant code context, tool activity, file changes, test signals, and continuation context — exposed through a CLI, an SDK, and a local API. Rather than repeatedly re-scanning a repository to rebuild context, the agent retrieves compact, relevant development state at the moment it needs it.

Memory is not a bigger context window

A larger context window gives a model more room, but it does not decide what should be remembered, when a fact has gone stale, which evidence matters, or how to separate durable task state from incidental output. Sentra Code Memory operates as a memory layer around the coding workflow rather than as additional raw capacity. The model continues to write code, run commands, and reason about failures; the memory layer provides a scoped recall channel the agent can query as it works.

Why this should matter on Terminal-Bench

Terminal-Bench tasks typically demand repeated repository inspection, build-system discovery, test interpretation, and incremental debugging. Without a memory layer, an agent pays model tokens to rediscover context it has already seen, often several times within a single run. With task-scoped memory available, the agent can preserve and recall relevant state across the run, reducing redundant discovery and leaving more of its budget for edits and verification.

Configuration and accounting

Memory isolation

Each task ran with isolated memory state. The task repository was indexed before the agent loop began, and an index watcher kept the memory current as the agent edited files during the run. Memory visible to the agent was scoped to the current task's repository rather than to any shared or global workspace.

Cost and token accounting

Model-cost figures reflect the source-reported model cost from the run artifacts. Sentra-side retrieval costs, limited to embedding and reranking, are not included in the model-cost field. To keep comparisons like-for-like, token counts follow the public task-detail conventions: API tokens are input + output; leaderboard-display tokens are input + output + cache. Model cost remains the most direct budget measure, since it comes from the source-reported cost field rather than a derived token formula.

Headline comparison

Reliability and consistency

Head-to-head at the task level, the Sentra-enabled configuration improved on 20 tasks, tied on 61, and regressed on 8, for a net gain of 22 successful trials. The outcome distribution shifted toward consistency: more tasks solved in every trial, fewer failing in every trial.

Spending less for more

Read this as: the public baseline spent 3.65× more model budget to achieve a lower score. Because the Sentra run also succeeded more often, the gap widens on a per-success basis, to 3.87×.

The savings are dominated by input tokens (−52.1%), with output tokens nearly unchanged (−13.0%) — a pattern consistent with the agent ingesting far less repeated context while producing a comparable volume of work.

Where the gains concentrate

The largest gains concentrate in tasks that demand heavy environment and repository discovery, where the baseline spent large budgets rebuilding context. On train-fasttext, the Sentra-enabled agent flipped zero baseline successes into two while cutting cost from $138.90 to $14.70; on compile-compcert it reached 5/5 at $13.47 versus $99.89. Of the five tasks the baseline failed in every trial, the Sentra-enabled agent solved four at least once.

Across the run, the Sentra-enabled agent gained 33 trials on 20 tasks and gave back 11 trials on 8 tasks, for the net improvement of +22. The regressions are concentrated in a small set of tasks and are under analysis ahead of the official benchmark submission.

Three measures moved together

Improvements on agentic benchmarks typically trade off against cost: higher scores are usually bought with more sampling, longer runs, or heavier reasoning. In this evaluation, the Sentra-enabled configuration scored higher while spending 72.6% less and consuming 41.2% fewer tokens than the public baseline.

The most plausible explanation is reduced rediscovery. Terminal-Bench rewards sustained, multi-step work inside a repository. With task-scoped memory available, the agent recalls context on demand and directs more of its budget toward edits, tests, and verification.

The composition of the token savings supports this reading: input tokens fell by 52.1% while output tokens fell by only 13.0%. The distribution of outcomes suggests the gains reflect greater reliability rather than a handful of fortunate trials — tasks solved in all five trials rose from 63 to 68, and four of the five tasks the baseline never solved were solved at least once with memory available.

Internal evaluation, pending verification

The results in this report were produced by Sentra using the official Terminal-Bench 2.1 task set, harness, and scoring, but they have not yet been verified by the Terminal-Bench team, and Sentra Code Memory does not currently appear on the public leaderboard. Before any leaderboard claim, the configuration will be rerun through the official submission path, and that run will serve as the source of truth.

• Baseline snapshot. Baseline figures were recomputed from the published per-task trial data for the public Codex CLI + GPT-5.5 (xhigh) entry as of June 2026. Public leaderboard data may be revised over time.
• Cost scope. Model-cost figures reflect the agent's reported model usage. Sentra-side retrieval costs (embedding and reranking) are not included.
• Generalization. This evaluation covers a single benchmark and a single base agent and model configuration. Results on other benchmarks, agents, or models may differ.

What happens next

Sentra will submit this configuration through the official Terminal-Bench 2.1 verification process and will release the complete run trajectories so the community can inspect the agent's behavior directly. Sentra Code Memory itself will be made available shortly. For questions about this report, early access, or partnership inquiries, contact ashwin@sentra.app or visit sentra.app.

1. Terminal-Bench 2.1 leaderboard. tbench.ai/leaderboard/terminal-bench/2.1
2. Public Codex CLI + GPT-5.5 (xhigh) task-detail pages, Terminal-Bench 2.1.
3. Terminal-Bench benchmark index. tbench.ai/benchmarks
4. Sentra. sentra.app
5. Sentra Terminal-Bench 2.1 run artifacts (445 trial records; trajectories to be released publicly).

Complete per-task results

Per-task results for all 89 Terminal-Bench 2.1 tasks: successful trials out of five for each configuration, the difference, total model cost per task, and total tokens per task under the leaderboard display convention (input + output + cache).