Benchmark · continual learning from serving experience

ServeLearnBench

How well can agents self-improve from serving experience?

Haizhong Zheng1, Yizhuo Di1, Ranajoy Sadhukhan1, Shuowei Jin2, Beidi Chen1

1Carnegie Mellon University  ·  2Amazon

PaperCodeEvaluate an agent →

How well can agents self-improve from serving experience? Agents in real-world services need to keep learning to remain reliable as policies and customer needs change. The knowledge they need is often undisclosed, so they must infer it through interactions and failures. ServeLearnBench measures how well agents learn from serving experience, how that learning affects future behavior, and what it costs.

MethodPrimeCHRAGMem0SkillOpt
ModelGLM-5.3 FlashDS-V4.1 FlashGLM-5.3Kimi K3Opus 5GPT-5.6 Terra

Cost vs. performance. Average Hidden reward versus total tokens and API cost per test task. Lines mark the Pareto frontier: no other pair is both cheaper and better.

Leaderboard

Full leaderboard with charts →Evaluate your method →
Benchmark Design

How ServeLearnBench works

Agents serve evolving task streams in Retail, Banking, and Pitch. Across three difficulty tiers, they must learn new environment knowledge, revise it as conditions change, and handle returns to earlier states.

The protocol
1 · Scale. Nine streams, 53 windows, 7,718 tasks.2 · Serve and learn. One stream, schematic. Serving tasks return a score to the agent's memory; test tasks return nothing. From scores alone, the agent infers a hidden policy that changes at window boundaries (red lines).
3domains
9scenarios
53environment windows
4,508serving tasks
3,210held-out test tasks
28model–method pairs

Explore the benchmark →

Findings

What we found

Agents can solve tasks, but still struggle to learn.

Models average 95.4 on Hidden tasks with the policy disclosed (Oracle), versus 14.1 without it (Blind). Yet on Retail L3 the best of 28 learning pairs reaches only 59.6 (median 22.0).

Learning is expensive and can hurt performance on tasks agents could already solve.

Inferring a hidden policy costs 4 to 101 times as much as following a disclosed one (Kimi K3: Prime $2,479 vs. Oracle $102), and it can degrade behavior that needed no learning.

Exploration is critical for harness evolution and continual learning.

Harnesses that try more varied answers tend to perform better on Hidden tasks. Across the five harnesses, answer diversity and Hidden-task reward have exactly the same ranking (Spearman rank correlation: 1.00).

Finding the right behavior is often harder than reusing it.

Once Prime and CH find the correct behavior, they quickly reuse it, averaging 78% and 76% correctness over the next five encounters. For these stronger harnesses, discovering useful behavior appears to be a larger bottleneck than reusing it.

Analysis

How the scores come about

How reliably do agents learn, what does learning cost, and which behaviors distinguish effective learning harnesses?

Methods that explore more tend to learn better

The five harnesses have exactly the same ranking by exploration and Hidden reward (ρ = 1.00). Prime and CH try more distinct answers, while weaker learners often keep repeating the same behavior.

(a) Diversity by method

(b) Reward vs. exploration

(c) Diversity by model

(d) Reward vs. exploration

Exploration predicts adaptation. Mean number of distinct answers after k Hidden serving tickets of a policy, and its area (exploration AUC) against mean Hidden reward over the six Retail and Banking settings; ρ is the Spearman rank correlation.

A large learning gap remains

Across the four open-weight models, Prime and CH achieve average Hidden scores of 61.7 and 57.6, ahead of the other learners at 38.3–43.4. Yet even the strongest learner remains far below Oracle's 94.5.

(a) By model

(b) By method

Hidden and Fully Specified test scores, averaged by model and by method over the four open-weight models.

Higher reward comes at a substantial cost

Prime achieves 59.0 Hidden reward at $0.195 per task. CH reaches 55.1 at $0.083, delivering nearly as much reward at less than half the cost.

(a) By model

(b) By method

Cost against reward. Average API cost of one task, serving stream (learning work included) and held-out test pooled, with Hidden and all-case reward averaged over the nine settings.

Learning speed depends strongly on the harness

Prime and CH learn new policies fastest in both domains. After five examples of a Banking policy, both answer roughly 60% of subsequent tasks correctly; progress is slower in Retail.

(a) Banking

(b) Retail

(c) Banking AULC10

(d) Retail AULC10

How fast does a method learn a new policy? Correctness on a policy's Hidden serving tickets against how many earlier tickets of the same policy it has seen, and AULC10, the mean correctness over the first ten.

Learning can damage already-correct behavior

On Fully Specified Retail tasks, every learning harness falls below Blind's 98.9% accuracy, reaching only 89.5–95.5%. This loss is not universal: Prime and Mem0 improve on Blind in Banking.

(a) Retail

(b) Banking

Fully Specified accuracy, averaged over the four open-weight models and three tiers in each domain.

Get started

Evaluate an agent

Find the code, installation instructions, and evaluation guide on GitHub.

Get started on GitHub →
Citation

Cite ServeLearnBench

@misc{zheng2026servelearnbench,
  title  = {ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?},
  author = {Zheng, Haizhong and Di, Yizhuo and Sadhukhan, Ranajoy and Jin, Shuowei and Chen, Beidi},
  year   = {2026},
  eprint = {2610.07792},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url    = {https://arxiv.org/abs/2610.07792}
}