Evolver#
An Evolver implements the learning method. It receives evidence from task executions and returns the adapter to use next. Read this page to choose a method or implement one that works with the Trainer.
Responsibility and boundaries#
The Evolver decides how to analyze evidence, propose changes, test candidates, and select an adapter. Its state can hold experience, candidate records, or search progress. The Trainer supplies the execution loop, checks the returned adapter, and records the run.
← adapter + resultEvolverState
The same interface supports a learner agent, a scripted procedure, or a search algorithm. A method still needs to understand the editable content of the selected template.
Interface#
The Trainer creates an Evolver instance and calls these methods:
Method |
When called |
Contract |
|---|---|---|
|
Before the first step of a fresh run. |
Optional initialization, such as creating learner state. Per-step material is not available yet. |
|
Once on a fresh run, after |
Optional: change the starting adapter (for example, from the task’s public information); the result is checked and committed as step 0. Not overridden, or returning |
|
Once per training batch, after its execution results have been collected. |
Read the evidence, leave one adapter in the working directory, and report status and usage. Required. |
|
On normal completion or a controlled stop. |
Optional final selection. The default returns the current working adapter. |
step hands back the adapter through the files it leaves in ctx.workspace.adapter.
Its return value describes the call; it does not contain the adapter files.
finalize may return a different adapter directory for the Trainer to collect.
Inputs#
ctx.workspace supplies paths, rather than copies of all the data inside a Python object:
Path |
Contents |
Evolver’s intended access |
|---|---|---|
|
Editable copy of the accepted adapter. |
Read/write. |
|
Current and previous training results. |
Read-only. |
|
Task description and objectives. |
Read-only. |
|
Earlier adapter versions and diffs. |
Read-only. |
|
Method-specific information kept across calls. |
Read/write. |
|
Temporary work, cleared for each step. |
Read/write. |
|
Reports and other call outputs, collected with the step. |
Read/write. |
See the learning material format for its record fields. These are the interface’s access boundaries; sandbox enforcement applies to programs launched through the sandbox, not arbitrary Python code in an in-process Evolver.
The rest of the context supplies:
Field |
Meaning |
|---|---|
|
Evolver defaults merged with this run’s parameters. |
|
Optional validation data and granted environment paths. |
|
Run, task, and template capabilities available to the method. |
|
Human-readable description of the call and available resources. |
|
The learner’s sandbox (it wraps the programs an Evolver starts: the workspace under fixed names below |
|
On step and bootstrap calls: epoch, step, whether the epoch ends here, and, for an Evolver that chooses its tasks, the epoch’s training tasks still to run ( |
|
Optional previous learner-session handle when session resume is enabled. |
The learner interface does not supply test tasks. Whether files remain inaccessible also depends on the run’s filesystem grants and execution isolation.
Outputs#
Output |
Where it goes |
Consumer |
|---|---|---|
Selected adapter |
|
Trainer checks it; the next acting batch uses the accepted version. |
Updated learner state |
|
Later Evolver calls and crash recovery. |
Status and usage |
Returned |
Trainer control flow and accounting. |
Reports |
|
Saved run records. |
@dataclass
class StepResult:
status: Literal["done", "failed", "out_of_budget", "stop"]
summary: str # one line: what changed, or that nothing did
rationale: str = "" # optional: why it changed the adapter this way, or why not
session: str | None = None # handle for resuming next time, if any
usage: list[Usage] = field(default_factory=list) # model usage of this call
extra: dict[str, Any] = field(default_factory=dict) # recorded as given, a model key's value scrubbed
next_tasks: list[str] | None = None # chooses_tasks: the next step's tasks, of progress.remaining
Usage reports model calls, token counts, and cost when known. Calls through the run’s metered model (ctx.agent, or
the llm tool) are already accounted for; do not report the same usage again in StepResult.
Choosing the next tasks#
By default the Trainer takes each step’s tasks in order: the next data.step_size of the epoch. An Evolver that
decides what to learn from next (a curriculum) declares it and names the next step’s tasks in
StepResult.next_tasks:
class Curriculum(Evolver):
chooses_tasks = True
def step(self, ctx):
...
remaining = ctx.progress.remaining # the epoch's training tasks still to run, in the Trainer's order
return StepResult("done", "...", next_tasks=[pick(remaining)])
Only an Evolver with chooses_tasks = True is given progress.remaining (also listed in the call’s message);
for every other Evolver it is empty, so a stream’s later tasks stay unseen. The request is checked when it is made
and holds for the next step only. It must be distinct task ids, at most data.step_size of them, from
progress.remaining; at the last step of an epoch, where nothing remains, from the training split, which the next
epoch runs again; progress.remaining is then empty, so an Evolver that chooses at an epoch’s end keeps the ids it
was given earlier (the bootstrap is given the first epoch’s whole order, and may name the first step’s tasks the
same way). An empty list names nothing. A request is taken from any step result, a failed step’s included; one
made at the run’s last step is recorded and has no step to apply to.
An epoch still runs every training task once: the tasks the Evolver leaves out come later in the epoch, in the
Trainer’s order. The step’s record keeps an accepted request as next_tasks and a refused one as
next_tasks_refused with the reason, and the next step then runs the order’s tasks; a step that ran the Evolver’s
tasks says chosen_by: evolver. A step may hold fewer tasks than data.step_size. An external program
(CommandEvolver) names them as next_tasks in output/step_result.json, from a subclass that sets
chooses_tasks = True.
Status handling#
Status |
What the Trainer does |
|---|---|
|
Check changed files, repair if necessary, record the step, continue. |
|
Keep the previous adapter and record the failed step. |
|
Keep the previous adapter, as for |
|
Process the returned adapter, then stop if |
An exception or wrong return type from step is recorded as a failure. Learner-state snapshots are saved with
completed step records, including failed steps; keeping the previous adapter does not imply rolling back learner state.
Built-in Evolvers#
For selection guidance, setup, and the complete parameter defaults, use Choose a learning method.
|
Method |
Needs |
|---|---|---|
|
A Codex, Claude Code, Pi, OpenCode, DeepSeek Harness or Hermes Agent learner edits the adapter from the learning material; |
The agent CLI |
|
GEPA: one iteration per step. It reflects on the step’s trajectories, keeps a child that does better on the same tasks, scores it on validation, and hands back the next parent from its Pareto front. |
|
|
SkillOpt: one SkillOpt step per training step. It turns the step’s trajectories into a few bounded edits under its learning rate (decaying when |
|
|
ACE: per rollout, a Reflector tags the playbook bullets the agent named and diagnoses the attempt (a wrong answer is run again with the reflection, up to |
|
|
OPRO, with |
|
|
AHE Adapter, with |
The Codex CLI, k rows per task sharing |
|
MetaWorld Model, with |
The Codex CLI, a task that keeps an MWM evidence store ( |
|
ReHarness, with |
|
|
Meta-Harness Adapter, with |
The proposer CLI, |
|
RRSI Adapter, with |
The proposer CLI (Codex, Claude Code or Pi), which also runs the analyst, digesters and critic, |
Change a parameter with evolver.params in the run config; each Evolver’s evolver.yaml under
hermit/agent/evolver/methods/ lists them with their defaults. For gepa, data.step_size is the reflection
minibatch; for skillopt, it is the batch size and trainer.epochs the number of epochs. Both select from candidates
they keep in state/, and finalize hands back the one with the best validation score. For reharness it is 1, one
instance of the stream per step, and finalize hands back the bank as it stands. For meta_harness_adapter it covers
the search set, so a step is an epoch, and finalize hands back the candidate with the best mean score there. For
rrsi_adapter it covers the evolve set with k rows per task, and finalize hands back the incumbent, H_T.
Interaction with the Trainer#
The Trainer first runs tasks using the current adapter, writes their results, and prepares a fresh working copy.
Only then does it call step. Scores in the current material therefore describe the adapter before this update.
The Evolver can keep many candidates internally, but it selects one working adapter for the Trainer. An unchanged
adapter is a valid result. The Trainer owns Git operations; the learner edits files and reads the supplied history.
The working copy is itself a Git repository holding the adapter’s ancestry (this run’s commits back to the
template’s root, nothing else), so git log and git diff work there; changes to its Git metadata are not
collected. The copy a task’s pipeline receives for rollouts has no Git metadata.
For candidate experiments, the run offers:
ctx.tools["run_agent"](task_ids=["task-1"], adapter=str(candidate_directory))
Only training or validation IDs are accepted, and the candidate directory must be inside the run directory.
Omitting adapter tests the current working copy. The response contains per-task status, output, judge results,
cost, and time, how the session ended when the pipeline reports it (exit_code, timed_out: the CLI agents do), and
for training tasks the path of the run’s log; it does not return the trajectories themselves. Its usage counts toward the run’s Evolver budget.
The Evolver may also call check_spec before returning. The Trainer still makes the acceptance decision from the
returned files. Passing specification checks does not establish better performance.
State and session lifetime#
state/ persists across calls and is snapshotted after completed steps. Its format belongs to the method.
The framework restores those snapshots when resuming an unfinished run. Python instance attributes alone do not
survive a process restart.
A host session handle is separate from this state. Set supports_resume = True only when the method can use a
previous session; the user enables it with evolver.resume. Agent-backed methods may start a new host process while
resuming its conversation. A new run started from an adapter begins with fresh learner state.
This separation lets a deployed adapter operate independently of the procedure that learned it.
Minimal example#
An Evolver can leave the adapter unchanged while updating its state. This method records observed scores without calling a model or performing optimization:
class RecordOnlyEvolver(Evolver):
def step(self, ctx: StepContext) -> StepResult:
summary_path = ctx.workspace.learning_material / "current" / "summary.json"
summary = json.loads(summary_path.read_text())
state_path = ctx.workspace.state / "observations.json"
observations = json.loads(state_path.read_text()) if state_path.exists() else []
observations.append({"step": summary["step"], "mean_score": summary["mean_score"]})
state_path.write_text(json.dumps(observations, indent=2) + "\n")
(ctx.workspace.output / "report.md").write_text("Recorded feedback; adapter unchanged.\n")
return StepResult(status="done", summary="Recorded feedback; adapter unchanged")
The custom Evolver guide provides its imports, manifest, and an offline end-to-end command.
For the exact context and result types, see Python types. Execution and budget limits are covered in Trainer.