Configuration field reference#
The framework merges built-in defaults, your YAML, and command-line overrides, in that order. This page covers every field in the default configuration. Method-specific parameters are included directly from their manifests in the learning-method pages.
Paths, values, and overrides#
Filesystem paths interpreted by the framework are relative to the config file’s directory, not the shell’s
working directory. They support home and environment-variable expansion through the config path resolver.
File callables such as pipeline.py:run also resolve against the config directory. Dotted Python modules use
the active Python environment. Values inside free-form method parameters can have their own path rules.
hermit train --config path/to/run.yaml trainer.run_name=trial-02 data.step_size=4
hermit eval --config path/to/run.yaml --adapter final trainer.run_name=trial-02 eval.repeats=3
Overrides are YAML values; quote shell-sensitive lists and dictionaries. Unknown framework keys are rejected.
evolver.params, agent.params, pipeline.params, llm and backends are free-form mappings at merge time; this is not a
guarantee that their consumer supports arbitrary fields.
Data and task#
Field |
Type / default |
Meaning and constraints |
|---|---|---|
|
path / null; required |
JSONL with unique |
|
path / null; required |
One training task id per line |
|
path / null |
Validation ids; required by methods declaring validation-data needs |
|
path / null |
Test ids; a nonempty test split is required by |
|
integer / 8 |
Positive number of task executions in a main training batch |
|
boolean / true |
Shuffle training order; set false for ordered streams |
|
integer / 0 |
Training shuffle seed; does not seed every model or method |
|
directory / null |
Learner-facing task description; provide |
|
string / null; required |
Built-in |
|
integer / 4 |
Maximum concurrent rollout/evaluation task executions; use a positive value |
|
number / null |
A sandboxed task call (its worker) running longer is stopped and recorded as an error; null: no limit |
|
mapping / {} |
Passed to Agent constructors; plain functions receive it when they declare a |
|
string / null |
Layer 2 in the per-layer form (Agent layers): |
|
string / mapping / null |
Layer 1: a |
|
mapping / null |
Layer 2’s settings for that Base Agent (the one |
|
string / null |
Legacy alias for |
|
integer / null |
Legacy concurrency override when the legacy pipeline alias is used |
|
mapping / {} |
Legacy parameters merged into |
|
file/module / null; required |
Module containing the scoring callable |
|
string / |
Callable accepting |
Split ids must exist in the task data. Use disjoint train/validation/test splits; check reports overlaps.
Task row fields become training material, so private labels require judge-owned storage.
Adapter and learning method#
Field |
Type / default |
Meaning and constraints |
|---|---|---|
|
string / null; required |
Built-in id, |
|
string / null |
Initial adapter tag, commit, or compatible directory; does not restore learner state |
|
string / null; required |
Built-in method or directory containing its Evolver definition |
|
boolean / false |
Request learner-session continuation; method must support it |
|
mapping / {} |
Overrides merged with the chosen method’s defaults |
|
string / null |
Layer 2 the method runs on: |
|
string / mapping / null |
Layer 1, as |
|
mapping / null |
Layer 2’s settings, read into |
|
list / [] |
Granted environment resources; required for MWM’s evidence integration |
|
list / [] |
Task tool declarations offered to the learner |
|
boolean / false |
Consent declaration checked for methods requesting that need; not a network filter |
The template defines what is editable. Method selection does not change those rules. See adapter metadata reference for exports, learnable files, and checkers.
Models and backends#
A model is named by a reference, <backend>/<model id>; a backend is an endpoint (base URL, key variable, the
dialects it serves, prices). See Model backends.
Field |
Type / default |
Meaning and constraints |
|---|---|---|
|
string / null |
The run’s default model, a |
|
mapping / {} |
Backends declared for this run by name, over |
|
reference, mapping / null |
The run’s chat-completions |
agent.params.model takes the same forms. For a Codex or Claude Code acting agent or Evolver a reference becomes
the CLI’s provider settings (args for Codex, env for Claude Code, provider for Pi, OpenCode, DeepSeek Harness and
Hermes Agent) and the model’s id; a mini-swe-agent acting agent calls the backend through litellm with the key passed
per request. A plain id without a backend goes to the CLI’s own provider as before (for mini-swe-agent: a litellm model
id, its key from litellm’s usual variable).
Learner model tool#
llm defaults to null. When configured, it creates the run’s chat-completions tool. It does not configure the acting
model in your pipeline or automatically configure an external learner CLI. The endpoint form:
Field inside |
Type / default |
Meaning |
|---|---|---|
|
string; required when configured |
Endpoint prefix; the client appends |
|
string; required when configured |
Learner model identifier (or a |
|
string / null |
Environment-variable name containing the bearer credential |
|
mapping / null |
USD per million tokens: |
|
mapping / {} |
Fields added to requests, such as endpoint-specific reasoning settings |
Without prices, cost is unknown. With a price mapping, omitted input/output prices default to zero, and omitted
cached-input price falls back to the input price. Supply all applicable prices rather than relying on those defaults.
Per-call arguments override matching extra_body fields. The underlying client defaults to a 600-second HTTP timeout and four retries; llm.timeout (seconds) is forwarded, llm.max_retries is not.
A model only a CLI reaches, as llm or as an acting pipeline’s agent.model: each chat call is one session with no
tools, which on Claude Code or Pi can only reply (Claude subscription). Any other key is
the Base Agent’s setting (hermit/agent/base/cli.py).
Field |
Type / default |
Meaning |
|---|---|---|
|
|
The Base Agent. Pi takes a |
|
string |
The CLI’s model name; required with |
|
|
The user’s subscription on that CLI |
|
string / |
|
|
mapping / {} |
With |
|
boolean |
Required on Codex; on Claude Code false by default, and it cannot be on |
|
string / null |
The default reasoning effort; a request’s |
|
number / 3600 |
One session’s clock |
|
number / 3600 |
The chat client’s; give it more than |
|
list / [] |
Request fields accepted without being applied ( |
|
see the guide |
The login’s plan. For a workspace session they are the Base Agent’s settings ( |
Trainer and evaluation#
Field |
Type / default |
Meaning and constraints |
|---|---|---|
|
string / null |
Derived from method/template when null; explicit names use letters/digits, underscores, dots, hyphens, starting alphanumeric, at most 81 characters |
|
path / |
Parent of the run directory and default adapter-tree directory |
|
path / null |
Overrides default |
|
integer / 1 |
Positive number of training passes |
|
boolean / false |
Honor an Evolver’s |
|
string / |
Use |
|
integer / null |
Checkpoint-tag frequency by step; null disables that frequency |
|
integer / 1 |
Checkpoint-tag frequency by epoch; null disables that frequency |
|
number / null |
Dollar budget; reliable accounting requires known usage and prices |
|
number / null |
Token budget |
|
number / null |
Training wall-time budget |
|
number / 60 |
Per-call timeout in minutes |
|
integer / 1 |
Attempts per test task; use a positive value |
Use at least one finite budget for a real experiment. Limits are observed at call/rollout boundaries; concurrent
work can overshoot. Separate check probes and eval calls are outside the training budget.
Checkpoint frequency controls tags; completed steps still have records and accepted adapter history.
Sandboxes#
Field |
Type / default |
Meaning |
|---|---|---|
|
string / |
|
|
list / [] |
Task paths shown read-only to the acting agent; never the data, the judge, its tests or the runs |
|
list / [] |
Paths it may write besides its own work directory |
|
list / [] |
Python distributions shown besides the framework’s environment (an editable install’s package, one on |
|
list / [] |
Variables passed in: names or patterns ( |
|
string / |
|
|
list / [] |
With |
|
list / [] |
NVIDIA GPUs shown in the task sandbox, by the N of |
|
mapping / {} |
|
|
string / null |
A Docker image: a Codex or Claude Code acting agent runs in a container of its own |
|
mapping / {} |
With |
|
string / |
|
|
list / [] |
|
|
list / [] |
|
|
boolean / true |
False runs everything unconfined |
|
string / |
|
|
list / [] |
Modules or packages of the framework’s environment covered in every sandbox but the judge’s |
|
list / [] |
More hosts the learner reaches, through the proxy |
|
list / [] |
Variables passed to the learner (names or patterns) |
|
mapping / {} |
|
The backend is Linux/bubblewrap (Docker for environment.image). A sandbox holds its models’ keys, as Harbor’s agents
do; what the run keeps is scrubbed of them. An arbitrary in-process Python Evolver’s own code runs in the Trainer; what
it starts runs in the learner’s sandbox. Read the boundary description before
granting resources.
Complete defaults#
# Framework defaults. A run.yaml and command-line overrides (key.sub=value) change only what they name.
data:
tasks: null
train: null
val: null
test: null
step_size: 8
shuffle: true # false: every epoch takes the train split in file order (an ordered stream)
shuffle_seed: 0
task_info: null
agent:
name: null # react | codex | claude-code | pi | mini-swe-agent | opencode | dsh | hermes, or
# file.py:Class / file.py:function of the task's own
concurrency: 4
timeout_min: null # a sandboxed task call that runs longer is stopped and recorded as an error; null: none
params: {} # Layer 3: the Acting Agent's own settings (docs/framework/agent-layers.md)
base: null # Layer 2: codex | claude | pi | mini-swe-agent | opencode | dsh | hermes | react |
# direct; its settings go in a section of its name
model: null # Layer 1: a <backend>/<model> reference, or {ref | name, login, effort, price, ...}
codex: null
claude: null
pi: null
mini-swe-agent: null
opencode: null
dsh: null
hermes: null
react: null
direct: null
pipeline: # the older spelling of `agent`: run -> name
run: null
concurrency: null
params: {}
environment: # the task sandbox, where the acting agent runs (rollouts, eval, check)
runner: sandbox # sandbox: in the task sandbox | host: in the Trainer's process, for a task whose own
# code isolates what it runs (AgentDyn's hooks in containers); warned
image: null # a Docker image: a CLI acting agent runs in a container of its own (Codex or
# Claude Code)
container: {} # with image: the container's network and egress proxy (hermit/agent/base/container.py)
packages: [] # no image: Python distributions shown besides the framework's environment (an editable
# install's package, one on PYTHONPATH), with their dependencies
read: [] # no image: task paths shown read-only (relative to this file); never the data, the
# judge or the runs
write: [] # paths the acting agent may write besides its own work directory
env: [] # environment variables passed in, names or patterns (MWM_*); the agent's model key
# passes anyway; a Claude login's token passes by name to each call
network: no-network # no-network | allowlist | public; the agent's model and the tool endpoint stay reachable
allowed_hosts: [] # allowlist: hosts reachable through the allowlist proxy
resources: {} # memory_mb, file_mb, cpu_s: per process
gpu: [] # NVIDIA GPUs shown, by the N of /dev/nvidia<N> (CUDA and EGL; inside they count from 0);
# none: no GPU (EGL renders in software where Mesa is installed)
judge:
path: null
name: score
mode: host # host: on the host, from the answer | shared: in the task sandbox after the agent |
# separate: in a fresh judge sandbox holding the collected paths
tests: [] # shared, separate: paths only the judge sees, read-only (its answers, test files)
collect: [] # separate: paths of the agent's work directory carried to the judge sandbox
template: null
start_from: null
evolver:
name: null
resume: false
params: {} # Layer 3: the method's settings
base: null # Layer 2: codex | claude | pi | opencode | dsh | hermes | direct
# (the Base Agent the method runs on)
model: null # Layer 1: as agent.model; with direct it is the run's `llm`
codex: null
claude: null
pi: null
opencode: null
dsh: null
hermes: null
direct: null
evolver_access:
environment: []
tools: []
send_material_out: false
sandbox: # docs/developer/sandbox-design.md
enabled: true # false: everything the run starts runs unconfined (a warning; not for reported results)
enforce: required # required: refuse to run without bubblewrap | best_effort: run unconfined, recorded
hide: [] # modules or packages of the framework's environment covered in every sandbox but the
# judge's (a benchmark's scoring the agent does not import): `pkg.scoring`
evolver: # the learner's sandbox: a fixed layout under /w, its models, the tool endpoint
allowed_hosts: [] # more hosts the learner may reach, through the allowlist proxy
env: [] # environment variables passed to the learner's sessions (its models' keys pass anyway;
# a Claude login's token goes by name with each session on it)
resources: {} # memory_mb, file_mb, cpu_s: per process
model: null # the run's default model, a `<backend>/<model id>` reference (see backends): what `llm`,
# agent.params.model and evolver.params.model use when they name none
backends: {} # this run's model backends by name: {base_url, api_key_env, apis, prices, ...} or
# {from: <other>, base_url: ...}; over ~/.config/hermit/backends.yaml and the built-ins
llm: null # the run's `llm` tool: `<backend>/<model>`, {model: <ref>, extra_body}, or
# {base_url, model, api_key_env, price: {input, cached_input, output}, extra_body}
trainer:
run_name: null
output_dir: runs
tree_dir: null # default: <output_dir>/trees/<template id>
epochs: 1
allow_early_stop: false
resume_mode: auto
save_freq: {steps: null, epochs: 1}
budget: {usd: null, tokens: null, hours: null}
call_timeout_min: 60
eval:
repeats: 1