Extend and contribute#
Start with the extension boundary you need. A task integration normally requires no Trainer change.
Change |
Entry point |
Verify |
|---|---|---|
New task, acting agent, or judge |
No-adapter probe, adapter consumption, split isolation, logs, scoring |
|
New instruction/skill/tool structure |
Learnable/fixed rules, checker, exported schemas |
|
New learning method |
Failure handling, state persistence, usage, final selection |
|
Reproduction of a published method |
The adapter and the Evolver named, the differences from the authors’ code, the protocol, the first evaluation on the shared settings |
|
New host exporter |
|
Supported exports, refusal/omission behavior, repeat generation |
New CLI Base Agent |
A |
Every hook, limits it cannot keep refused, its stream read (reply, steps, usage, served) from a recorded stream |
Trainer behavior |
Trainer contract, |
Rollback, resume boundaries, records, budget observations |
Repository map#
Location |
Responsibility |
|---|---|
|
Command parsing and config resolution |
|
The agent layers: |
|
The Acting Agent interface and the built-in acting agents |
|
The Evolver interface and the agent Evolver; |
|
Task records, callable loading, pipeline/judge types |
|
Manifest loading and acting consumption |
|
Built-in adapter seeds and specifications |
|
Training loop, rollouts, material, budgets, recovery |
|
Tool serving, sandboxing, and runtime support |
|
Task integrations and benchmark-owned acting/scoring logic |
|
Regression tests: |
|
Small offline examples included in documentation |
Local development#
python -m pip install -e '.[test]'
python -m pytest --changed
--changed runs the test groups the branch touches
(tests/README.md): the core group for every
change but documentation, a method’s group for its own files, every group for a shared layer.
Optimizer tests need the method’s dependency group (for example python -m pip install -e . --group gepa) and skip
without it; sandbox and MWM integration
tests can require Linux, learner CLIs, external packages, or environment fixtures. Inspect their prerequisites
rather than interpreting every skipped test as coverage.
For a new Evolver, put its tests under tests/methods/<name>/ and add its paths to METHODS in
tests/selection.py (tests/core/test_selection.py fails until every method has a group); use an offline task or
fake client to test real failure and state behavior without paid model calls.
A new or changed test follows the
test checklist.
Documentation changes#
python -m pip install -r docs/requirements.txt
bash docs/build.sh
bash docs/serve.sh 8000
The build treats warnings as errors. Link new pages from docs/index.md, use repository-relative
literalinclude for code and defaults, and execute walkthrough commands in a fresh run directory.
Every recipe should state its prerequisites, data preparation, model setup, launch command, output locations, evaluation protocol, reproduction limits, and available result cards. Keep observed results distinct from smoke tests and deterministic demonstrations.
For changes to interactive figures, inspect desktop and narrow layouts plus keyboard operation. The existing component contracts are described in Adapter, Evolver, and Trainer.
Proposing a change#
Describe the user-visible behavior, the affected contract, how you verified it, and any environment requirements. Include a small reproducer for failures. Update commands, defaults, and docs together; do not describe a proposed feature as available. The generic service lifecycle proposal is kept separately from supported usage. The sandbox design records how the task, judge and evolver sandboxes are built.