Commands#

Command	Purpose
`hermit check --config run.yaml`	Check the settings before training
`hermit train --config run.yaml`	Run the Trainer
`hermit eval --config run.yaml --adapter ...`	Evaluate one adapter on the test split
`hermit deploy --adapter DIR --workspace WORKSPACE --host codex`	Generate a host workspace from an adapter
`hermit convert --adapter DIR --output OUT --target claude-code`	Emit a plugin in a host’s native format
`hermit serve DIR`	Serve adapter tools over stdio, normally started by a host

The first three accept key=value overrides after the config (see Configuration).

Typical workflow#

hermit check --config run.yaml                      # optional: catch setting mistakes before spending
hermit train --config run.yaml                      # train; resumes automatically after a crash
hermit eval  --config run.yaml --adapter final      # evaluate the final selection
hermit eval  --config run.yaml --adapter none       # optional: the agent without any adapter

check#

check is a separate command, outside the run. The user runs it when they want and starts training once satisfied; train does not run it for them. The probe calls the pipeline and can incur model costs. Pass --no-probe to skip it.

What `check` looks at	Example of a problem it finds
Template	The starting adapter fails `check_spec`
Evolver	The class does not load; a need is not given; resume is requested but not supported
Task	`task-info/README.md` missing; splits overlap; the judge cannot score a sample task
Budgets	No hard budget or no iteration budget
Pipeline probe	One training task without an adapter; score, time, reported API speed, and cost are printed
Risky settings	The warnings listed in Configuration

check prints a disclaimer: it checks only these basic items. Passing check does not guarantee that the settings are free of problems or that the run will succeed.

train#

Runs the loop described in Trainer. At start it loads and validates the configured components and checks the Evolver’s declared needs (see Configuration). It does not run the full standalone check command. If a run with the same run_name is unfinished, resume_mode: auto continues it.

eval#

hermit eval --config run.yaml --adapter final         # a run's final adapter
hermit eval --config run.yaml --adapter <tag|commit|path>
hermit eval --config run.yaml --adapter root          # the template's initial content
hermit eval --config run.yaml --adapter none          # no harness adapter: a pure baseline

Item	Rule
Data	The test split only
Deployment and scoring	The same pipeline and judge as the training rollouts
Repeats	`eval.repeats`; reports the mean and the share of tasks solved at least once
Records	Per-task score, cost, time and API speed under `<run>/eval/<adapter>/`; the summary counts runs that errored
`CARD.md`	With the literal `--adapter final` selector, the results are written into the run’s final card

eval evaluates exactly what it is given and runs no baseline by itself. A baseline is one more eval, run when the user chooses: --adapter none runs the acting agent with no harness adapter at all (the pipeline receives adapter=None); --adapter root runs the template’s initial content, which may already deploy tools or context.

deploy#

hermit deploy --adapter DIR --workspace WORKSPACE --host codex
hermit deploy --adapter DIR --workspace WORKSPACE --host claude-code
hermit deploy --adapter DIR --workspace WORKSPACE --host pi

This generates host instruction files, role/skill files, and tool configuration. It does not launch the acting agent. Pi has no subagents, so --host pi refuses an adapter that exports roles; its .pi/ files (tools, skills) load once the user trusts the project in Pi, while AGENTS.md needs no trust. It is the per-project way to use an adapter; the per-agent way, convert, and how the two compare are in What is a harness adapter?. The workspace must be an empty directory or one previously generated for the same host, separate from the adapter directory. See the file mapping for exact paths and update behavior.

convert#

hermit convert --adapter DIR --output OUT --target claude-code
hermit convert --adapter DIR --output OUT --target codex
hermit convert --adapter DIR --output OUT --target pi
hermit convert --adapter DIR --output OUT --target hermes

deploy prepares a workspace you work in; convert emits a plugin in the host’s native format, installed once and available in every project. The adapter is copied into the plugin, and its MCP entry runs hermit serve on that copy, so hermit must be installed and on PATH wherever the plugin is used. Nothing in the plugin names the machine that produced it.

What each host’s plugin carries#

Adapter part	Claude Code	Codex	Pi	Hermes
Context (`ADAPTER.md`)	output style, forced while enabled	refused	refused	system prompt section, up to 4,000 characters
Skills	`skills/`	`skills/`	`skills/`	`ctx.register_skill`
Tools	`.mcp.json`	`mcp.json`	generated extension, `pi.registerMcpServer`	`ctx.register_tool`, forwarding to `hermit serve`
Roles	`agents/`	refused	refused	refused

A part the target’s plugin format cannot carry is refused: convert exits 2 and names every such part before writing anything. Since every adapter has an ADAPTER.md, a non-empty context makes Codex and Pi refuse, and a context over 4,000 characters makes Hermes refuse. --allow-lossy emits the plugin without those parts and lists them under omitted in the result.

Why the refusals:

Codex reads no instruction file and no subagent from a plugin. Its plugin format is Agent Plugins v1: root plugin.json, skills/, mcp.json. (The .codex-plugin/plugin.json layout is a compatibility fallback that does not expand ${PLUGIN_ROOT}.) deploy --host codex does register roles, in a workspace.
Pi builds its system prompt from context files in the agent and working directories, never from a package, and has no built-in subagents.
Hermes drops a plugin’s system prompt section over 4,000 characters with only a log warning, and has no file-defined subagents. It also drops a section that would take all plugins’ sections past 8,000 characters together, which depends on the user’s other plugins and is not checked here.

Install#

Target	Output	Install
Claude Code	a marketplace with one plugin	`claude plugin marketplace add OUT`, then `claude plugin install ID@ID-adapters`
Codex	a marketplace (`.agents/plugins/marketplace.json`) with one plugin	`codex plugin marketplace add OUT`, then `codex plugin add ID@ID-adapters`
Pi	a package	`pi install OUT`
Hermes	a plugin directory	copy OUT to `~/.hermes/plugins/ID`, then `hermes plugins enable ID`

The tool-call audit log goes to each host’s per-plugin data directory (${CLAUDE_PLUGIN_DATA}, ${PLUGIN_DATA}, the Hermes plugin data directory); Pi has none, so it goes to the package directory.

Checking the result#

claude mcp list and codex mcp list show whether the tool server starts. hermes plugins doctor OUT runs Hermes’ discovery, manifest parsing and register(ctx) without installing.

hermit.json next to the embedded adapter records the adapter’s id and version and the framework version that made the plugin; hermit serve warns on stderr when the installed framework is another version. Converting again into the same output replaces the generated parts, so a skill, a role or a context the adapter no longer has does not survive from the earlier plugin; the host’s audit log there is kept.

serve#

hermit serve DIR
hermit serve DIR --audit-log tool-calls.jsonl

Starts the implemented per-session MCP tool server on stdio. Generated plugin entries normally run it for the host. It is not an HTTP preview server or the proposed standalone-service launcher. --standalone and --serve-url are not supported.

Other command options#

deploy and convert accept --python to choose the Python executable used for tools; it defaults to the running interpreter. deploy --output is an alias for --workspace. convert --allow-lossy allows explicitly reported omitted exports. check --no-probe skips only the acting-task probe. Use hermit COMMAND --help for argument syntax.

Implementation notes#

check also verifies that every sandbox.read path exists. It does not check the pipeline’s host: adapters are meant to be usable by any acting agent, and each pipeline deploys the exports (context, tools, roles, skills) in its own way.
hermit deploy --adapter DIR --workspace WORKSPACE --host codex|claude-code|pi deploys an adapter’s exports into a host workspace (context block, MCP tools, native roles, skills).
hermit convert --adapter DIR --output OUT --target ... emits a plugin in the host’s native format instead, and exits 2 naming every part the target’s plugin cannot carry.