FAQ

How Is a Harness Adapter Different from a Plugin?

A plugin packages capabilities for an agent to use. A harness adapter adds a training contract: what the learner may change, what stays fixed, and how candidate updates are checked. Hermit uses that contract to train and version the adapter from task feedback.

The two can overlap: a compatible harness adapter can be distributed as a native plugin. The training contract belongs to the adapter and framework; installing the plugin does not start training in the host. See the adapter concept guide.

What Can a Harness Adapter Contain?

A harness adapter can contain instructions, skills, memory, tools, executable code, and agent coordination logic. It can be an entire repository implementing an external harness.

Its specification defines how the agent uses it, which parts can be trained, and how updates are checked—while keeping the original model and harness unchanged.

Why Not Evolve the Original Harness Directly?

Many harnesses are closed-source. Even open-source harnesses have carefully designed component interactions that direct edits can disrupt. Harness adapters keep learned changes in an external layer while leaving the original harness and model fixed.

This requires an extension point through which the agent can consume the adapter. Changes to inaccessible host internals cannot automatically be expressed as an adapter. See using and training adapters.

When Should I Use Hermit?

There are three starting points:

• Use a trained adapter: deploy it to a compatible agent without running training.

• Adapt your agent to your tasks: bring an acting agent, tasks, and a way to score outputs; start with an existing adapter template and learning method.

• Develop an evolution method: implement an Evolver and reuse the training loop, task pipeline, checks, and run records.

Start with the offline Quick Start to try the complete loop without model access.

What Changes During Training?

The Evolver revises files declared learnable in the adapter’s specification. Depending on the template, these can include instructions, skills, roles, memory, or tool implementations. The Trainer checks the result against the specification and records the accepted adapter version.

The acting agent’s model weights and original harness stay fixed. The acting agent uses the adapter to attempt tasks; the Evolver is the learner that updates it. Codex or Claude Code can serve as a learner, while other Evolvers implement their own optimization logic. See adapter specifications and the Evolver interface.

Do I Have to Train an Adapter Before Using It?

No. You can use a manually written adapter or an existing trained one. hermit deploy generates a supported host workspace; hermit convert creates a native plugin package. Neither command launches training or starts the acting agent.

Follow Deploy an adapter for the commands and required dependencies.

Can I Reuse an Adapter Across Models and Harnesses?

Yes, when the receiving agent supports the exports the adapter needs. Hermit currently generates workspaces for Codex, Claude Code, and Pi, and converts native plugin packages for Claude Code, Codex, Pi, and Hermes. Each route has different capabilities; a target may refuse unsupported exports.

Compatibility does not guarantee the same performance on a different model, harness, or task. Evaluate the adapter in its new setting. See the host capability matrix and compatibility limits.

Which Harness Evolution Methods Can I Use?

Hermit includes GEPA, SkillOpt, MetaWorld Model (MWM), Harness², and Continual Harness, along with Codex/Claude coding-agent learners and a prompt-only baseline. They share the Trainer interface, but have different data requirements and compatible adapter templates.

Use the method selection guide to choose a method and the recipe catalog for runnable benchmark setups. Methods labeled Coming Soon in the README are planned integrations.

How Do I Know Whether Training Helped?

Compare the trained adapter against both the starting adapter and no adapter on held-out tasks, using the same acting model, harness, and scoring protocol. Include learning and evaluation costs when comparing methods.

The Trainer’s acceptance checks establish structural validity; they do not guarantee a score improvement. Candidate selection depends on the Evolver and experiment protocol. See evaluation and run records for scores, changes, and recorded usage.