Recipe catalog

Pick a recipe by the task and evaluation protocol. Recipes specify actual integrations; the method guide explains algorithm selection independently.

Recipe	What you need	Methods	What it demonstrates
OfficeQA	Gated dataset, parsed source pages, SkillOpt acting dependencies, model access	GEPA, SkillOpt, Harness², Continual Harness; direct chat or Codex harness	Skill optimization for a document agent in the SkillOpt paper’s split, seed skill, and scoring
ServeLearnBench	Benchmark package, Linux learner environment, model access	MWM; prompt-only comparison configs	Ordered serving episodes and per-window held-out tests
CL-Bench	CL-Bench and MetaWorld checkouts, venue environments, Linux, model access	MWM; comparison examples in the repository	Persistent environment learning over ordered database/poker streams
Lean collaboration	Lean/Mathlib, agent hosts, team service	Existing adapter	Multi-agent consumption with shared tools and state

Before a recipe, the offline Quick Start and First real-model training run the same loop without benchmark assets.

Where the files are

Every recipe’s code, configs and trained adapters are under examples/<task>/; the layout of a task folder and how to run one are in examples/README.md. Configuration paths and eval --adapter directories resolve against the method config’s directory. Stream launchers can override output directories; use their recipe’s explicit output settings. MWM venue text lives under utils/task-info/mwm/, with a configurable deployment path.

What counts as a result

The offline tutorial has constructed scores. A smoke config checks integration on a small subset. A trained adapter’s CARD.md records its experiment and measured evaluation. These provide different kinds of evidence.

When reproducing, preserve the framework revision, benchmark revision, model identifiers, task splits, complete config, seed adapter, and scorer. Endpoint availability and model behavior can change independently of the code. Expected runtime is a planning observation, not a promise; costs depend on models, trajectories, retries, and candidate evaluation.

Reproduce a result gives a checklist for matching an existing experiment.