OPRO

OPRO, Optimization by PROmpting (Yang et al., ICLR 2024, code), uses a model as the optimizer: it reads earlier instructions with their training scores and a few training problems, and writes new instructions; every instruction it writes is scored on the whole training set and joins the history. Choose it when the adapter is one instruction document, the training split is small enough to run once per candidate, and you want the search to see the best instructions so far with their scores.

Setup

template: prompt-only@1.0
evolver:
  name: opro
  params: {n_candidates: 4, select: validation, top_k: 3}
data:
  step_size: 50          # the training split's size: one OPRO step per framework step
  val: splits/val.txt    # for select: validation
trainer: {epochs: 3}     # OPRO steps
llm: {model: <backend>/<model>}   # the optimizer

The instruction is ADAPTER.md. The optimizer is the run’s model (llm); on a model reachable only through a ChatGPT login, OfficeQA’s GPT-5.6-Luna setting makes each call one Codex session (llm: {cli: codex}). The judge’s scores must be in [0, 1]: the meta-prompt shows them from 0 to 100, and a step whose rollouts score outside that range fails.

One framework step

The Trainer’s rollouts run the current instruction on every training task. Then (run_evolution, one i_step):

1. at the first step the rollouts are the initial instruction’s score (step −1 in the history); later they re-score the current best and are recorded, not merged;

2. num_exemplars training problems are drawn at random for this step, from the runs that did not fail;

3. the meta-prompt: the best max_num_instructions instructions above score_threshold, in ascending order with scores 0–100, then the problems, then OPRO’s request for a better instruction between <INS> and </INS>;

4. n_candidates optimizer calls on that one meta-prompt; each reply’s instruction is extracted, polished and compared by md5 with every instruction generated before (as upstream, not with the initial one);

5. each new instruction passes the template check, runs on every training task (run_agent), and joins the history with its mean score. A wrong answer scores 0; a run that fails for an infrastructure reason (a container or proxy error) is run again, and an instruction whose runs fail twice is left unscored rather than scored 0;

6. the instruction with the best training score so far becomes the adapter.

        flowchart TD
  R[the step's rollouts: the current instruction<br/>on every training task] --> H[history: every instruction<br/>with its training score]
  H --> M[meta-prompt: the best old instructions, ascending,<br/>then num_exemplars training problems]
  M --> L[n_candidates calls to the run's llm:<br/>one instruction between INS tags each]
  L --> D{new, non-empty,<br/>passes the template check?}
  D -- no --> S[skipped, recorded]
  D -- yes --> A[run_agent on every training task:<br/>its mean joins the history]
  A --> B[the best on training becomes the adapter]
  B --> F[finalize: the best on training, or the best<br/>validation mean of the top_k on training]
    

finalize hands back the best on training (select: train, as the paper reports), or, with select: validation, the best validation mean among the top_k best on training. Validation tasks never reach the meta-prompt. state/opro.json holds the history, each candidate’s per-task scores and one record per step; each step’s output/ holds the meta-prompt, the replies and report.md.

Parameters

description: OPRO, Optimization by PROmpting (Yang et al., ICLR 2024, arXiv:2309.03409; google-deepmind/opro) on the
  `prompt-only` template, one OPRO step per training step. The run's `llm` tool reads OPRO's meta-prompt (the old
  instructions with their training scores, ascending, and random training problems) and writes n_candidates new
  instructions; each is scored with run_agent on every training task and joins the history. Set data.step_size to the
  training split's size. finalize keeps the best on training, or the best on validation of the top_k on training.
class: evolver.py:OPROEvolver
needs: []
params:
  n_candidates: 8                 # new instructions per step, one optimizer call each (upstream: 8)
  max_num_instructions: 20        # old instructions shown, the best ones (upstream: 20)
  score_threshold: 0.0            # old instructions scoring below this are left out (upstream: 0.0 for text-bison
                                  # scorers, 0.3 for GPT scorers)
  num_score_buckets: 100          # scores shown as integers 0-100 (upstream: 100); null: rounded to 3 decimals
  num_exemplars: 3                # random training problems per step (upstream: 3)
  question_field: null            # the task field shown as the problem; null: the task's fields
  success_score: 1.0              # a judge score at or above this shows the agent's answer as the exemplar's answer
  max_instruction_chars: 0        # 0: no limit (upstream skips instructions over 500 characters)
  select: validation              # validation: the best validation mean of the top_k on training; train: OPRO's
  top_k: 3
  max_tokens: null                # null: the endpoint's default (upstream: 512 for GPT optimizers)
  temperature: 1.0                # upstream: 1.0
  seed: 0

The authors’ implementation against this one

The authors’ repository has no build configuration, so meta_prompt.py ports the meta-prompt, the reply parsing and the de-duplication function by function (with a NOTICE), and evolver.py follows run_evolution. The meta-prompt is the one of the authors’ GPT optimizers with the instruction at the beginning of the question and no “Q:”/”A:”, the setting of their README’s command; a test checks it against a stored output of their function.

Dimension	google-deepmind/opro	This Evolver	Kind
When learning happens	Each step: 8 new instructions, each scored on the whole training subset	The same; one OPRO step per framework step, the step’s rollouts on every training task (the first step refuses a smaller step_size). After the first step, those rollouts re-score the current best instruction: one more training pass per step, recorded, not merged into its score	Framework: the Trainer rolls out before each step
What the optimizer sees	Up to 20 instructions with scores, ascending; 3 random training problems with their gold answers, drawn with np.random.seed(i_step) from the whole training set	The same; the answer shown is the agent’s answer when judged right, else the judge’s feedback (OfficeQA’s names the expected answer; a run with no answer shows its fail reason instead). The problems are drawn with Random(seed × 100003 + i_step) from the step’s runs that did not fail	Task: the task has no gold field
What the acting agent sees	The instruction before the question in a one-shot prompt	ADAPTER.md as the agent’s instructions, then the task	Framework
The instruction	One sentence, skipped past 500 characters; polish_sentence on each	One ADAPTER.md (OfficeQA’s seed: 886 characters); max_instruction_chars off by default. polish_sentence is the authors’ and runs on the whole document, so it removes Markdown’s ** bold markers	Decision: the row’s starting skill
Score threshold	0.0 for text-bison scorers, 0.3 for GPT scorers	score_threshold: 0.0	Decision: the scorer is an agent
Final selection	None in the code (it scores instructions on the eval split every 3 steps); the paper reports the best training instructions	select: train, or the best validation mean of the top top_k on training	Decision
Failures	Optimizer and scorer API errors retried until they succeed; an empty instruction is scored	A failed optimizer call tried once more, then dropped (a step can make fewer than n_candidates); a failed run tried once more, then the instruction is left unscored (the initial instruction’s failed rollouts are run again); an empty instruction or one failing the template check is skipped and not generated again; at the hard budget, the instructions scored so far are kept, and select: validation falls back to the best on training. A candidate’s runs that were run again are only in tools.jsonl; the initial instruction’s are in the step’s record. A run without llm or with an unknown select stops in setup, before its first rollouts	Framework
Models	GPT-3.5/GPT-4 (512 tokens) or PaLM 2 (1024) optimizer at temperature 1.0; text-bison or GPT scorer at temperature 0	The run’s llm as the optimizer (temperature 1.0 when the endpoint applies it; a Codex session does not); the task’s acting agent as the scorer	Framework
Budget	200 steps; GSM8K’s 261 training questions	Config; OfficeQA: 3 steps of 4, 50 training questions	Decision: cost
Not ported	The PaLM meta-prompt, instructions_only, other exemplar criteria, the temperature schedule, few-shot evaluation of new instructions, the periodic eval-split scores, the GSM8K digit filter	—	Off by default or task-specific

What it is not

The scorer is an acting agent, not a one-shot model call: an instruction is scored by running the agent on every training task, so one OPRO step costs n_candidates × |train| agent sessions (and the Trainer’s pass of the current best, |train| more) and a single noisy pass per instruction decides its place in the history. The exemplars’ answers come from the step’s rollouts and the judge’s feedback, since the task has no gold field. Selection on validation is a reproduction decision; OPRO itself has none.