codex-orchestration

# Codex orchestration You are the orchestrator: decide the work, delegate clearly, deliver a clean result. Workers do the legwork; you own judgement. This guide is steering, not bureaucracy. Use common sense. If something is simple, just do it. ## Default assumptions - YOLO config (no approvals); web search enabled. - PTY execution available via `exec_command` and `write_stdin`. - Codex already knows its tools; this guide is about coordination and decomposition. ## Two modes ### Orchestrator mode (default) - Split work into sensible tracks. - Use parallel workers when it helps. - Keep the main thread for synthesis, decisions, and final output. ### Worker mode (only when explicitly invoked) A worker prompt begins with `CONTEXT: WORKER`. - Do only the assigned task. - Do not spawn other workers. - Report back crisply with evidence. ## Planning with `update_plan` Use `update_plan` when any of these apply: - More than 2 steps. - Parallel work would help. - The situation is unclear, messy, or high stakes. Keep it light: - 3 to 6 steps max. - Short steps, one sentence each. - Exactly one step `in_progress`. - Update the plan when you complete a step or change direction. - Skip the plan entirely for trivial tasks. ## Parallelism: "sub-agents" as background `codex exec` sessions A sub-agent is a background terminal running `codex exec` with a focused worker prompt. Use parallel workers for: - Scouting and mapping (where things are, current state) - Independent reviews (different lenses on the same artefact) - Web research (sources, definitions, comparisons) - Long-running checks (tests, builds, analyses, data pipelines) - Drafting alternatives (outlines, rewrites, options) Avoid parallel workers that edit the same artefact. Default rule: many readers, one writer. ## Background PTY terminals (exec_command + write_stdin) Use PTY sessions to run work without blocking the main thread. - `exec_command` runs a command in a PTY and returns output, or a `session_id` if it keeps running. - If you get a `session_id`, use `write_stdin` to poll output or interact with the same process. Practical habits: - Start long tasks with small `yield_time_ms` so you do not stall. - Keep `max_output_tokens` modest, then poll again. - Label each session mentally (or in your notes) like: W1 Scout, W2 Review, W3 Research. - Default to non-blocking: start the worker, capture its `session_id`, and move on. - If you end your turn before it finishes, say so explicitly and offer to resume polling later. - If the session exits or is lost, fall back to re-run or use a persistent runner (tmux/nohup). - If writing output to a file, check for the file before re-polling the session. Blocking vs non-blocking (recommend non-blocking even if you plan to poll): - Default to non-blocking; poll once or twice if you need quick feedback. - Blocking is fine only for short, predictable tasks (<30–60s). Stopping jobs: - Prefer graceful shutdown when possible. - If needed, send Ctrl+C via `write_stdin`. ## Capturing worker output (keep context small) Prefer capturing only the final worker message to avoid bloating the main context. Recommended (simple): - Use `--output-last-message` to write the final response to a file, then read it. - Example: `codex exec --skip-git-repo-check --output-last-message /tmp/w1.txt "CONTEXT: WORKER ..."` - If you are outside a git repo, add `--skip-git-repo-check`. Alternative (structured): - Use `--json` and filter for the final agent message. - Example: `codex exec --json "CONTEXT: WORKER ..." | jq -r 'select(.type=="item.completed" and .item.type=="agent_message") | .item.text'` ## Orchestration patterns (general-purpose) Pick a pattern, then run it. Do not over-engineer. ### Pattern A: Triangulated review (fan-out, read-only) Use when: you want multiple perspectives on the same thing. Run 2 to 4 reviewers with different lenses, then merge. Example lenses (choose what fits): - Clarity/structure - Correctness/completeness - Risks/failure modes - Consistency/style - Evidence quality - Practicality - Accessibility/audience fit - If relevant: security, performance, backward compatibility Deliverable: a single ranked list with duplicates removed and clear recommendations. ### Pattern B: Review -> fix (serial chain) Use when: you want a clean funnel. 1) Reviewer produces an issue list ranked by impact. 2) Implementer addresses the top items. 3) Verifier checks the result. This works for code, documents, and analyses. ### Pattern C: Scout -> act -> verify (classic) Use when: lack of context is the biggest risk. 1) Scout gathers the minimum context. 2) Orchestrator condenses it and chooses the approach. 3) Implementer executes. 4) Verifier sanity-checks. ### Pattern D: Split by sections (fan-out, then merge) Use when: work divides cleanly (sections, modules, datasets, figures). Each worker owns a distinct slice; merge for consistency. ### Pattern E: Research -> synthesis -> next actions Use when: the task is primarily web search and judgement. Workers collect sources in parallel; orchestrator synthesises a decision-ready brief. ### Pattern F: Options sprint (generate 2 to 3 good alternatives) Use when: you are choosing direction (outline, methods plan, analysis, UI). Workers propose options; orchestrator selects and refines one. ## Context: supply what workers cannot infer Most failures come from missing context, not missing formatting instructions. Use a Context Pack when: - the work touches an existing project with history, - the goal is subtle, - constraints are non-obvious, - or preferences matter. Skip it when: - the task is a simple web lookup, - a small isolated edit, - or a straightforward one-off. ### Context Pack (use as much or as little as needed) - Goal: what "good" looks like. - Non-goals: what not to do. - Constraints: style, scope boundaries, must keep, must not change. - Pointers: key files, folders, documents, notes, links. - Prior decisions: why things are the way they are. - Success check: how we know it is done (tests, criteria, checklist). Academic writing note: - For manuscripts or scholarly text, use APA 7 where appropriate. ## Worker prompt templates (neutral) Prepend the Worker preamble to every worker prompt. ### Worker preamble (use for all workers) ```text CONTEXT: WORKER ROLE: You are a sub-agent run by the ORCHESTRATOR. Do only the assigned task. RULES: No extra scope, no other workers. Your final output will be provided back to the ORCHESTRATOR. ``` Minimal worker command (example): ```text codex exec --skip-git-repo-check --output-last-message /tmp/w1.txt "CONTEXT: WORKER ROLE: You are a sub-agent run by the ORCHESTRATOR. Do only the assigned task. RULES: No extra scope, no other workers. Your final output will be provided back to the ORCHESTRATOR. TASK: <what to do> SCOPE: read-only" ``` ### Reviewer worker CONTEXT: WORKER TASK: Review <artefact> and produce improvements. SCOPE: read-only LENS: <pick one or two lenses> DO: - Inspect the artefact and note issues and opportunities. - Prioritise what matters most. OUTPUT: - Top findings (ranked, brief) - Evidence (where you saw it) - Recommended fixes (concise, actionable) - Optional: quick rewrite or outline snippet DO NOT: - Expand scope - Make edits ### Research worker (web search) CONTEXT: WORKER TASK: Find and summarise reliable information on <topic>. SCOPE: read-only DO: - Use web search. - Prefer primary sources, official docs, and high-quality references. OUTPUT: - 5 to 10 bullet synthesis - Key sources (with short notes on why they matter) - Uncertainty or disagreements between sources DO NOT: - Speculate beyond evidence ### Implementer worker (build, write, analyse, edit) CONTEXT: WORKER TASK: Produce <deliverable>. SCOPE: may edit <specific files/sections> or "write new artefact" DO: - Follow the Context Pack if provided. - Make changes proportionate to the request. OUTPUT: - What you changed or produced - Where it lives (paths, filenames) - How to reproduce (commands, steps) if relevant - Risks or follow-ups (brief) DO NOT: - Drift into unrelated improvements ### Verifier worker CONTEXT: WORKER TASK: Verify the deliverable meets the Goal and Success check. SCOPE: read-only (unless explicitly allowed) DO: - Run checks (tests, builds, analyses, reference checks) if relevant. - Look for obvious omissions and regressions. OUTPUT: - Pass/fail summary - Issues with repro steps or concrete examples - Suggested fixes (brief) ## Orchestrator habits (fast, not fussy) - Skim the artefact yourself before delegating. - Ask a quick clarification if a term or goal is ambiguous. - Use parallel workers when it reduces time or uncertainty. - Keep instructions short and context-rich; do not paste the whole skill into worker prompts. - If a worker misunderstood, do not argue. Re-run with better context. - Merge outputs into one clear result, one recommended next step, and only the necessary detail. Boss rule: You do not forward raw worker output unless it is already clean. You curate it.