# Orch Test Documentation Roadmap

## Purpose

This roadmap tracks the human-readable Markdown test plan for `orch`.

It exists so a new agent can immediately answer four questions without re-reading the whole codebase:

- which test-plan documents already exist
- which cases have already been written down
- which cases are still missing
- what file should be updated next

This roadmap is for the Markdown test-plan set under `docs/tests/orch/`.
It is not a replacement for automated Go tests.

## Current Snapshot

Snapshot date:

- `2026-03-19`

Current state:

- `orch` CLI is implemented for the current scheduler, strict worktree, wait, and council review surfaces
- automated Go integration tests already cover the main scheduler lifecycle, dependency gating, blocked-answer flow, worktree dispatch behavior, waits, retries, reassignments, cleanup, and council start/wait/tally/report flows
- this roadmap now exists under `docs/tests/orch/ROADMAP.md`
- the initial `docs/tests/orch/` skeleton now exists with shared, workflow, and per-command index documents
- command-case Markdown files have not been authored yet; the next work is filling the backlog in a stable order

Progress summary for planned test-plan documents, excluding `ROADMAP.md`:

- planned document files: `22`
- authored document files: `22`
- planned case slugs in this roadmap: `35`
- authored case slugs in this roadmap: `0`

## Scope

In scope:

- `orch run init`
- `orch run show`
- `orch task add`
- `orch dep add`
- `orch ready`
- `orch dispatch`
- `orch reconcile`
- `orch wait`
- `orch blocked`
- `orch answer`
- `orch retry`
- `orch reassign`
- `orch cancel`
- `orch cleanup`
- `orch status`
- `orch council start`
- `orch council wait`
- `orch council tally`
- `orch council report`
- cross-command workflows
- shared test conventions for JSON output, exit codes, temp databases, repo fixtures, and worktree assertions

Out of scope:

- `inbox` command behavior except as supporting setup for `orch` scenarios
- implementation details that are not visible through the `orch` CLI contract
- future `orch` commands that are not currently implemented

## Tracking Rules

Directory model:

- one folder per leaf command or shared area
- each folder keeps a `README.md` entrypoint
- command folders use `README.md` as an index only
- each command case lives in its own Markdown file named after the case slug
- cross-command workflow cases remain grouped in `docs/tests/orch/workflows/README.md`

Case identity:

- do not use numeric IDs
- identify each command case by its concrete file path
- identify each workflow case by `path + case slug`
- command case file naming pattern:

```text
<case-slug>.md
```

- workflow case heading pattern:

```md
## case: run-dispatch-reconcile-status-happy-path
```

Per-case structure inside the case document:

- `用例意义`
- `前置条件`
- `输入`
- `预期输出`
- `断言结论`

How to update this roadmap when a new case is written:

1. if it is a command case, create or update the target `<case-slug>.md` file under the relevant leaf-command folder
2. if it is a command case, add or update the entry in that folder `README.md` index
3. if it is a workflow case, add or update the case inside `docs/tests/orch/workflows/README.md`
4. move the case slug from `Pending Case Backlog` to `Authored Case Register`
5. update the authored counts in `Current Snapshot`
6. if a new Markdown file is created, update `Document Progress`

Allowed status values in this roadmap:

- `pending`
- `in_progress`
- `done`
- `deferred`

## Existing Automated Coverage Reference

The Markdown test-plan set starts at zero, but these automated tests already exist and should be used as source material when writing the docs:

- [integration_test.go](../../../internal/cli/orch/integration_test.go#L12) `TestOrchRunDispatchReconcileLifecycle`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L184) `TestOrchDependencyBlockedAndAnswerFlow`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L458) `TestOrchDispatchRejectsNonReadyTask`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L512) `TestOrchDispatchCreatesStrictWorktree`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L631) `TestOrchStrictWorktreeRejectsDirtyRepoWithoutBaseRef`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L680) `TestOrchStrictWorktreeAllowsExplicitBaseRefOnDirtyRepo`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L733) `TestOrchDispatchAutoEnablesWorktreeForCodeLikeTask`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L782) `TestOrchDispatchDoesNotAutoEnableWorktreeForNonCodeTask`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L829) `TestOrchWaitWakesOnBlockedEvent`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L944) `TestOrchWaitTimesOutWithoutMatchingEvent`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L981) `TestOrchRetryCreatesNewAttempt`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1079) `TestOrchReassignCancelsOldThreadAndDispatchesNewAttempt`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1186) `TestOrchCancelTaskAndRun`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1315) `TestOrchCleanupRemovesCompletedWorktree`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1402) `TestOrchCouncilStartDispatchesThreeReviewers`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1545) `TestOrchCouncilWaitWakesWhenAllReviewersComplete`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1678) `TestOrchCouncilWaitTimesOutWhenReviewersIncomplete`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1715) `TestOrchCouncilTallyGroupsReviewerFindingsNormal`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1813) `TestOrchCouncilTallyStrictKeepsDistinctProposals`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1873) `TestOrchCouncilReportDefaultShowsConsensusAndMajority`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1950) `TestOrchCouncilReportShowAllIncludesMinority`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1979) `TestOrchCouncilReportJSONShape`

These tests do not remove the need for the Markdown plan. They only reduce discovery work.

## Planned Directory Tree

```text
docs/tests/orch/
  ROADMAP.md
  README.md
  _shared/
    README.md
  workflows/
    README.md
  run-init/
    README.md
  run-show/
    README.md
  task-add/
    README.md
  dep-add/
    README.md
  ready/
    README.md
  dispatch/
    README.md
  reconcile/
    README.md
  wait/
    README.md
  blocked/
    README.md
  answer/
    README.md
  retry/
    README.md
  reassign/
    README.md
  cancel/
    README.md
  cleanup/
    README.md
  status/
    README.md
  council-start/
    README.md
  council-wait/
    README.md
  council-tally/
    README.md
  council-report/
    README.md
```

## Document Progress

| Path | Purpose | Planned Cases | Authored Cases | Status |
| --- | --- | ---: | ---: | --- |
| `docs/tests/orch/README.md` | Global testing conventions and glossary | 0 | 0 | done |
| `docs/tests/orch/_shared/README.md` | Shared fixtures, JSON assertions, exit-code rules, and worktree conventions | 0 | 0 | done |
| `docs/tests/orch/workflows/README.md` | Cross-command scenarios | 4 | 0 | done |
| `docs/tests/orch/run-init/README.md` | `run init` command case index | 1 | 0 | done |
| `docs/tests/orch/run-show/README.md` | `run show` command case index | 1 | 0 | done |
| `docs/tests/orch/task-add/README.md` | `task add` command case index | 1 | 0 | done |
| `docs/tests/orch/dep-add/README.md` | `dep add` command case index | 1 | 0 | done |
| `docs/tests/orch/ready/README.md` | `ready` command case index | 1 | 0 | done |
| `docs/tests/orch/dispatch/README.md` | `dispatch` command case index | 6 | 0 | done |
| `docs/tests/orch/reconcile/README.md` | `reconcile` command case index | 2 | 0 | done |
| `docs/tests/orch/wait/README.md` | `wait` command case index | 2 | 0 | done |
| `docs/tests/orch/blocked/README.md` | `blocked` command case index | 1 | 0 | done |
| `docs/tests/orch/answer/README.md` | `answer` command case index | 1 | 0 | done |
| `docs/tests/orch/retry/README.md` | `retry` command case index | 1 | 0 | done |
| `docs/tests/orch/reassign/README.md` | `reassign` command case index | 1 | 0 | done |
| `docs/tests/orch/cancel/README.md` | `cancel` command case index | 2 | 0 | done |
| `docs/tests/orch/cleanup/README.md` | `cleanup` command case index | 1 | 0 | done |
| `docs/tests/orch/status/README.md` | `status` command case index | 1 | 0 | done |
| `docs/tests/orch/council-start/README.md` | `council start` command case index | 1 | 0 | done |
| `docs/tests/orch/council-wait/README.md` | `council wait` command case index | 2 | 0 | done |
| `docs/tests/orch/council-tally/README.md` | `council tally` command case index | 2 | 0 | done |
| `docs/tests/orch/council-report/README.md` | `council report` command case index | 3 | 0 | done |

## Authoring Order

1. global conventions in `docs/tests/orch/README.md`
2. shared fixtures and assertion helpers in `docs/tests/orch/_shared/README.md`
3. workflow cases in `docs/tests/orch/workflows/README.md`
4. core scheduler command docs: `run-init`, `task-add`, `dep-add`, `ready`, `dispatch`, `reconcile`, `status`
5. interactive leader command docs: `wait`, `blocked`, `answer`, `retry`, `reassign`, `cancel`, `cleanup`
6. council workflow docs: `council-start`, `council-wait`, `council-tally`, `council-report`

## Pending Case Backlog

### Workflows

- `docs/tests/orch/workflows/README.md :: run-dispatch-reconcile-status-happy-path` - end-to-end happy path from run creation through final `status`
- `docs/tests/orch/workflows/README.md :: dependency-blocked-answer-resume-flow` - dependency gating plus blocked question and answer flow
- `docs/tests/orch/workflows/README.md :: strict-worktree-dispatch-to-cleanup` - worktree creation through cleanup of a completed attempt
- `docs/tests/orch/workflows/README.md :: council-review-end-to-end` - council start, wait, tally, and report sequence

### Command Cases

- `docs/tests/orch/run-init/run-init-creates-new-run.md` - create a run with goal and optional summary
- `docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md` - show aggregate run metadata after activity
- `docs/tests/orch/task-add/task-add-creates-ready-root-task.md` - adding a dependency-free task becomes `ready`
- `docs/tests/orch/dep-add/dep-add-blocks-dependent-task-until-prerequisite-completes.md` - dependency edge prevents immediate readiness
- `docs/tests/orch/ready/ready-lists-only-eligible-tasks.md` - ready list excludes blocked by dependency tasks
- `docs/tests/orch/dispatch/dispatch-creates-attempt-and-thread-for-ready-task.md` - happy-path dispatch with new attempt metadata
- `docs/tests/orch/dispatch/dispatch-rejects-non-ready-task.md` - invalid-state rejection when dispatching a gated task
- `docs/tests/orch/dispatch/dispatch-creates-strict-worktree.md` - explicit strict worktree dispatch persists workspace metadata
- `docs/tests/orch/dispatch/dispatch-rejects-dirty-repo-without-base-ref.md` - strict worktree guard on dirty repository state
- `docs/tests/orch/dispatch/dispatch-allows-explicit-base-ref-on-dirty-repo.md` - explicit base ref bypasses dirty working tree rejection
- `docs/tests/orch/dispatch/dispatch-auto-enables-worktree-for-code-like-task.md` - inferred worktree mode for code-like tasks
- `docs/tests/orch/dispatch/dispatch-skips-auto-worktree-for-non-code-task.md` - non-code tasks stay on the normal dispatch path
- `docs/tests/orch/reconcile/reconcile-maps-claimed-or-in-progress-thread-to-running.md` - reconcile turns claimed/in-progress inbox state into running task state
- `docs/tests/orch/reconcile/reconcile-maps-done-or-failed-thread-to-terminal-task-state.md` - reconcile turns terminal inbox state into task terminal state
- `docs/tests/orch/wait/wait-wakes-on-matching-run-event.md` - wait wakes on a later matching task event
- `docs/tests/orch/wait/wait-times-out-without-matching-event.md` - wait returns a non-woken timeout result
- `docs/tests/orch/blocked/blocked-lists-latest-question-for-blocked-task.md` - blocked list includes the active question payload
- `docs/tests/orch/answer/answer-appends-answer-to-active-thread.md` - answer writes an inbox answer message for the blocked attempt
- `docs/tests/orch/retry/retry-creates-new-attempt-for-failed-task.md` - retry creates a successor attempt after failure
- `docs/tests/orch/reassign/reassign-cancels-old-thread-and-dispatches-new-attempt.md` - reassign moves work to a new worker and attempt
- `docs/tests/orch/cancel/cancel-cancels-single-task.md` - cancel transitions one task to `cancelled`
- `docs/tests/orch/cancel/cancel-cancels-entire-run.md` - cancel cascades terminal state across the run
- `docs/tests/orch/cleanup/cleanup-removes-completed-worktree.md` - cleanup removes a completed attempt worktree
- `docs/tests/orch/status/status-returns-run-summary-and-task-list.md` - status returns run aggregate state and per-task statuses
- `docs/tests/orch/council-start/council-start-dispatches-three-reviewers.md` - council start creates reviewer tasks and metadata
- `docs/tests/orch/council-wait/council-wait-wakes-when-all-reviewers-complete.md` - council wait completes successfully when all reviewers finish
- `docs/tests/orch/council-wait/council-wait-times-out-when-reviewers-incomplete.md` - council wait timeout behavior
- `docs/tests/orch/council-tally/council-tally-groups-reviewer-findings-in-normal-mode.md` - normal similarity grouping behavior
- `docs/tests/orch/council-tally/council-tally-keeps-distinct-proposals-in-strict-mode.md` - strict similarity grouping behavior
- `docs/tests/orch/council-report/council-report-defaults-to-consensus-and-majority.md` - default report buckets and markdown artifact behavior
- `docs/tests/orch/council-report/council-report-show-all-includes-minority.md` - `--show all` includes minority output
- `docs/tests/orch/council-report/council-report-json-shape-is-stable.md` - JSON output shape and artifact metadata contract

## Authored Case Register

- none yet