ai-workflow-skill/docs/tests/orch/ROADMAP.md

Orch Test Documentation Roadmap

Purpose

This roadmap tracks the human-readable Markdown test plan for orch.

It exists so a new agent can immediately answer four questions without re-reading the whole codebase:

• which test-plan documents already exist
• which cases have already been written down
• which cases are still missing
• what file should be updated next

This roadmap is for the Markdown test-plan set under docs/tests/orch/. It is not a replacement for automated Go tests.

Current Snapshot

Snapshot date:

• 2026-03-19

Current state:

• orch CLI is implemented for the current scheduler, strict worktree, wait, and council review surfaces
• automated Go integration tests already cover the main scheduler lifecycle, dependency gating, blocked-answer flow, worktree dispatch behavior, waits, retries, reassignments, cleanup, and council start/wait/tally/report flows
• this roadmap now exists under docs/tests/orch/ROADMAP.md
• the initial docs/tests/orch/ skeleton now exists with shared, workflow, and per-command index documents
• command-case Markdown files have not been authored yet; the next work is filling the backlog in a stable order

Progress summary for planned test-plan documents, excluding ROADMAP.md:

• planned document files: 22
• authored document files: 22
• planned case slugs in this roadmap: 35
• authored case slugs in this roadmap: 0

Scope

In scope:

• orch run init
• orch run show
• orch task add
• orch dep add
• orch ready
• orch dispatch
• orch reconcile
• orch wait
• orch blocked
• orch answer
• orch retry
• orch reassign
• orch cancel
• orch cleanup
• orch status
• orch council start
• orch council wait
• orch council tally
• orch council report
• cross-command workflows
• shared test conventions for JSON output, exit codes, temp databases, repo fixtures, and worktree assertions

Out of scope:

• inbox command behavior except as supporting setup for orch scenarios
• implementation details that are not visible through the orch CLI contract
• future orch commands that are not currently implemented

Tracking Rules

Directory model:

• one folder per leaf command or shared area
• each folder keeps a README.md entrypoint
• command folders use README.md as an index only
• each command case lives in its own Markdown file named after the case slug
• cross-command workflow cases remain grouped in docs/tests/orch/workflows/README.md

Case identity:

• do not use numeric IDs
• identify each command case by its concrete file path
• identify each workflow case by path + case slug
• command case file naming pattern:

<case-slug>.md

• workflow case heading pattern:

## case: run-dispatch-reconcile-status-happy-path

Per-case structure inside the case document:

• 用例意义
• 前置条件
• 输入
• 预期输出
• 断言结论

How to update this roadmap when a new case is written:

1. if it is a command case, create or update the target <case-slug>.md file under the relevant leaf-command folder
2. if it is a command case, add or update the entry in that folder README.md index
3. if it is a workflow case, add or update the case inside docs/tests/orch/workflows/README.md
4. move the case slug from Pending Case Backlog to Authored Case Register
5. update the authored counts in Current Snapshot
6. if a new Markdown file is created, update Document Progress

Allowed status values in this roadmap:

• pending
• in_progress
• done
• deferred

Existing Automated Coverage Reference

The Markdown test-plan set starts at zero, but these automated tests already exist and should be used as source material when writing the docs:

• integration_test.go TestOrchRunDispatchReconcileLifecycle
• integration_test.go TestOrchDependencyBlockedAndAnswerFlow
• integration_test.go TestOrchDispatchRejectsNonReadyTask
• integration_test.go TestOrchDispatchCreatesStrictWorktree
• integration_test.go TestOrchStrictWorktreeRejectsDirtyRepoWithoutBaseRef
• integration_test.go TestOrchStrictWorktreeAllowsExplicitBaseRefOnDirtyRepo
• integration_test.go TestOrchDispatchAutoEnablesWorktreeForCodeLikeTask
• integration_test.go TestOrchDispatchDoesNotAutoEnableWorktreeForNonCodeTask
• integration_test.go TestOrchWaitWakesOnBlockedEvent
• integration_test.go TestOrchWaitTimesOutWithoutMatchingEvent
• integration_test.go TestOrchRetryCreatesNewAttempt
• integration_test.go TestOrchReassignCancelsOldThreadAndDispatchesNewAttempt
• integration_test.go TestOrchCancelTaskAndRun
• integration_test.go TestOrchCleanupRemovesCompletedWorktree
• integration_test.go TestOrchCouncilStartDispatchesThreeReviewers
• integration_test.go TestOrchCouncilWaitWakesWhenAllReviewersComplete
• integration_test.go TestOrchCouncilWaitTimesOutWhenReviewersIncomplete
• integration_test.go TestOrchCouncilTallyGroupsReviewerFindingsNormal
• integration_test.go TestOrchCouncilTallyStrictKeepsDistinctProposals
• integration_test.go TestOrchCouncilReportDefaultShowsConsensusAndMajority
• integration_test.go TestOrchCouncilReportShowAllIncludesMinority
• integration_test.go TestOrchCouncilReportJSONShape

These tests do not remove the need for the Markdown plan. They only reduce discovery work.

Planned Directory Tree

docs/tests/orch/
  ROADMAP.md
  README.md
  _shared/
    README.md
  workflows/
    README.md
  run-init/
    README.md
  run-show/
    README.md
  task-add/
    README.md
  dep-add/
    README.md
  ready/
    README.md
  dispatch/
    README.md
  reconcile/
    README.md
  wait/
    README.md
  blocked/
    README.md
  answer/
    README.md
  retry/
    README.md
  reassign/
    README.md
  cancel/
    README.md
  cleanup/
    README.md
  status/
    README.md
  council-start/
    README.md
  council-wait/
    README.md
  council-tally/
    README.md
  council-report/
    README.md

Document Progress

Path	Purpose	Planned Cases	Authored Cases	Status
docs/tests/orch/README.md	Global testing conventions and glossary	0	0	done
docs/tests/orch/_shared/README.md	Shared fixtures, JSON assertions, exit-code rules, and worktree conventions	0	0	done
docs/tests/orch/workflows/README.md	Cross-command scenarios	4	0	done
docs/tests/orch/run-init/README.md	run init command case index	1	0	done
docs/tests/orch/run-show/README.md	run show command case index	1	0	done
docs/tests/orch/task-add/README.md	task add command case index	1	0	done
docs/tests/orch/dep-add/README.md	dep add command case index	1	0	done
docs/tests/orch/ready/README.md	ready command case index	1	0	done
docs/tests/orch/dispatch/README.md	dispatch command case index	6	0	done
docs/tests/orch/reconcile/README.md	reconcile command case index	2	0	done
docs/tests/orch/wait/README.md	wait command case index	2	0	done
docs/tests/orch/blocked/README.md	blocked command case index	1	0	done
docs/tests/orch/answer/README.md	answer command case index	1	0	done
docs/tests/orch/retry/README.md	retry command case index	1	0	done
docs/tests/orch/reassign/README.md	reassign command case index	1	0	done
docs/tests/orch/cancel/README.md	cancel command case index	2	0	done
docs/tests/orch/cleanup/README.md	cleanup command case index	1	0	done
docs/tests/orch/status/README.md	status command case index	1	0	done
docs/tests/orch/council-start/README.md	council start command case index	1	0	done
docs/tests/orch/council-wait/README.md	council wait command case index	2	0	done
docs/tests/orch/council-tally/README.md	council tally command case index	2	0	done
docs/tests/orch/council-report/README.md	council report command case index	3	0	done

Authoring Order

1. global conventions in docs/tests/orch/README.md
2. shared fixtures and assertion helpers in docs/tests/orch/_shared/README.md
3. workflow cases in docs/tests/orch/workflows/README.md
4. core scheduler command docs: run-init, task-add, dep-add, ready, dispatch, reconcile, status
5. interactive leader command docs: wait, blocked, answer, retry, reassign, cancel, cleanup
6. council workflow docs: council-start, council-wait, council-tally, council-report

Pending Case Backlog

Workflows

• docs/tests/orch/workflows/README.md :: run-dispatch-reconcile-status-happy-path - end-to-end happy path from run creation through final status
• docs/tests/orch/workflows/README.md :: dependency-blocked-answer-resume-flow - dependency gating plus blocked question and answer flow
• docs/tests/orch/workflows/README.md :: strict-worktree-dispatch-to-cleanup - worktree creation through cleanup of a completed attempt
• docs/tests/orch/workflows/README.md :: council-review-end-to-end - council start, wait, tally, and report sequence

Command Cases

• docs/tests/orch/run-init/run-init-creates-new-run.md - create a run with goal and optional summary
• docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md - show aggregate run metadata after activity
• docs/tests/orch/task-add/task-add-creates-ready-root-task.md - adding a dependency-free task becomes ready
• docs/tests/orch/dep-add/dep-add-blocks-dependent-task-until-prerequisite-completes.md - dependency edge prevents immediate readiness
• docs/tests/orch/ready/ready-lists-only-eligible-tasks.md - ready list excludes blocked by dependency tasks
• docs/tests/orch/dispatch/dispatch-creates-attempt-and-thread-for-ready-task.md - happy-path dispatch with new attempt metadata
• docs/tests/orch/dispatch/dispatch-rejects-non-ready-task.md - invalid-state rejection when dispatching a gated task
• docs/tests/orch/dispatch/dispatch-creates-strict-worktree.md - explicit strict worktree dispatch persists workspace metadata
• docs/tests/orch/dispatch/dispatch-rejects-dirty-repo-without-base-ref.md - strict worktree guard on dirty repository state
• docs/tests/orch/dispatch/dispatch-allows-explicit-base-ref-on-dirty-repo.md - explicit base ref bypasses dirty working tree rejection
• docs/tests/orch/dispatch/dispatch-auto-enables-worktree-for-code-like-task.md - inferred worktree mode for code-like tasks
• docs/tests/orch/dispatch/dispatch-skips-auto-worktree-for-non-code-task.md - non-code tasks stay on the normal dispatch path
• docs/tests/orch/reconcile/reconcile-maps-claimed-or-in-progress-thread-to-running.md - reconcile turns claimed/in-progress inbox state into running task state
• docs/tests/orch/reconcile/reconcile-maps-done-or-failed-thread-to-terminal-task-state.md - reconcile turns terminal inbox state into task terminal state
• docs/tests/orch/wait/wait-wakes-on-matching-run-event.md - wait wakes on a later matching task event
• docs/tests/orch/wait/wait-times-out-without-matching-event.md - wait returns a non-woken timeout result
• docs/tests/orch/blocked/blocked-lists-latest-question-for-blocked-task.md - blocked list includes the active question payload
• docs/tests/orch/answer/answer-appends-answer-to-active-thread.md - answer writes an inbox answer message for the blocked attempt
• docs/tests/orch/retry/retry-creates-new-attempt-for-failed-task.md - retry creates a successor attempt after failure
• docs/tests/orch/reassign/reassign-cancels-old-thread-and-dispatches-new-attempt.md - reassign moves work to a new worker and attempt
• docs/tests/orch/cancel/cancel-cancels-single-task.md - cancel transitions one task to cancelled
• docs/tests/orch/cancel/cancel-cancels-entire-run.md - cancel cascades terminal state across the run
• docs/tests/orch/cleanup/cleanup-removes-completed-worktree.md - cleanup removes a completed attempt worktree
• docs/tests/orch/status/status-returns-run-summary-and-task-list.md - status returns run aggregate state and per-task statuses
• docs/tests/orch/council-start/council-start-dispatches-three-reviewers.md - council start creates reviewer tasks and metadata
• docs/tests/orch/council-wait/council-wait-wakes-when-all-reviewers-complete.md - council wait completes successfully when all reviewers finish
• docs/tests/orch/council-wait/council-wait-times-out-when-reviewers-incomplete.md - council wait timeout behavior
• docs/tests/orch/council-tally/council-tally-groups-reviewer-findings-in-normal-mode.md - normal similarity grouping behavior
• docs/tests/orch/council-tally/council-tally-keeps-distinct-proposals-in-strict-mode.md - strict similarity grouping behavior
• docs/tests/orch/council-report/council-report-defaults-to-consensus-and-majority.md - default report buckets and markdown artifact behavior
• docs/tests/orch/council-report/council-report-show-all-includes-minority.md - --show all includes minority output
• docs/tests/orch/council-report/council-report-json-shape-is-stable.md - JSON output shape and artifact metadata contract

Authored Case Register

• none yet