Author orch Markdown test plan
This commit is contained in:
+138
-71
@@ -25,15 +25,16 @@ Current state:
|
||||
- `orch` CLI is implemented for the current scheduler, strict worktree, wait, and council review surfaces
|
||||
- automated Go integration tests already cover the main scheduler lifecycle, dependency gating, blocked-answer flow, worktree dispatch behavior, waits, retries, reassignments, cleanup, and council start/wait/tally/report flows
|
||||
- this roadmap now exists under `docs/tests/orch/ROADMAP.md`
|
||||
- the initial `docs/tests/orch/` skeleton now exists with shared, workflow, and per-command index documents
|
||||
- command-case Markdown files have not been authored yet; the next work is filling the backlog in a stable order
|
||||
- all planned global, shared, workflow, command-index, and command-case Markdown documents in the current `orch` test-plan set have been authored
|
||||
- every implemented `orch` leaf-command folder now uses `README.md` as an index plus one Markdown file per planned case
|
||||
- workflow cases now exist in `docs/tests/orch/workflows/README.md`, and command-case coverage is aligned to the current automated integration suite
|
||||
|
||||
Progress summary for planned test-plan documents, excluding `ROADMAP.md`:
|
||||
|
||||
- planned document files: `22`
|
||||
- authored document files: `22`
|
||||
- planned case slugs in this roadmap: `35`
|
||||
- authored case slugs in this roadmap: `0`
|
||||
- planned document files: `64`
|
||||
- authored document files: `64`
|
||||
- planned case slugs in this roadmap: `46`
|
||||
- authored case slugs in this roadmap: `46`
|
||||
|
||||
## Scope
|
||||
|
||||
@@ -203,26 +204,68 @@ docs/tests/orch/
|
||||
| --- | --- | ---: | ---: | --- |
|
||||
| `docs/tests/orch/README.md` | Global testing conventions and glossary | 0 | 0 | done |
|
||||
| `docs/tests/orch/_shared/README.md` | Shared fixtures, JSON assertions, exit-code rules, and worktree conventions | 0 | 0 | done |
|
||||
| `docs/tests/orch/workflows/README.md` | Cross-command scenarios | 4 | 0 | done |
|
||||
| `docs/tests/orch/run-init/README.md` | `run init` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/run-show/README.md` | `run show` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/task-add/README.md` | `task add` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/dep-add/README.md` | `dep add` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/ready/README.md` | `ready` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/dispatch/README.md` | `dispatch` command case index | 6 | 0 | done |
|
||||
| `docs/tests/orch/reconcile/README.md` | `reconcile` command case index | 2 | 0 | done |
|
||||
| `docs/tests/orch/wait/README.md` | `wait` command case index | 2 | 0 | done |
|
||||
| `docs/tests/orch/blocked/README.md` | `blocked` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/answer/README.md` | `answer` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/retry/README.md` | `retry` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/reassign/README.md` | `reassign` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/cancel/README.md` | `cancel` command case index | 2 | 0 | done |
|
||||
| `docs/tests/orch/cleanup/README.md` | `cleanup` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/status/README.md` | `status` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/council-start/README.md` | `council start` command case index | 1 | 0 | done |
|
||||
| `docs/tests/orch/council-wait/README.md` | `council wait` command case index | 2 | 0 | done |
|
||||
| `docs/tests/orch/council-tally/README.md` | `council tally` command case index | 2 | 0 | done |
|
||||
| `docs/tests/orch/council-report/README.md` | `council report` command case index | 3 | 0 | done |
|
||||
| `docs/tests/orch/workflows/README.md` | Cross-command scenarios | 4 | 4 | done |
|
||||
| `docs/tests/orch/run-init/README.md` | `run init` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/run-init/run-init-creates-new-run.md` | `run init` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/run-show/README.md` | `run show` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md` | `run show` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/task-add/README.md` | `task add` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/task-add/task-add-creates-ready-root-task.md` | `task add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-invalid-acceptance-json.md` | `task add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-invalid-priority.md` | `task add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dep-add/README.md` | `dep add` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/dep-add/dep-add-blocks-dependent-task-until-prerequisite-completes.md` | `dep add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/ready/README.md` | `ready` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/ready/ready-lists-only-eligible-tasks.md` | `ready` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/ready/ready-orders-by-priority-and-respects-limit.md` | `ready` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dispatch/README.md` | `dispatch` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-creates-attempt-and-thread-for-ready-task.md` | `dispatch` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-rejects-non-ready-task.md` | `dispatch` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-creates-strict-worktree.md` | `dispatch` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-rejects-dirty-repo-without-base-ref.md` | `dispatch` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-allows-explicit-base-ref-on-dirty-repo.md` | `dispatch` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-auto-enables-worktree-for-code-like-task.md` | `dispatch` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-skips-auto-worktree-for-non-code-task.md` | `dispatch` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/reconcile/README.md` | `reconcile` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-claimed-or-in-progress-thread-to-running.md` | `reconcile` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-done-or-failed-thread-to-terminal-task-state.md` | `reconcile` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/wait/README.md` | `wait` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/wait/wait-wakes-on-matching-run-event.md` | `wait` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/wait/wait-times-out-without-matching-event.md` | `wait` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/blocked/README.md` | `blocked` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/blocked/blocked-lists-latest-question-for-blocked-task.md` | `blocked` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/answer/README.md` | `answer` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/answer/answer-appends-answer-to-active-thread.md` | `answer` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/answer/answer-accepts-payload-json-without-body.md` | `answer` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/answer/answer-rejects-empty-body-and-payload.md` | `answer` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/retry/README.md` | `retry` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/retry/retry-creates-new-attempt-for-failed-task.md` | `retry` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/reassign/README.md` | `reassign` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/reassign/reassign-cancels-old-thread-and-dispatches-new-attempt.md` | `reassign` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/cancel/README.md` | `cancel` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/cancel/cancel-cancels-single-task.md` | `cancel` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/cancel/cancel-cancels-entire-run.md` | `cancel` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/cleanup/README.md` | `cleanup` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/cleanup/cleanup-removes-completed-worktree.md` | `cleanup` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/cleanup/cleanup-rejects-attempt-without-task.md` | `cleanup` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/cleanup/cleanup-returns-no-matching-work-when-filters-miss.md` | `cleanup` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/status/README.md` | `status` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/status/status-returns-run-summary-and-task-list.md` | `status` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-start/README.md` | `council start` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/council-start/council-start-dispatches-three-reviewers.md` | `council start` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-wait/README.md` | `council wait` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/council-wait/council-wait-wakes-when-all-reviewers-complete.md` | `council wait` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-wait/council-wait-times-out-when-reviewers-incomplete.md` | `council wait` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-tally/README.md` | `council tally` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/council-tally/council-tally-groups-reviewer-findings-in-normal-mode.md` | `council tally` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-tally/council-tally-keeps-distinct-proposals-in-strict-mode.md` | `council tally` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-report/README.md` | `council report` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/council-report/council-report-defaults-to-consensus-and-majority.md` | `council report` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-report/council-report-show-all-includes-minority.md` | `council report` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-report/council-report-json-shape-is-stable.md` | `council report` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-report/council-report-rejects-before-tally.md` | `council report` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-report/council-report-rejects-invalid-show.md` | `council report` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/council-report/council-report-defaults-to-consensus-when-run-is-only-unanimous.md` | `council report` command case | 1 | 1 | done |
|
||||
|
||||
## Authoring Order
|
||||
|
||||
@@ -233,50 +276,74 @@ docs/tests/orch/
|
||||
5. interactive leader command docs: `wait`, `blocked`, `answer`, `retry`, `reassign`, `cancel`, `cleanup`
|
||||
6. council workflow docs: `council-start`, `council-wait`, `council-tally`, `council-report`
|
||||
|
||||
## Pending Case Backlog
|
||||
|
||||
### Workflows
|
||||
|
||||
- `docs/tests/orch/workflows/README.md :: run-dispatch-reconcile-status-happy-path` - end-to-end happy path from run creation through final `status`
|
||||
- `docs/tests/orch/workflows/README.md :: dependency-blocked-answer-resume-flow` - dependency gating plus blocked question and answer flow
|
||||
- `docs/tests/orch/workflows/README.md :: strict-worktree-dispatch-to-cleanup` - worktree creation through cleanup of a completed attempt
|
||||
- `docs/tests/orch/workflows/README.md :: council-review-end-to-end` - council start, wait, tally, and report sequence
|
||||
|
||||
### Command Cases
|
||||
|
||||
- `docs/tests/orch/run-init/run-init-creates-new-run.md` - create a run with goal and optional summary
|
||||
- `docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md` - show aggregate run metadata after activity
|
||||
- `docs/tests/orch/task-add/task-add-creates-ready-root-task.md` - adding a dependency-free task becomes `ready`
|
||||
- `docs/tests/orch/dep-add/dep-add-blocks-dependent-task-until-prerequisite-completes.md` - dependency edge prevents immediate readiness
|
||||
- `docs/tests/orch/ready/ready-lists-only-eligible-tasks.md` - ready list excludes blocked by dependency tasks
|
||||
- `docs/tests/orch/dispatch/dispatch-creates-attempt-and-thread-for-ready-task.md` - happy-path dispatch with new attempt metadata
|
||||
- `docs/tests/orch/dispatch/dispatch-rejects-non-ready-task.md` - invalid-state rejection when dispatching a gated task
|
||||
- `docs/tests/orch/dispatch/dispatch-creates-strict-worktree.md` - explicit strict worktree dispatch persists workspace metadata
|
||||
- `docs/tests/orch/dispatch/dispatch-rejects-dirty-repo-without-base-ref.md` - strict worktree guard on dirty repository state
|
||||
- `docs/tests/orch/dispatch/dispatch-allows-explicit-base-ref-on-dirty-repo.md` - explicit base ref bypasses dirty working tree rejection
|
||||
- `docs/tests/orch/dispatch/dispatch-auto-enables-worktree-for-code-like-task.md` - inferred worktree mode for code-like tasks
|
||||
- `docs/tests/orch/dispatch/dispatch-skips-auto-worktree-for-non-code-task.md` - non-code tasks stay on the normal dispatch path
|
||||
- `docs/tests/orch/reconcile/reconcile-maps-claimed-or-in-progress-thread-to-running.md` - reconcile turns claimed/in-progress inbox state into running task state
|
||||
- `docs/tests/orch/reconcile/reconcile-maps-done-or-failed-thread-to-terminal-task-state.md` - reconcile turns terminal inbox state into task terminal state
|
||||
- `docs/tests/orch/wait/wait-wakes-on-matching-run-event.md` - wait wakes on a later matching task event
|
||||
- `docs/tests/orch/wait/wait-times-out-without-matching-event.md` - wait returns a non-woken timeout result
|
||||
- `docs/tests/orch/blocked/blocked-lists-latest-question-for-blocked-task.md` - blocked list includes the active question payload
|
||||
- `docs/tests/orch/answer/answer-appends-answer-to-active-thread.md` - answer writes an inbox answer message for the blocked attempt
|
||||
- `docs/tests/orch/retry/retry-creates-new-attempt-for-failed-task.md` - retry creates a successor attempt after failure
|
||||
- `docs/tests/orch/reassign/reassign-cancels-old-thread-and-dispatches-new-attempt.md` - reassign moves work to a new worker and attempt
|
||||
- `docs/tests/orch/cancel/cancel-cancels-single-task.md` - cancel transitions one task to `cancelled`
|
||||
- `docs/tests/orch/cancel/cancel-cancels-entire-run.md` - cancel cascades terminal state across the run
|
||||
- `docs/tests/orch/cleanup/cleanup-removes-completed-worktree.md` - cleanup removes a completed attempt worktree
|
||||
- `docs/tests/orch/status/status-returns-run-summary-and-task-list.md` - status returns run aggregate state and per-task statuses
|
||||
- `docs/tests/orch/council-start/council-start-dispatches-three-reviewers.md` - council start creates reviewer tasks and metadata
|
||||
- `docs/tests/orch/council-wait/council-wait-wakes-when-all-reviewers-complete.md` - council wait completes successfully when all reviewers finish
|
||||
- `docs/tests/orch/council-wait/council-wait-times-out-when-reviewers-incomplete.md` - council wait timeout behavior
|
||||
- `docs/tests/orch/council-tally/council-tally-groups-reviewer-findings-in-normal-mode.md` - normal similarity grouping behavior
|
||||
- `docs/tests/orch/council-tally/council-tally-keeps-distinct-proposals-in-strict-mode.md` - strict similarity grouping behavior
|
||||
- `docs/tests/orch/council-report/council-report-defaults-to-consensus-and-majority.md` - default report buckets and markdown artifact behavior
|
||||
- `docs/tests/orch/council-report/council-report-show-all-includes-minority.md` - `--show all` includes minority output
|
||||
- `docs/tests/orch/council-report/council-report-json-shape-is-stable.md` - JSON output shape and artifact metadata contract
|
||||
|
||||
## Authored Case Register
|
||||
|
||||
- none yet
|
||||
| Path | Case Slug | Coverage Note | Status |
|
||||
| --- | --- | --- | --- |
|
||||
| `docs/tests/orch/workflows/README.md` | `run-dispatch-reconcile-status-happy-path` | end-to-end happy path from run creation through final status | done |
|
||||
| `docs/tests/orch/workflows/README.md` | `dependency-blocked-answer-resume-flow` | dependency gating plus blocked question and answer recovery | done |
|
||||
| `docs/tests/orch/workflows/README.md` | `strict-worktree-dispatch-to-cleanup` | worktree-backed code task flows from dispatch through cleanup | done |
|
||||
| `docs/tests/orch/workflows/README.md` | `council-review-end-to-end` | council workflow runs from reviewer dispatch through final report | done |
|
||||
| `docs/tests/orch/run-init/run-init-creates-new-run.md` | `run-init-creates-new-run` | creates a run with goal and optional summary | done |
|
||||
| `docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md` | `run-show-returns-run-summary-and-task-counts` | shows aggregate run metadata after activity | done |
|
||||
| `docs/tests/orch/task-add/task-add-creates-ready-root-task.md` | `task-add-creates-ready-root-task` | dependency-free task becomes ready immediately | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-invalid-acceptance-json.md` | `task-add-rejects-invalid-acceptance-json` | malformed `--acceptance-json` returns stable invalid_input | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-invalid-priority.md` | `task-add-rejects-invalid-priority` | unsupported priorities are rejected with invalid_input | done |
|
||||
| `docs/tests/orch/dep-add/dep-add-blocks-dependent-task-until-prerequisite-completes.md` | `dep-add-blocks-dependent-task-until-prerequisite-completes` | dependency edge prevents immediate readiness | done |
|
||||
| `docs/tests/orch/ready/ready-lists-only-eligible-tasks.md` | `ready-lists-only-eligible-tasks` | ready list excludes dependency-gated tasks | done |
|
||||
| `docs/tests/orch/ready/ready-orders-by-priority-and-respects-limit.md` | `ready-orders-by-priority-and-respects-limit` | ready output orders by priority and applies explicit limit truncation | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-creates-attempt-and-thread-for-ready-task.md` | `dispatch-creates-attempt-and-thread-for-ready-task` | ready task dispatch creates attempt, thread, and task message | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-rejects-non-ready-task.md` | `dispatch-rejects-non-ready-task` | dispatch on gated task returns invalid_state | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-creates-strict-worktree.md` | `dispatch-creates-strict-worktree` | explicit strict worktree dispatch provisions isolated workspace metadata | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-rejects-dirty-repo-without-base-ref.md` | `dispatch-rejects-dirty-repo-without-base-ref` | dirty repository without explicit base ref is rejected in strict mode | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-allows-explicit-base-ref-on-dirty-repo.md` | `dispatch-allows-explicit-base-ref-on-dirty-repo` | explicit base ref allows dispatch from a dirty repository | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-auto-enables-worktree-for-code-like-task.md` | `dispatch-auto-enables-worktree-for-code-like-task` | code-like task metadata auto-enables worktree mode | done |
|
||||
| `docs/tests/orch/dispatch/dispatch-skips-auto-worktree-for-non-code-task.md` | `dispatch-skips-auto-worktree-for-non-code-task` | clearly non-code tasks stay on normal dispatch path | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-claimed-or-in-progress-thread-to-running.md` | `reconcile-maps-claimed-or-in-progress-thread-to-running` | reconcile maps active inbox execution to running task state | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-done-or-failed-thread-to-terminal-task-state.md` | `reconcile-maps-done-or-failed-thread-to-terminal-task-state` | reconcile maps terminal inbox states to terminal task states | done |
|
||||
| `docs/tests/orch/wait/wait-wakes-on-matching-run-event.md` | `wait-wakes-on-matching-run-event` | wait wakes on a later matching run-scoped event | done |
|
||||
| `docs/tests/orch/wait/wait-times-out-without-matching-event.md` | `wait-times-out-without-matching-event` | wait timeout returns a normal non-woken result | done |
|
||||
| `docs/tests/orch/blocked/blocked-lists-latest-question-for-blocked-task.md` | `blocked-lists-latest-question-for-blocked-task` | blocked view includes latest question payload for the task | done |
|
||||
| `docs/tests/orch/answer/answer-appends-answer-to-active-thread.md` | `answer-appends-answer-to-active-thread` | answer appends an inbox answer message to the blocked attempt thread | done |
|
||||
| `docs/tests/orch/answer/answer-accepts-payload-json-without-body.md` | `answer-accepts-payload-json-without-body` | payload-only answers stay valid and machine-readable | done |
|
||||
| `docs/tests/orch/answer/answer-rejects-empty-body-and-payload.md` | `answer-rejects-empty-body-and-payload` | empty answer requests fail with invalid_input | done |
|
||||
| `docs/tests/orch/retry/retry-creates-new-attempt-for-failed-task.md` | `retry-creates-new-attempt-for-failed-task` | retry dispatches a successor attempt after failure | done |
|
||||
| `docs/tests/orch/reassign/reassign-cancels-old-thread-and-dispatches-new-attempt.md` | `reassign-cancels-old-thread-and-dispatches-new-attempt` | reassign cancels old execution and opens a new attempt | done |
|
||||
| `docs/tests/orch/cancel/cancel-cancels-single-task.md` | `cancel-cancels-single-task` | single-task cancel moves only the targeted task to cancelled | done |
|
||||
| `docs/tests/orch/cancel/cancel-cancels-entire-run.md` | `cancel-cancels-entire-run` | run cancel cascades terminal state across the run | done |
|
||||
| `docs/tests/orch/cleanup/cleanup-removes-completed-worktree.md` | `cleanup-removes-completed-worktree` | cleanup removes completed attempt worktree artifacts | done |
|
||||
| `docs/tests/orch/cleanup/cleanup-rejects-attempt-without-task.md` | `cleanup-rejects-attempt-without-task` | cleanup enforces `--task` when `--attempt` is specified | done |
|
||||
| `docs/tests/orch/cleanup/cleanup-returns-no-matching-work-when-filters-miss.md` | `cleanup-returns-no-matching-work-when-filters-miss` | cleanup returns no_matching_work when selectors find no candidates | done |
|
||||
| `docs/tests/orch/status/status-returns-run-summary-and-task-list.md` | `status-returns-run-summary-and-task-list` | status reports aggregate run state and per-task statuses | done |
|
||||
| `docs/tests/orch/council-start/council-start-dispatches-three-reviewers.md` | `council-start-dispatches-three-reviewers` | council start creates and dispatches three fixed reviewer tasks | done |
|
||||
| `docs/tests/orch/council-wait/council-wait-wakes-when-all-reviewers-complete.md` | `council-wait-wakes-when-all-reviewers-complete` | council wait wakes when all reviewer tasks complete | done |
|
||||
| `docs/tests/orch/council-wait/council-wait-times-out-when-reviewers-incomplete.md` | `council-wait-times-out-when-reviewers-incomplete` | council wait timeout stays machine-readable | done |
|
||||
| `docs/tests/orch/council-tally/council-tally-groups-reviewer-findings-in-normal-mode.md` | `council-tally-groups-reviewer-findings-in-normal-mode` | normal similarity groups semantically aligned reviewer findings | done |
|
||||
| `docs/tests/orch/council-tally/council-tally-keeps-distinct-proposals-in-strict-mode.md` | `council-tally-keeps-distinct-proposals-in-strict-mode` | strict similarity preserves wording-level proposal separation | done |
|
||||
| `docs/tests/orch/council-report/council-report-defaults-to-consensus-and-majority.md` | `council-report-defaults-to-consensus-and-majority` | default report keeps main output on consensus and majority buckets | done |
|
||||
| `docs/tests/orch/council-report/council-report-show-all-includes-minority.md` | `council-report-show-all-includes-minority` | `--show all` includes minority recommendations in final report | done |
|
||||
| `docs/tests/orch/council-report/council-report-json-shape-is-stable.md` | `council-report-json-shape-is-stable` | JSON response shape and report artifact metadata remain stable | done |
|
||||
| `docs/tests/orch/council-report/council-report-rejects-before-tally.md` | `council-report-rejects-before-tally` | report generation before tally fails with invalid_state | done |
|
||||
| `docs/tests/orch/council-report/council-report-rejects-invalid-show.md` | `council-report-rejects-invalid-show` | unsupported `--show` values return invalid_input | done |
|
||||
| `docs/tests/orch/council-report/council-report-defaults-to-consensus-when-run-is-only-unanimous.md` | `council-report-defaults-to-consensus-when-run-is-only-unanimous` | omitted `--show` collapses to consensus for only-unanimous runs | done |
|
||||
|
||||
## Pending Case Backlog
|
||||
|
||||
No pending case slugs remain in the current plan.
|
||||
|
||||
When a new `orch` CLI contract or workflow needs coverage:
|
||||
|
||||
1. if it is a command case, create a new `<case-slug>.md` file under the relevant leaf-command folder and add it to that folder `README.md` index
|
||||
2. if it is a workflow case, add it to `docs/tests/orch/workflows/README.md`
|
||||
3. add the new slug to `Authored Case Register`
|
||||
4. update `Current Snapshot` and `Document Progress`
|
||||
|
||||
## Definition Of Done
|
||||
|
||||
This roadmap is complete only when all of the following are true:
|
||||
|
||||
- every implemented `orch` leaf command has a corresponding document folder
|
||||
- each planned command index and case document exists
|
||||
- each pending case slug has been either authored or explicitly deferred
|
||||
- the authored-case register matches the actual Markdown files on disk
|
||||
- a new agent can pick any future case and know exactly where it should be written
|
||||
|
||||
@@ -116,6 +116,22 @@ When worktree behavior is under test, assert at least:
|
||||
- attempt `worktree_path`
|
||||
- attempt `workspace_status`
|
||||
|
||||
## Direct DB Inspection
|
||||
|
||||
Most `orch` cases should stay at the CLI contract level, but a few manual reproduction flows need direct SQL reads to recover attempt-to-thread mappings that the current `orch` CLI does not print in a standalone query command.
|
||||
|
||||
When a case truly needs that mapping:
|
||||
|
||||
- use a read-only `sqlite3` query against `TMPDIR/coord.db`
|
||||
- prefer querying `task_attempts` by stable keys such as `run_id`, `task_id`, and `attempt_no`
|
||||
- treat the SQL read as fixture setup for the next CLI command, not as the main assertion target
|
||||
|
||||
Typical example:
|
||||
|
||||
```bash
|
||||
sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_001' AND task_id = 'CR1' AND attempt_no = 1;"
|
||||
```
|
||||
|
||||
## Workflow Authoring Rule
|
||||
|
||||
If a case spans multiple `orch` commands, place the end-to-end narrative in `workflows/README.md` first, then add narrower command-level cases only when they are easier to reason about in isolation.
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
# Orch `answer` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `answer-appends-answer-to-active-thread` | [answer-appends-answer-to-active-thread.md](./answer-appends-answer-to-active-thread.md) | appends an inbox answer message onto the active blocked attempt thread |
|
||||
| `answer-accepts-payload-json-without-body` | [answer-accepts-payload-json-without-body.md](./answer-accepts-payload-json-without-body.md) | accepts structured `--payload-json` input even when no body text is provided |
|
||||
| `answer-rejects-empty-body-and-payload` | [answer-rejects-empty-body-and-payload.md](./answer-rejects-empty-body-and-payload.md) | rejects an answer request that provides neither body text nor payload JSON |
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# Case: `answer-accepts-payload-json-without-body`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `answer` 在未提供 `--body` 的情况下，仍可通过纯 `--payload-json` 向当前阻塞尝试写回结构化决策。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 运行 `run_blog_002` 中的任务 `T2` 已处于 `blocked`
|
||||
- `blocked` 列表中可见 `T2`
|
||||
- 已知该阻塞尝试对应线程为 `THREAD_ID`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json answer --run run_blog_002 --task T2 --payload-json '{"decision":"stdout","source":"leader"}'
|
||||
inbox --db TMPDIR/coord.db --json show --thread THREAD_ID
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `answer` 退出码为 `0`
|
||||
- `answer.data.message.kind == "answer"`
|
||||
- `answer.data.message.payload_json.decision == "stdout"`
|
||||
- `answer.data.message.payload_json.source == "leader"`
|
||||
- `show.data.messages` 末尾新增一条 `kind=answer` 的消息
|
||||
- 末尾消息的 `payload_json.decision == "stdout"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `answer` 不要求 leader 必须提供纯文本正文；结构化 payload 本身就可以构成有效答复
|
||||
- worker 可以从同一条 `answer` 消息里读取结构化决策，而不必依赖约定俗成的正文格式
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `--payload-json` 必须是合法 JSON；非法值应返回 `invalid_input`
|
||||
- `--body` 与 `--body-file` 仍然互斥，即使本用例不使用它们
|
||||
@@ -0,0 +1,36 @@
|
||||
# Case: `answer-appends-answer-to-active-thread`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `answer` 会把 leader 的答复写回当前阻塞尝试的 inbox 线程，并以 `answer` 消息形式供 worker 继续消费。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 运行 `run_blog_002` 中的任务 `T2` 已处于 `blocked`
|
||||
- `blocked` 列表中可见 `T2`
|
||||
- 已知该阻塞尝试对应线程为 `THREAD_ID`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json answer --run run_blog_002 --task T2 --body "Use stdout for MVP."
|
||||
inbox --db TMPDIR/coord.db --json show --thread THREAD_ID
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `answer` 退出码为 `0`
|
||||
- `answer.data.message.kind == "answer"`
|
||||
- `answer.data.task.task_id == "T2"`
|
||||
- `show.data.messages` 末尾新增一条 `kind=answer` 的消息
|
||||
- 末尾消息 `body == "Use stdout for MVP."`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `answer` 的本质是向活动线程追加 leader 决策消息，而不是直接修改任务状态
|
||||
- worker 仍需继续通过 `inbox` 或后续 `reconcile` 推进任务状态
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `answer` 支持 `--body-file` 与 `--payload-json`
|
||||
- `--body` 与 `--body-file` 互斥；若两者都为空，则至少需要提供 `--payload-json`
|
||||
@@ -0,0 +1,29 @@
|
||||
# Case: `answer-rejects-empty-body-and-payload`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `answer` 在既没有正文也没有结构化 payload 时返回稳定输入错误，而不是写入空答复消息。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 运行 `run_blog_002` 中的任务 `T2` 已处于 `blocked`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json answer --run run_blog_002 --task T2
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 退出码为 `30`
|
||||
- JSON 错误码为 `invalid_input`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `answer` 至少需要一种有效输入载荷：正文或 `payload-json`
|
||||
- 空答复会在写入线程前被拒绝，而不是生成一条语义不明的 `answer` 消息
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 若同时传入 `--body` 和 `--body-file`，也应返回 `invalid_input`
|
||||
@@ -1,7 +1,7 @@
|
||||
# Orch `blocked` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `blocked-lists-latest-question-for-blocked-task` | [blocked-lists-latest-question-for-blocked-task.md](./blocked-lists-latest-question-for-blocked-task.md) | lists blocked tasks together with the latest worker question payload |
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# Case: `blocked-lists-latest-question-for-blocked-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `blocked` 会列出当前阻塞任务，并附带最新问题消息，便于 leader 直接做决策。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已创建运行 `run_blog_002`
|
||||
- 已创建任务 `T1`、`T2`，且 `T2` 依赖 `T1`
|
||||
- `T1` 已完成并经 `reconcile` 推进，使 `T2` 变为 `ready`
|
||||
- `T2` 已完成 `dispatch`
|
||||
- `worker-b` 已 `claim` `T2` 对应线程，并通过 `inbox update --status blocked` 写入问题
|
||||
- 最近一次 `reconcile` 已执行
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json blocked --run run_blog_002
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 退出码为 `0`
|
||||
- `blocked.data.blocked` 长度为 `1`
|
||||
- 唯一条目的 `task.task_id == "T2"`
|
||||
- `question.kind == "question"`
|
||||
- `question.summary == "Need logging decision"`
|
||||
- `question.payload_json.question == "stdout or stderr?"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `blocked` 返回的不只是任务状态，还会附带 leader 真正需要回答的问题消息
|
||||
- 该命令适合作为 leader 的“待答复队列”入口，而不是只做状态列表展示
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 若没有阻塞任务，非 JSON 输出会打印 `no blocked tasks`
|
||||
@@ -1,7 +1,8 @@
|
||||
# Orch `cancel` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `cancel-cancels-single-task` | [cancel-cancels-single-task.md](./cancel-cancels-single-task.md) | cancels one task without implicitly cancelling unrelated tasks in the same run |
|
||||
| `cancel-cancels-entire-run` | [cancel-cancels-entire-run.md](./cancel-cancels-entire-run.md) | cancels the run and forces every task into the cancelled terminal state |
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# Case: `cancel-cancels-entire-run`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证不带 `--task` 的 `cancel` 会取消整个运行，并把所有任务推进到 `cancelled`。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 运行 `run_blog_cancel_001` 已存在
|
||||
- 该运行下至少有 `T1`、`T2` 两个任务
|
||||
- 在执行本用例前，可能已有单任务取消发生
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json cancel --run run_blog_cancel_001 --reason "Stop the run."
|
||||
orch --db TMPDIR/coord.db --json status --run run_blog_cancel_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `cancel` 退出码为 `0`
|
||||
- `cancel.data.run.status == "cancelled"`
|
||||
- `status.data.run.status == "cancelled"`
|
||||
- `status.data.tasks` 中所有任务的 `status` 都为 `cancelled`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- 运行级取消会级联终止运行下的全部任务
|
||||
- 该命令是 leader 主动停止整个调度的主入口，而不是只做标记
|
||||
@@ -0,0 +1,32 @@
|
||||
# Case: `cancel-cancels-single-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `cancel --task` 只取消指定任务，不会隐式取消同一运行中的其他任务。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已创建运行 `run_blog_cancel_001`
|
||||
- 已创建任务 `T1`、`T2`
|
||||
- `T1` 已完成 `dispatch`
|
||||
- 已知 `T1` 对应线程为 `THREAD_ID`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json cancel --run run_blog_cancel_001 --task T1 --reason "Task is no longer needed."
|
||||
orch --db TMPDIR/coord.db --json status --run run_blog_cancel_001
|
||||
inbox --db TMPDIR/coord.db --json show --thread THREAD_ID
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `cancel` 退出码为 `0`
|
||||
- `status` 中 `T1.status == "cancelled"`
|
||||
- `status` 中 `T2` 仍保持非 `cancelled` 状态
|
||||
- `show.data.thread.status == "cancelled"`，指向 `T1` 的原线程
|
||||
|
||||
## 断言结论
|
||||
|
||||
- 单任务取消是局部控制动作，不会把运行整体终止
|
||||
- 对已分派任务，取消也会同步终止对应 inbox 线程，避免 worker 继续执行
|
||||
@@ -1,7 +1,9 @@
|
||||
# Orch `cleanup` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `cleanup-removes-completed-worktree` | [cleanup-removes-completed-worktree.md](./cleanup-removes-completed-worktree.md) | removes a completed attempt worktree and records the cleanup result |
|
||||
| `cleanup-rejects-attempt-without-task` | [cleanup-rejects-attempt-without-task.md](./cleanup-rejects-attempt-without-task.md) | rejects `--attempt` when no matching `--task` selector is provided |
|
||||
| `cleanup-returns-no-matching-work-when-filters-miss` | [cleanup-returns-no-matching-work-when-filters-miss.md](./cleanup-returns-no-matching-work-when-filters-miss.md) | returns the stable no-matching-work contract when cleanup filters yield no candidates |
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# Case: `cleanup-rejects-attempt-without-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `cleanup` 在使用 `--attempt` 精确选择尝试时，要求同时提供 `--task`，避免对 run 级别尝试号产生歧义。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已创建运行 `run_blog_cleanup_002`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json cleanup --run run_blog_cleanup_002 --attempt 1
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 退出码为 `30`
|
||||
- JSON 错误码为 `invalid_input`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `cleanup` 的选择器组合在查询前就会进行基本输入校验
|
||||
- `--attempt` 不是独立的 run 级过滤器，必须依附具体 `task`
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 若既未提供 `--task`，也未提供 `--attempt` 或 `--all-completed`，同样应返回 `invalid_input`
|
||||
@@ -0,0 +1,37 @@
|
||||
# Case: `cleanup-removes-completed-worktree`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `cleanup` 会移除已完成尝试的 worktree，并把清理结果返回给 leader。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已创建运行 `run_blog_cleanup_001`
|
||||
- 已创建任务 `T1`
|
||||
- `T1` 已通过严格 worktree 模式完成 `dispatch`
|
||||
- `worker-a` 已完成 `claim` 并通过 `inbox done` 把线程推进到 `done`
|
||||
- 最近一次 `reconcile` 已执行，使任务状态同步为 `done`
|
||||
- 已知当前尝试的 worktree 路径为 `WORKTREE_PATH`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json cleanup --run run_blog_cleanup_001 --task T1
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 退出码为 `0`
|
||||
- `cleanup.data.cleaned` 长度为 `1`
|
||||
- 唯一记录对应 `T1` 的已完成尝试
|
||||
- `WORKTREE_PATH` 在文件系统上已不存在
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `cleanup` 针对的是尝试工作区资源，不会改变任务的完成结果
|
||||
- 成功清理后，leader 可以安全回收已终态尝试占用的 worktree
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `cleanup` 支持按 `--task`、`--attempt` 或 `--all-completed` 选择范围
|
||||
- `--force` 用于非常规清理；本用例验证的是常规完成态清理路径
|
||||
@@ -0,0 +1,33 @@
|
||||
# Case: `cleanup-returns-no-matching-work-when-filters-miss`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `cleanup` 在筛选条件没有命中任何可清理 worktree 时，返回稳定的“无匹配工作”契约，而不是成功空列表。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已创建运行 `run_blog_cleanup_003`
|
||||
- 已创建任务 `T1`
|
||||
- 当前 run 中不存在 `workspace_status` 为 `completed` 或 `abandoned` 的 worktree 尝试
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_cleanup_003 --goal "Validate cleanup empty result"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_cleanup_003 --task T1 --title "Prepare cleanup target"
|
||||
orch --db TMPDIR/coord.db --json cleanup --run run_blog_cleanup_003 --task T1
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `cleanup` 退出码为 `10`
|
||||
- JSON 错误码为 `no_matching_work`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `cleanup` 对空筛选结果使用显式 no-matching-work 信号，而不是返回成功空数组
|
||||
- leader 或脚本可以据此区分“没有候选 worktree”与“清理已成功完成”
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 该契约同样适用于使用 `--all-completed` 或 `--attempt` 时筛选不到候选的场景
|
||||
@@ -1,7 +1,12 @@
|
||||
# Orch `council report` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `council-report-defaults-to-consensus-and-majority` | [council-report-defaults-to-consensus-and-majority.md](./council-report-defaults-to-consensus-and-majority.md) | renders a markdown report that omits minority recommendations by default and writes a markdown artifact |
|
||||
| `council-report-show-all-includes-minority` | [council-report-show-all-includes-minority.md](./council-report-show-all-includes-minority.md) | includes minority recommendations when `--show all` is requested |
|
||||
| `council-report-json-shape-is-stable` | [council-report-json-shape-is-stable.md](./council-report-json-shape-is-stable.md) | returns the stable JSON report contract with summary, filtered groups, and artifact metadata |
|
||||
| `council-report-rejects-before-tally` | [council-report-rejects-before-tally.md](./council-report-rejects-before-tally.md) | rejects report generation with `invalid_state` when grouped recommendations have not been tallied yet |
|
||||
| `council-report-rejects-invalid-show` | [council-report-rejects-invalid-show.md](./council-report-rejects-invalid-show.md) | rejects unsupported `--show` bucket values with `invalid_input` |
|
||||
| `council-report-defaults-to-consensus-when-run-is-only-unanimous` | [council-report-defaults-to-consensus-when-run-is-only-unanimous.md](./council-report-defaults-to-consensus-when-run-is-only-unanimous.md) | defaults omitted `--show` to `consensus` when the run was started with `--only-unanimous` |
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
# Case: `council-report-defaults-to-consensus-and-majority`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council report` 默认只展示 `consensus` 与 `majority` bucket，同时生成 markdown artifact。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 使用隔离的临时目录 `TMPDIR`
|
||||
- 已准备好能产出 `consensus`、`majority`、`minority` 三类 recommendation 的 reviewer 输出 JSON
|
||||
- 本地可使用 `sqlite3` 从 `task_attempts` 中读取 reviewer thread ID
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
cat <<'EOF' > TMPDIR/architecture-review.json
|
||||
{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}},{"title":"Share helpers","summary":"Council report rendering paths are repeated.","proposal":"Introduce shared council coordinator helpers for report rendering.","rationale":"This keeps report assembly consistent.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
cat <<'EOF' > TMPDIR/implementation-review.json
|
||||
{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"high","tags":["maintainability"],"target_refs":{"repo_path":"."}},{"title":"Reuse report helpers","summary":"Formatting logic should stay shared.","proposal":"Introduce shared council coordinator helpers for report rendering","rationale":"This avoids formatter drift.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
cat <<'EOF' > TMPDIR/risk-review.json
|
||||
{"reviewer_role":"risk-reviewer","findings":[{"title":"Lock contracts","summary":"Contract drift becomes risky over time.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This reduces integration regressions.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}},{"title":"Cover JSON output","summary":"The council report response should stay stable.","proposal":"Add regression tests for council report JSON output.","rationale":"This catches contract regressions earlier.","confidence":"high","tags":["testing"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
orch --db TMPDIR/coord.db --json council start \
|
||||
--run council_blog_report_001 \
|
||||
--target "Review the council reporting flow."
|
||||
|
||||
THREAD_ID_CR1=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_report_001' AND task_id = 'CR1' AND attempt_no = 1;")
|
||||
THREAD_ID_CR2=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_report_001' AND task_id = 'CR2' AND attempt_no = 1;")
|
||||
THREAD_ID_CR3=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_report_001' AND task_id = 'CR3' AND attempt_no = 1;")
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent architecture-reviewer --thread "$THREAD_ID_CR1"
|
||||
inbox --db TMPDIR/coord.db --json done --agent architecture-reviewer --thread "$THREAD_ID_CR1" --summary "Review complete" --body-file TMPDIR/architecture-review.json
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent implementation-reviewer --thread "$THREAD_ID_CR2"
|
||||
inbox --db TMPDIR/coord.db --json done --agent implementation-reviewer --thread "$THREAD_ID_CR2" --summary "Review complete" --body-file TMPDIR/implementation-review.json
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent risk-reviewer --thread "$THREAD_ID_CR3"
|
||||
inbox --db TMPDIR/coord.db --json done --agent risk-reviewer --thread "$THREAD_ID_CR3" --summary "Review complete" --body-file TMPDIR/risk-review.json
|
||||
|
||||
orch --db TMPDIR/coord.db --json council tally \
|
||||
--run council_blog_report_001 \
|
||||
--similarity normal
|
||||
|
||||
orch --db TMPDIR/coord.db council report \
|
||||
--run council_blog_report_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `council report` 退出码为 `0`
|
||||
- stdout 是 markdown，而不是 JSON
|
||||
- 报告正文包含 `# Council Review Report`
|
||||
- 报告正文包含 `## Consensus`
|
||||
- 报告正文包含 `## Majority`
|
||||
- 报告正文不包含 `## Minority`
|
||||
- `TMPDIR/.orch/reports/council_blog_report_001.md` 被创建，且内容与 stdout 一致
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `council report` 的默认呈现策略是“主报告展示 consensus + majority，隐藏 minority”
|
||||
- 该命令既是渲染命令，也是 artifact 产出命令
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 默认 bucket 行为受 `council run` 的 `only_unanimous` 配置影响；当前常规路径默认仍是 `consensus,majority`
|
||||
+73
@@ -0,0 +1,73 @@
|
||||
# Case: `council-report-defaults-to-consensus-when-run-is-only-unanimous`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证当 council run 以 `--only-unanimous` 启动时，省略 `--show` 的 `council report --json` 默认只返回 `consensus` bucket。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 使用隔离的临时目录 `TMPDIR`
|
||||
- 已准备好与 `council-report-defaults-to-consensus-and-majority` 相同的 3 份 reviewer 输出 JSON
|
||||
- 本地可使用 `sqlite3` 从 `task_attempts` 中读取 reviewer thread ID
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
cat <<'EOF' > TMPDIR/architecture-review.json
|
||||
{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}},{"title":"Share helpers","summary":"Council report rendering paths are repeated.","proposal":"Introduce shared council coordinator helpers for report rendering.","rationale":"This keeps report assembly consistent.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
cat <<'EOF' > TMPDIR/implementation-review.json
|
||||
{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"high","tags":["maintainability"],"target_refs":{"repo_path":"."}},{"title":"Reuse report helpers","summary":"Formatting logic should stay shared.","proposal":"Introduce shared council coordinator helpers for report rendering","rationale":"This avoids formatter drift.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
cat <<'EOF' > TMPDIR/risk-review.json
|
||||
{"reviewer_role":"risk-reviewer","findings":[{"title":"Lock contracts","summary":"Contract drift becomes risky over time.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This reduces integration regressions.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}},{"title":"Cover JSON output","summary":"The council report response should stay stable.","proposal":"Add regression tests for council report JSON output.","rationale":"This catches contract regressions earlier.","confidence":"high","tags":["testing"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
orch --db TMPDIR/coord.db --json council start \
|
||||
--run council_blog_report_011 \
|
||||
--target "Review the council reporting flow." \
|
||||
--only-unanimous
|
||||
|
||||
THREAD_ID_CR1=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_report_011' AND task_id = 'CR1' AND attempt_no = 1;")
|
||||
THREAD_ID_CR2=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_report_011' AND task_id = 'CR2' AND attempt_no = 1;")
|
||||
THREAD_ID_CR3=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_report_011' AND task_id = 'CR3' AND attempt_no = 1;")
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent architecture-reviewer --thread "$THREAD_ID_CR1"
|
||||
inbox --db TMPDIR/coord.db --json done --agent architecture-reviewer --thread "$THREAD_ID_CR1" --summary "Review complete" --body-file TMPDIR/architecture-review.json
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent implementation-reviewer --thread "$THREAD_ID_CR2"
|
||||
inbox --db TMPDIR/coord.db --json done --agent implementation-reviewer --thread "$THREAD_ID_CR2" --summary "Review complete" --body-file TMPDIR/implementation-review.json
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent risk-reviewer --thread "$THREAD_ID_CR3"
|
||||
inbox --db TMPDIR/coord.db --json done --agent risk-reviewer --thread "$THREAD_ID_CR3" --summary "Review complete" --body-file TMPDIR/risk-review.json
|
||||
|
||||
orch --db TMPDIR/coord.db --json council tally \
|
||||
--run council_blog_report_011 \
|
||||
--similarity normal
|
||||
|
||||
orch --db TMPDIR/coord.db --json council report \
|
||||
--run council_blog_report_011
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 最后一条 `council report` 命令退出码为 `0`
|
||||
- `ok == true`
|
||||
- `data.run_id == "council_blog_report_011"`
|
||||
- `data.show == ["consensus"]`
|
||||
- `data.summary.consensus == 1`
|
||||
- `data.summary.majority == 1`
|
||||
- `data.summary.minority == 1`
|
||||
- `data.grouped_recommendations` 长度为 `1`
|
||||
- 唯一返回的 recommendation 的 `bucket == "consensus"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `--only-unanimous` 不会删除持久化的 `majority` 或 `minority` 数据，但会改变省略 `--show` 时的默认输出策略
|
||||
- leader 若希望在 unanimous-only run 中仍查看 `majority`，必须显式传入 `--show`
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 即使这里使用 `--json` 断言 `show` 默认值，命令仍会写出 markdown artifact
|
||||
@@ -0,0 +1,41 @@
|
||||
# Case: `council-report-json-shape-is-stable`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council report --json` 返回稳定 JSON 契约，包含 `show`、`summary`、过滤后的 grouped recommendations，以及 report artifact 元数据。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已按 `council-report-defaults-to-consensus-and-majority` 的前置流程完成 reviewer 输出与 `council tally`
|
||||
- 运行 ID 为 `council_blog_report_003`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json council report \
|
||||
--run council_blog_report_003
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 退出码为 `0`
|
||||
- `ok == true`
|
||||
- `command == "council report"`
|
||||
- `data.run_id == "council_blog_report_003"`
|
||||
- `data.show == ["consensus","majority"]`
|
||||
- `data.summary.consensus == 1`
|
||||
- `data.summary.majority == 1`
|
||||
- `data.summary.minority == 1`
|
||||
- `data.report_artifacts` 长度为 `1`
|
||||
- 首个 artifact 的 `kind == "markdown"`
|
||||
- `data.grouped_recommendations` 长度为 `2`
|
||||
- 第一组 recommendation 的 `bucket == "consensus"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `--json` 模式返回的是 leader 可继续消费的稳定 machine-readable contract
|
||||
- 默认 JSON 输出只返回被当前 `show` 过滤后的 recommendation，而 summary 仍保留全量 bucket 统计
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 即使 `--json` 模式返回 artifact path，markdown artifact 仍应实际落盘
|
||||
@@ -0,0 +1,32 @@
|
||||
# Case: `council-report-rejects-before-tally`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council report` 在还没有持久化 grouped recommendations 时会返回稳定的 `invalid_state` 契约，而不是生成空报告。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 使用隔离的临时目录 `TMPDIR`
|
||||
- 当前数据库中尚未对该 council run 执行 `council tally`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json council start \
|
||||
--run council_blog_report_010 \
|
||||
--target "Review the council reporting flow."
|
||||
|
||||
orch --db TMPDIR/coord.db --json council report \
|
||||
--run council_blog_report_010
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 第二条 `council report` 命令退出码为 `30`
|
||||
- JSON 错误码为 `invalid_state`
|
||||
- 错误消息指出 grouped recommendations 尚不可用，需先执行 `council tally`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `council report` 不是“边读 reviewer 输出边临时汇总”的命令
|
||||
- report 阶段依赖已持久化的 `council_groups`，因此 `tally -> report` 的顺序是稳定 CLI 契约
|
||||
@@ -0,0 +1,29 @@
|
||||
# Case: `council-report-rejects-invalid-show`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council report --show` 对非法 bucket 值返回稳定的 `invalid_input`，避免 leader 误以为未知 bucket 会被静默忽略。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已按 `council-report-defaults-to-consensus-and-majority` 的前置流程完成 reviewer 输出与 `council tally`
|
||||
- 运行 ID 为 `council_blog_report_001`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json council report \
|
||||
--run council_blog_report_001 \
|
||||
--show consensus,invalid
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 退出码为 `30`
|
||||
- JSON 错误码为 `invalid_input`
|
||||
- 错误消息说明 `--show` 只接受 `consensus`、`majority`、`minority` 或 `all`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `--show` 不是宽松过滤参数；未知 bucket 会触发显式输入错误
|
||||
- leader 侧脚本可以依赖这一点来尽早发现错误配置，而不是事后对空报告排障
|
||||
@@ -0,0 +1,29 @@
|
||||
# Case: `council-report-show-all-includes-minority`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council report --show all` 会把默认被省略的 `minority` recommendation 一并展示出来。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已按 `council-report-defaults-to-consensus-and-majority` 的前置流程完成 reviewer 输出与 `council tally`
|
||||
- 运行 ID 为 `council_blog_report_002`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db council report \
|
||||
--run council_blog_report_002 \
|
||||
--show all
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `council report` 退出码为 `0`
|
||||
- stdout markdown 同时包含 `## Consensus`、`## Majority`、`## Minority`
|
||||
- markdown 中出现 minority proposal，例如 `Add regression tests for council report JSON output.`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `--show all` 会覆盖默认的 bucket 过滤策略
|
||||
- `minority` recommendation 会保留在持久化数据里，只是默认不进入主报告
|
||||
@@ -1,7 +1,7 @@
|
||||
# Orch `council start` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `council-start-dispatches-three-reviewers` | [council-start-dispatches-three-reviewers.md](./council-start-dispatches-three-reviewers.md) | creates the council run, dispatches the fixed three reviewer roles, and exposes the expected default metadata |
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# Case: `council-start-dispatches-three-reviewers`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council start` 会创建一个新的 council run，并立即分派固定的三位 reviewer：`architecture-reviewer`、`implementation-reviewer`、`risk-reviewer`。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 使用隔离的临时目录 `TMPDIR`
|
||||
- 目标数据库 `TMPDIR/coord.db` 尚不存在
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json council start \
|
||||
--run council_blog_001 \
|
||||
--target "Review the current blog architecture and propose optimizations." \
|
||||
--target-type mixed \
|
||||
--output both
|
||||
|
||||
orch --db TMPDIR/coord.db --json status --run council_blog_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `council start` 退出码为 `0`
|
||||
- `start.data.run_id == "council_blog_001"`
|
||||
- `start.data.mode == "brainstorm"`
|
||||
- `start.data.target_type == "mixed"`
|
||||
- `start.data.output == "both"`
|
||||
- `start.data.only_unanimous == false`
|
||||
- `start.data.reviewers` 长度为 `3`
|
||||
- 三个 reviewer 的 `reviewer_role` 分别为 `architecture-reviewer`、`implementation-reviewer`、`risk-reviewer`
|
||||
- 三个 reviewer 的 `status` 都是 `dispatched`
|
||||
- 后续 `status` 返回 `3` 个 task，run 处于活动中而不是终态
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `council start` 不只是创建 run 元数据，还会直接完成 reviewer task 的创建与分派
|
||||
- v1 reviewer 集合是固定的三角色集合，而不是由用户动态指定
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 未显式传入 `--mode` 时，默认回退到 `brainstorm`
|
||||
- 未显式传入 `--only-unanimous` 时，默认值是 `false`
|
||||
- council reviewer task 在当前实现里不应自动申请 code worktree
|
||||
@@ -1,7 +1,8 @@
|
||||
# Orch `council tally` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `council-tally-groups-reviewer-findings-in-normal-mode` | [council-tally-groups-reviewer-findings-in-normal-mode.md](./council-tally-groups-reviewer-findings-in-normal-mode.md) | groups semantically similar reviewer outputs into majority and minority buckets in `normal` mode |
|
||||
| `council-tally-keeps-distinct-proposals-in-strict-mode` | [council-tally-keeps-distinct-proposals-in-strict-mode.md](./council-tally-keeps-distinct-proposals-in-strict-mode.md) | preserves wording differences as separate minority groups in `strict` mode |
|
||||
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
# Case: `council-tally-groups-reviewer-findings-in-normal-mode`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council tally --similarity normal` 会把语义相近的 reviewer proposal 合并到同一组，并产出 `majority` / `minority` bucket。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 使用隔离的临时目录 `TMPDIR`
|
||||
- 本地可使用 `sqlite3` 从 `task_attempts` 中读取 reviewer thread ID
|
||||
- 已准备好三份 reviewer 输出 JSON；其中 architecture 与 implementation proposal 语义相近，risk proposal 独立
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
cat <<'EOF' > TMPDIR/architecture-review.json
|
||||
{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture","coupling"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
cat <<'EOF' > TMPDIR/implementation-review.json
|
||||
{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract API contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"medium","tags":["maintainability"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
cat <<'EOF' > TMPDIR/risk-review.json
|
||||
{"reviewer_role":"risk-reviewer","findings":[{"title":"Add auth integration tests","summary":"Login regressions are hard to catch.","proposal":"Add integration tests for auth flows.","rationale":"This catches regressions earlier.","confidence":"high","tags":["risk","testing"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
orch --db TMPDIR/coord.db --json council start \
|
||||
--run council_blog_tally_001 \
|
||||
--target "Review the current blog architecture."
|
||||
|
||||
THREAD_ID_CR1=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_tally_001' AND task_id = 'CR1' AND attempt_no = 1;")
|
||||
THREAD_ID_CR2=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_tally_001' AND task_id = 'CR2' AND attempt_no = 1;")
|
||||
THREAD_ID_CR3=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_tally_001' AND task_id = 'CR3' AND attempt_no = 1;")
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent architecture-reviewer --thread "$THREAD_ID_CR1"
|
||||
inbox --db TMPDIR/coord.db --json done --agent architecture-reviewer --thread "$THREAD_ID_CR1" --summary "Review complete" --body-file TMPDIR/architecture-review.json
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent implementation-reviewer --thread "$THREAD_ID_CR2"
|
||||
inbox --db TMPDIR/coord.db --json done --agent implementation-reviewer --thread "$THREAD_ID_CR2" --summary "Review complete" --body-file TMPDIR/implementation-review.json
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent risk-reviewer --thread "$THREAD_ID_CR3"
|
||||
inbox --db TMPDIR/coord.db --json done --agent risk-reviewer --thread "$THREAD_ID_CR3" --summary "Review complete" --body-file TMPDIR/risk-review.json
|
||||
|
||||
orch --db TMPDIR/coord.db --json council tally \
|
||||
--run council_blog_tally_001 \
|
||||
--similarity normal
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `council tally` 退出码为 `0`
|
||||
- `tally.data.similarity == "normal"`
|
||||
- `tally.data.counts.majority == 1`
|
||||
- `tally.data.counts.minority == 1`
|
||||
- `tally.data.grouped_recommendations` 长度为 `2`
|
||||
- 第一组 recommendation 的 `bucket == "majority"`
|
||||
- 第一组 recommendation 的 `support_count == 2`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `normal` 模式会优先按归一化意图合并 proposal，而不是逐字面比较
|
||||
- tally 输出不仅返回统计摘要，还返回分组后的 recommendation 明细
|
||||
|
||||
## 补充约束
|
||||
|
||||
- reviewer `done` 消息体必须是结构化 JSON；无效 JSON 或缺失 `reviewer_role`/`proposal` 会让 tally 返回 `invalid_input`
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
# Case: `council-tally-keeps-distinct-proposals-in-strict-mode`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council tally --similarity strict` 不会合并 wording 不同的 proposal，即使它们语义接近，也会保留为独立 recommendation。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 使用隔离的临时目录 `TMPDIR`
|
||||
- 本地可使用 `sqlite3` 从 `task_attempts` 中读取 reviewer thread ID
|
||||
- 已准备好三份 reviewer 输出 JSON；其中 architecture 与 implementation proposal 语义相近但措辞不同
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
cat <<'EOF' > TMPDIR/architecture-review.json
|
||||
{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
cat <<'EOF' > TMPDIR/implementation-review.json
|
||||
{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract API contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"medium","tags":["maintainability"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
cat <<'EOF' > TMPDIR/risk-review.json
|
||||
{"reviewer_role":"risk-reviewer","findings":[{"title":"Add auth integration tests","summary":"Login regressions are hard to catch.","proposal":"Add integration tests for auth flows.","rationale":"This catches regressions earlier.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}}]}
|
||||
EOF
|
||||
|
||||
orch --db TMPDIR/coord.db --json council start \
|
||||
--run council_blog_tally_002 \
|
||||
--target "Review the current blog architecture."
|
||||
|
||||
THREAD_ID_CR1=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_tally_002' AND task_id = 'CR1' AND attempt_no = 1;")
|
||||
THREAD_ID_CR2=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_tally_002' AND task_id = 'CR2' AND attempt_no = 1;")
|
||||
THREAD_ID_CR3=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_tally_002' AND task_id = 'CR3' AND attempt_no = 1;")
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent architecture-reviewer --thread "$THREAD_ID_CR1"
|
||||
inbox --db TMPDIR/coord.db --json done --agent architecture-reviewer --thread "$THREAD_ID_CR1" --summary "Review complete" --body-file TMPDIR/architecture-review.json
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent implementation-reviewer --thread "$THREAD_ID_CR2"
|
||||
inbox --db TMPDIR/coord.db --json done --agent implementation-reviewer --thread "$THREAD_ID_CR2" --summary "Review complete" --body-file TMPDIR/implementation-review.json
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent risk-reviewer --thread "$THREAD_ID_CR3"
|
||||
inbox --db TMPDIR/coord.db --json done --agent risk-reviewer --thread "$THREAD_ID_CR3" --summary "Review complete" --body-file TMPDIR/risk-review.json
|
||||
|
||||
orch --db TMPDIR/coord.db --json council tally \
|
||||
--run council_blog_tally_002 \
|
||||
--similarity strict
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `council tally` 退出码为 `0`
|
||||
- `tally.data.similarity == "strict"`
|
||||
- `tally.data.counts.minority == 3`
|
||||
- `tally.data.grouped_recommendations` 长度为 `3`
|
||||
- 三组 recommendation 都应落入 `minority`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `strict` 模式的目标是保留 proposal 的字面差异，而不是宽松合并
|
||||
- 当没有 proposal 被合并时，support count 会退化成单 reviewer 支持
|
||||
@@ -1,7 +1,8 @@
|
||||
# Orch `council wait` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `council-wait-wakes-when-all-reviewers-complete` | [council-wait-wakes-when-all-reviewers-complete.md](./council-wait-wakes-when-all-reviewers-complete.md) | wakes successfully once all three reviewer threads reach terminal success |
|
||||
| `council-wait-times-out-when-reviewers-incomplete` | [council-wait-times-out-when-reviewers-incomplete.md](./council-wait-times-out-when-reviewers-incomplete.md) | returns a stable timeout result while reviewer work remains incomplete |
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
# Case: `council-wait-times-out-when-reviewers-incomplete`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council wait` 在 reviewer 尚未全部完成时返回稳定的超时结果，而不是误判为成功唤醒。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 使用隔离的临时目录 `TMPDIR`
|
||||
- 目标数据库 `TMPDIR/coord.db` 尚不存在
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json council start \
|
||||
--run council_blog_wait_002 \
|
||||
--target "Review the current blog architecture."
|
||||
|
||||
orch --db TMPDIR/coord.db --json council wait \
|
||||
--run council_blog_wait_002 \
|
||||
--timeout-seconds 1
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `council wait` 退出码为 `0`
|
||||
- `wait.data.woke == false`
|
||||
- `wait.data.all_complete == false`
|
||||
- `wait.data.reviewers` 长度为 `3`
|
||||
- 返回 reviewer 状态集合时，不要求每个 reviewer 已完成
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `council wait` 的超时结果是显式的“未唤醒”状态，而不是错误退出
|
||||
- leader 可以基于同一个返回结构同时处理唤醒与超时两种路径
|
||||
@@ -0,0 +1,53 @@
|
||||
# Case: `council-wait-wakes-when-all-reviewers-complete`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `council wait` 在三位 reviewer 都完成后会被唤醒，并返回完整 reviewer 状态集合。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 使用隔离的临时目录 `TMPDIR`
|
||||
- 已通过 `council start` 创建 run `council_blog_wait_001`
|
||||
- 本地可使用 `sqlite3` 从 `task_attempts` 中读取 reviewer thread ID，用于构造 `inbox` 完成态
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json council start \
|
||||
--run council_blog_wait_001 \
|
||||
--target "Review the current blog architecture."
|
||||
|
||||
THREAD_ID_CR1=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_wait_001' AND task_id = 'CR1' AND attempt_no = 1;")
|
||||
THREAD_ID_CR2=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_wait_001' AND task_id = 'CR2' AND attempt_no = 1;")
|
||||
THREAD_ID_CR3=$(sqlite3 TMPDIR/coord.db "SELECT thread_id FROM task_attempts WHERE run_id = 'council_blog_wait_001' AND task_id = 'CR3' AND attempt_no = 1;")
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent architecture-reviewer --thread "$THREAD_ID_CR1"
|
||||
inbox --db TMPDIR/coord.db --json done --agent architecture-reviewer --thread "$THREAD_ID_CR1" --summary "Review complete"
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent implementation-reviewer --thread "$THREAD_ID_CR2"
|
||||
inbox --db TMPDIR/coord.db --json done --agent implementation-reviewer --thread "$THREAD_ID_CR2" --summary "Review complete"
|
||||
|
||||
inbox --db TMPDIR/coord.db --json claim --agent risk-reviewer --thread "$THREAD_ID_CR3"
|
||||
inbox --db TMPDIR/coord.db --json done --agent risk-reviewer --thread "$THREAD_ID_CR3" --summary "Review complete"
|
||||
|
||||
orch --db TMPDIR/coord.db --json council wait \
|
||||
--run council_blog_wait_001 \
|
||||
--timeout-seconds 2
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `council wait` 退出码为 `0`
|
||||
- `wait.data.woke == true`
|
||||
- `wait.data.all_complete == true`
|
||||
- `wait.data.reviewers` 长度为 `3`
|
||||
- 三个 reviewer 的 `status` 都是 `done`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `council wait` 的唤醒条件是“三位 reviewer 全部达到终态成功”
|
||||
- 返回结果不仅告知已唤醒，还会携带完整 reviewer 状态快照，便于 leader 继续执行 tally/report
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 当前手工复现实例需要通过 `task_attempts` 提取 reviewer `thread_id`，因为 `orch` CLI 还不直接暴露 attempt-thread mapping
|
||||
@@ -1,7 +1,7 @@
|
||||
# Orch `dep add` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `dep-add-blocks-dependent-task-until-prerequisite-completes` | [dep-add-blocks-dependent-task-until-prerequisite-completes.md](./dep-add-blocks-dependent-task-until-prerequisite-completes.md) | adds a dependency edge that keeps the dependent task out of the ready set |
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# Case: `dep-add-blocks-dependent-task-until-prerequisite-completes`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `dep add` 会建立依赖边，并让被依赖任务在前置任务完成前保持不可调度。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_002`
|
||||
- run 下已存在任务 `T1` 与 `T2`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_002 --goal "Build dependency-aware workflow"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_002 --task T1 --title "Build backend" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_002 --task T2 --title "Build frontend" --default-to worker-b
|
||||
orch --db TMPDIR/coord.db --json dep add --run run_blog_002 --task T2 --depends-on T1
|
||||
orch --db TMPDIR/coord.db --json ready --run run_blog_002
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `dep add` 退出码为 `0`
|
||||
- `data.dependency.task_id == "T2"`
|
||||
- `data.dependency.depends_on_task_id == "T1"`
|
||||
- 后续 `ready` 只返回 `T1`
|
||||
- `T2` 不出现在 `ready.data.tasks` 中
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `dep add` 会立刻影响 ready 计算结果
|
||||
- 依赖关系属于调度门控，而不是仅供展示的元数据
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `--task` 不能依赖自己；自依赖应返回 `invalid_input`
|
||||
- 重复添加同一条依赖边应返回 `invalid_state`
|
||||
@@ -1,7 +1,13 @@
|
||||
# Orch `dispatch` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `dispatch-creates-attempt-and-thread-for-ready-task` | [dispatch-creates-attempt-and-thread-for-ready-task.md](./dispatch-creates-attempt-and-thread-for-ready-task.md) | dispatches a ready task into a new attempt, inbox thread, and initial task message |
|
||||
| `dispatch-rejects-non-ready-task` | [dispatch-rejects-non-ready-task.md](./dispatch-rejects-non-ready-task.md) | rejects dispatch when the task is still gated by dependencies |
|
||||
| `dispatch-creates-strict-worktree` | [dispatch-creates-strict-worktree.md](./dispatch-creates-strict-worktree.md) | provisions a strict worktree and writes workspace metadata into the attempt and payload |
|
||||
| `dispatch-rejects-dirty-repo-without-base-ref` | [dispatch-rejects-dirty-repo-without-base-ref.md](./dispatch-rejects-dirty-repo-without-base-ref.md) | blocks strict worktree dispatch from a dirty repository without an explicit base ref |
|
||||
| `dispatch-allows-explicit-base-ref-on-dirty-repo` | [dispatch-allows-explicit-base-ref-on-dirty-repo.md](./dispatch-allows-explicit-base-ref-on-dirty-repo.md) | accepts dirty repository state when `--base-ref` resolves to a concrete commit |
|
||||
| `dispatch-auto-enables-worktree-for-code-like-task` | [dispatch-auto-enables-worktree-for-code-like-task.md](./dispatch-auto-enables-worktree-for-code-like-task.md) | auto-enables worktree mode for code-like tasks when no explicit worktree flags are supplied |
|
||||
| `dispatch-skips-auto-worktree-for-non-code-task` | [dispatch-skips-auto-worktree-for-non-code-task.md](./dispatch-skips-auto-worktree-for-non-code-task.md) | keeps clearly non-code tasks on the normal non-worktree dispatch path |
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
# Case: `dispatch-allows-explicit-base-ref-on-dirty-repo`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 strict worktree dispatch 在仓库已变脏时，只要显式给出可解析的 `--base-ref`，仍可继续创建 attempt。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- `TMPDIR/repo` 是一个 Git 仓库
|
||||
- 仓库工作区存在未提交变更
|
||||
- `HEAD` 仍指向合法 commit
|
||||
- 已存在 run `run_blog_worktree_003` 与任务 `T1`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_worktree_003 --goal "Validate explicit base ref on dirty repo"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_worktree_003 --task T1 --title "Implement backend" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_worktree_003 --task T1 --repo-path TMPDIR/repo --workspace-root .orch/worktrees --strict-worktree --base-ref HEAD
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `dispatch` 退出码为 `0`
|
||||
- `data.attempt.base_ref == "HEAD"`
|
||||
- `data.attempt.base_commit` 等于 dirty 之前当前可解析的 `HEAD` commit
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `--base-ref` 是 dirty repo strict dispatch 的显式解锁条件
|
||||
- worktree 基线来自 commit，而不是当前未提交工作区内容
|
||||
@@ -0,0 +1,34 @@
|
||||
# Case: `dispatch-auto-enables-worktree-for-code-like-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `dispatch` 在未显式传 worktree flags 时，会对 code-like 任务自动启用 worktree 流程。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- `TMPDIR/repo` 是一个干净的 Git 仓库
|
||||
- 已存在 code-like 任务 `T1`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_auto_worktree_001 --goal "Validate auto worktree detection"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_auto_worktree_001 --task T1 --title "Implement backend API" --default-to backend-worker
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_auto_worktree_001 --task T1 --repo-path TMPDIR/repo
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `dispatch` 退出码为 `0`
|
||||
- `data.attempt.worktree_path` 为非空
|
||||
- `data.attempt.workspace_status == "created"`
|
||||
- 返回的 worktree 路径在磁盘上存在
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `dispatch` 存在自动 worktree 推断逻辑，不要求 leader 每次显式写 `--strict-worktree`
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 当前推断主要依赖任务角色与 acceptance JSON 的 code-like 标记
|
||||
- 未指定 `--workspace-root` 时，自动 worktree 模式默认写到仓库下的 `.orch/worktrees`
|
||||
@@ -0,0 +1,38 @@
|
||||
# Case: `dispatch-creates-attempt-and-thread-for-ready-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `dispatch` 在任务已 `ready` 时，会创建 attempt、映射 inbox thread，并写入首条任务消息。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_001`
|
||||
- 已存在无依赖任务 `T1`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_001 --goal "Build blog MVP" --summary "Public blog plus admin CRUD"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_001 --task T1 --title "Implement retry policy" --summary "Add retry policy to HTTP client" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_001 --task T1 --body "Implement retry handling for the HTTP client."
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `dispatch` 退出码为 `0`
|
||||
- `data.task.status == "dispatched"`
|
||||
- `data.attempt.attempt_no == 1`
|
||||
- 返回 `data.attempt.thread_id`
|
||||
- `data.attempt.assigned_to == "worker-a"`
|
||||
- `data.thread.thread_id` 与 `data.attempt.thread_id` 一致
|
||||
- `data.message.kind == "task"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `dispatch` 是把调度意图物化为一次 attempt 和 inbox thread 的命令
|
||||
- 任务进入 `dispatched` 后，leader 可以用 thread 映射等待 worker 侧进展
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 未显式传 `--to` 时，会回退使用任务的 `default_to`
|
||||
- `--body` 与 `--body-file` 互斥；不可读的 `--body-file` 应返回 `invalid_input`
|
||||
@@ -0,0 +1,39 @@
|
||||
# Case: `dispatch-creates-strict-worktree`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证显式 `--strict-worktree` dispatch 会创建隔离 worktree，并把 workspace 元数据持久化到 attempt 与任务 payload 中。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- `TMPDIR/repo` 是一个干净的 Git 仓库
|
||||
- 仓库内已存在至少一个已提交文件
|
||||
- 已存在 run `run_blog_worktree_001` 与任务 `T1`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_worktree_001 --goal "Validate strict worktree dispatch"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_worktree_001 --task T1 --title "Implement backend" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_worktree_001 --task T1 --repo-path TMPDIR/repo --workspace-root .orch/worktrees --strict-worktree --body "Implement inside isolated worktree."
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `dispatch` 退出码为 `0`
|
||||
- `data.attempt.base_ref == "HEAD"`
|
||||
- `data.attempt.base_commit` 等于仓库当前 `HEAD` commit
|
||||
- `data.attempt.branch_name == "orch/run-blog-worktree-001/T1/attempt-1"`
|
||||
- 返回非空 `data.attempt.worktree_path`
|
||||
- `data.attempt.workspace_status == "created"`
|
||||
- `data.message.payload_json.worktree_path` 与 attempt 中的路径一致
|
||||
|
||||
## 断言结论
|
||||
|
||||
- strict worktree dispatch 会创建真正的隔离工作目录，而不是只记录一组字符串元数据
|
||||
- worker 读取任务 payload 时可以拿到同一份 worktree 路径
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 未显式传 `--base-ref` 且仓库干净时，会默认回退到 `HEAD`
|
||||
- `--workspace-root` 为相对路径时，会相对于仓库根目录解析
|
||||
@@ -0,0 +1,30 @@
|
||||
# Case: `dispatch-rejects-dirty-repo-without-base-ref`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 strict worktree dispatch 在仓库存在未提交修改且未显式指定 `--base-ref` 时，会拒绝继续执行。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- `TMPDIR/repo` 是一个 Git 仓库
|
||||
- 仓库工作区存在未提交变更
|
||||
- 已存在 run `run_blog_worktree_002` 与任务 `T1`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_worktree_002 --goal "Validate dirty repo rejection"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_worktree_002 --task T1 --title "Implement backend" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_worktree_002 --task T1 --repo-path TMPDIR/repo --workspace-root .orch/worktrees --strict-worktree
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `dispatch` 退出码为 `30`
|
||||
- JSON 错误码为 `invalid_state`
|
||||
- `.orch/worktrees/run_blog_worktree_002/T1/attempt-1` 不应被创建
|
||||
|
||||
## 断言结论
|
||||
|
||||
- strict 模式不会隐式吞掉未提交工作区状态
|
||||
- 当 leader 依赖脏工作区时，必须显式给出 `--base-ref`
|
||||
@@ -0,0 +1,34 @@
|
||||
# Case: `dispatch-rejects-non-ready-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `dispatch` 在任务仍被依赖阻塞时会返回稳定的 `invalid_state` 契约，而不是偷偷创建 attempt。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_003`
|
||||
- 任务 `T2` 依赖 `T1`
|
||||
- `T1` 尚未完成
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_003 --goal "Validate ready gating"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_003 --task T1 --title "Backend"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_003 --task T2 --title "Frontend"
|
||||
orch --db TMPDIR/coord.db --json dep add --run run_blog_003 --task T2 --depends-on T1
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_003 --task T2
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `dispatch` 退出码为 `30`
|
||||
- JSON 错误码为 `invalid_state`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- 依赖未满足时，`dispatch` 失败得很早，并且不会越过 ready gate
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 该错误是调度状态错误，不是 `not_found`
|
||||
@@ -0,0 +1,35 @@
|
||||
# Case: `dispatch-skips-auto-worktree-for-non-code-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `dispatch` 在未显式传 worktree flags 时，不会把明显非代码任务错误地推进到 worktree 执行路径。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- `TMPDIR/repo` 是一个干净的 Git 仓库
|
||||
- 已存在 run `run_blog_auto_worktree_002`
|
||||
- 已存在非 code-like 任务 `T1`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_auto_worktree_002 --goal "Validate non-code dispatch fallback"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_auto_worktree_002 --task T1 --title "Review QA findings" --summary "Summarize test failures and next steps" --default-to qa-worker
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_auto_worktree_002 --task T1 --repo-path TMPDIR/repo
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `dispatch` 退出码为 `0`
|
||||
- `data.attempt.worktree_path == ""`
|
||||
- `data.attempt.workspace_status == ""`
|
||||
- 仍会正常返回 `thread_id` 与首条任务消息
|
||||
|
||||
## 断言结论
|
||||
|
||||
- 自动 worktree 推断不是“见仓库就建 worktree”
|
||||
- 非代码任务仍走标准 dispatch 路径，不会平白引入分支和工作目录
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 当前非代码判断通常来自任务标题、摘要、角色和 acceptance 信息都缺少 code-like 信号
|
||||
@@ -1,7 +1,8 @@
|
||||
# Orch `ready` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `ready-lists-only-eligible-tasks` | [ready-lists-only-eligible-tasks.md](./ready-lists-only-eligible-tasks.md) | returns only dependency-cleared tasks in priority-aware ready order |
|
||||
| `ready-orders-by-priority-and-respects-limit` | [ready-orders-by-priority-and-respects-limit.md](./ready-orders-by-priority-and-respects-limit.md) | sorts ready tasks by priority and applies `--limit` after ordering |
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
# Case: `ready-lists-only-eligible-tasks`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `ready` 只返回当前真正可调度的任务，而不会把仍受依赖阻塞的任务混入结果。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_002`
|
||||
- `T1` 与 `T2` 已创建
|
||||
- `T2` 已通过 `dep add` 依赖 `T1`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_002 --goal "Build dependency-aware workflow"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_002 --task T1 --title "Build backend" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_002 --task T2 --title "Build frontend" --default-to worker-b
|
||||
orch --db TMPDIR/coord.db --json dep add --run run_blog_002 --task T2 --depends-on T1
|
||||
orch --db TMPDIR/coord.db --json ready --run run_blog_002
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `ready` 退出码为 `0`
|
||||
- `data.tasks` 长度为 `1`
|
||||
- 唯一返回项是 `T1`
|
||||
- 返回任务状态为 `ready`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `ready` 是经过依赖和状态过滤后的结果，不是“所有未完成任务”的简单列表
|
||||
- 新 agent 可以依赖该命令决定可立即 dispatch 的工作
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 未显式传 `--limit` 时，默认上限是 `20`
|
||||
- `--run` 指向不存在的 run 时，应返回 `not_found`
|
||||
@@ -0,0 +1,39 @@
|
||||
# Case: `ready-orders-by-priority-and-respects-limit`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `ready` 会先按优先级排序可调度任务，再应用 `--limit` 截断结果，而不是按创建顺序直接裁剪。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_005`
|
||||
- 该 run 下有至少三个无依赖且处于 `ready` 的任务
|
||||
- 三个任务优先级分别为 `high`、`normal`、`low`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_005 --goal "Validate ready ordering"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_005 --task T1 --title "Low priority task" --priority low
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_005 --task T2 --title "Normal priority task" --priority normal
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_005 --task T3 --title "High priority task" --priority high
|
||||
orch --db TMPDIR/coord.db --json ready --run run_blog_005 --limit 2
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `ready` 退出码为 `0`
|
||||
- `data.tasks` 长度为 `2`
|
||||
- 第一个返回项是高优先级任务 `T3`
|
||||
- 第二个返回项是普通优先级任务 `T2`
|
||||
- 低优先级任务 `T1` 不出现在本次结果中
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `ready` 的用户可见顺序是 `high -> normal -> low`
|
||||
- `--limit` 的截断发生在优先级排序之后，因此 leader 可以依赖该命令优先看到更重要的可调度任务
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 当多个 ready 任务优先级相同时，当前实现会按创建时间升序稳定返回
|
||||
- 未显式传 `--limit` 时，默认上限仍是 `20`
|
||||
@@ -1,7 +1,7 @@
|
||||
# Orch `reassign` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `reassign-cancels-old-thread-and-dispatches-new-attempt` | [reassign-cancels-old-thread-and-dispatches-new-attempt.md](./reassign-cancels-old-thread-and-dispatches-new-attempt.md) | cancels the old blocked thread and creates a new attempt for another worker |
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
# Case: `reassign-cancels-old-thread-and-dispatches-new-attempt`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `reassign` 会取消旧的阻塞线程，并为新 worker 创建新的尝试与线程。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已创建运行 `run_blog_reassign_001`
|
||||
- 已创建任务 `T1`
|
||||
- `T1` 已通过严格 worktree 模式完成首次 `dispatch`
|
||||
- `worker-a` 已 `claim` 首次尝试线程，并通过 `inbox update --status blocked` 写入问题
|
||||
- 最近一次 `reconcile` 已执行，使任务进入 `blocked`
|
||||
- 已知旧线程为 `OLD_THREAD_ID`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json reassign --run run_blog_reassign_001 --task T1 --to worker-b --reason "Try another worker with clearer ownership."
|
||||
inbox --db TMPDIR/coord.db --json show --thread OLD_THREAD_ID
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `reassign` 退出码为 `0`
|
||||
- `reassign.data.attempt.assigned_to == "worker-b"`
|
||||
- `reassign.data.attempt.attempt_no == 2`
|
||||
- `reassign.data.attempt.thread_id != OLD_THREAD_ID`
|
||||
- `show.data.thread.status == "cancelled"`，指向旧线程
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `reassign` 不是简单修改 `assigned_to` 字段，而是显式终止旧尝试并派生新尝试
|
||||
- 旧线程被取消后，worker 侧不会继续在过期上下文上执行
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `reassign` 只接受 `blocked` 或 `failed` 任务
|
||||
- `--to` 是必填参数；`--reason` 建议始终填写，便于审计和人工排障
|
||||
@@ -1,7 +1,8 @@
|
||||
# Orch `reconcile` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `reconcile-maps-claimed-or-in-progress-thread-to-running` | [reconcile-maps-claimed-or-in-progress-thread-to-running.md](./reconcile-maps-claimed-or-in-progress-thread-to-running.md) | maps worker claim or in-progress inbox state back into a running orch task |
|
||||
| `reconcile-maps-done-or-failed-thread-to-terminal-task-state` | [reconcile-maps-done-or-failed-thread-to-terminal-task-state.md](./reconcile-maps-done-or-failed-thread-to-terminal-task-state.md) | maps terminal inbox states into terminal task states and updates run aggregates |
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# Case: `reconcile-maps-claimed-or-in-progress-thread-to-running`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `reconcile` 会把 worker 侧的 `claim` / `in_progress` 进展同步回 `orch`，将任务推进到 `running`。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_001`
|
||||
- 任务 `T1` 已通过 `dispatch` 创建 attempt 和 thread
|
||||
- worker 已对该 thread 完成 `claim`，并可选地追加 `in_progress` 更新
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_001 --goal "Build blog MVP"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_001 --task T1 --title "Implement retry policy" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_001 --task T1 --body "Implement retry handling for the HTTP client."
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-a --thread THREAD_ID
|
||||
inbox --db TMPDIR/coord.db --json update --agent worker-a --thread THREAD_ID --status in_progress --summary "Implementation started"
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `reconcile` 退出码为 `0`
|
||||
- `data.updated_tasks` 长度为 `1`
|
||||
- 唯一更新任务的 `status == "running"`
|
||||
- `data.run.run_id == "run_blog_001"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `reconcile` 是 leader 侧把 inbox 执行状态投影回 scheduler 状态机的关键同步点
|
||||
- claim 与 in-progress 的 worker 信号不会停留在 inbox 层
|
||||
+34
@@ -0,0 +1,34 @@
|
||||
# Case: `reconcile-maps-done-or-failed-thread-to-terminal-task-state`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `reconcile` 会把 worker 侧 thread 的终态同步到 `orch` 任务，并刷新 run 聚合状态。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run 和已 dispatch 的任务
|
||||
- worker 已对该 thread 完成 `done` 或 `fail`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_001 --goal "Build blog MVP"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_001 --task T1 --title "Implement retry policy" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_001 --task T1 --body "Implement retry handling for the HTTP client."
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-a --thread THREAD_ID
|
||||
inbox --db TMPDIR/coord.db --json done --agent worker-a --thread THREAD_ID --summary "Retry policy implemented" --body "The HTTP client now retries transient failures."
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_001
|
||||
orch --db TMPDIR/coord.db --json status --run run_blog_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `reconcile` 退出码为 `0`
|
||||
- `data.updated_tasks` 包含 `T1`
|
||||
- `T1.status == "done"`；若输入是 `fail`，则应为 `failed`
|
||||
- 后续 `status.data.run.status` 与终态任务聚合结果一致
|
||||
|
||||
## 断言结论
|
||||
|
||||
- 任务终态依赖 `reconcile` 落回 `orch`，而不是由 worker 直接改写 task 表
|
||||
- run 级聚合状态会随终态任务一并刷新
|
||||
@@ -1,7 +1,7 @@
|
||||
# Orch `retry` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `retry-creates-new-attempt-for-failed-task` | [retry-creates-new-attempt-for-failed-task.md](./retry-creates-new-attempt-for-failed-task.md) | creates a successor attempt, thread, and worktree after a failed attempt |
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
# Case: `retry-creates-new-attempt-for-failed-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `retry` 会在失败任务上创建新的尝试记录，而不是复用旧线程或旧 worktree。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已创建运行 `run_blog_retry_001`
|
||||
- 已创建任务 `T1`
|
||||
- `T1` 已通过严格 worktree 模式完成首次 `dispatch`
|
||||
- `worker-a` 已 `claim` 首次尝试线程并通过 `inbox fail` 把线程推进到 `failed`
|
||||
- 最近一次 `reconcile` 已执行，使任务状态同步为 `failed`
|
||||
- 已知首次尝试的线程为 `OLD_THREAD_ID`，worktree 为 `OLD_WORKTREE_PATH`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json retry --run run_blog_retry_001 --task T1 --body "Retry after fixing the failure."
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 退出码为 `0`
|
||||
- `retry.data.task.status == "dispatched"`
|
||||
- `retry.data.attempt.attempt_no == 2`
|
||||
- `retry.data.attempt.thread_id != OLD_THREAD_ID`
|
||||
- `retry.data.attempt.worktree_path != OLD_WORKTREE_PATH`
|
||||
- 新 worktree 路径在文件系统上存在
|
||||
- `retry.data.previous_attempt.attempt_no == 1`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `retry` 会为失败任务生成新的执行尝试，而不是把旧尝试重新打开
|
||||
- 对代码任务而言，重试会分配新的 worktree，避免旧失败环境污染下一次执行
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `--to` 可选；未显式传入时，默认沿用当前任务/尝试的既有分配信息
|
||||
- `retry` 支持 `--body-file`，并遵守与 `--body` 的互斥规则
|
||||
@@ -1,7 +1,7 @@
|
||||
# Orch `run init` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `run-init-creates-new-run` | [run-init-creates-new-run.md](./run-init-creates-new-run.md) | creates a new active run and persists goal and summary |
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# Case: `run-init-creates-new-run`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `run init` 会创建新的 orchestration run，并返回稳定的 run 元数据。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- `TMPDIR/coord.db` 尚不存在或为空路径
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_001 --goal "Build blog MVP" --summary "Public blog plus admin CRUD"
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 命令退出码为 `0`
|
||||
- `data.run.run_id == "run_blog_001"`
|
||||
- `data.run.goal == "Build blog MVP"`
|
||||
- `data.run.summary == "Public blog plus admin CRUD"`
|
||||
- `data.run.status == "active"`
|
||||
- 返回 `created_at` 与 `updated_at`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `run init` 会创建 run 记录，而不是只做内存态初始化
|
||||
- 新建 run 的默认状态是 `active`
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `--run` 与 `--goal` 是必填；缺失任一项都应返回 `invalid_input`
|
||||
- 当同一 `run_id` 已存在时，应返回 `invalid_state`，而不是覆盖旧 run
|
||||
@@ -1,7 +1,7 @@
|
||||
# Orch `run show` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `run-show-returns-run-summary-and-task-counts` | [run-show-returns-run-summary-and-task-counts.md](./run-show-returns-run-summary-and-task-counts.md) | returns aggregate run metadata and task-count summary without listing task rows |
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
# Case: `run-show-returns-run-summary-and-task-counts`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `run show` 会返回 run 元数据与聚合任务统计，适合作为 leader 端的轻量总览命令。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 `run_blog_001`
|
||||
- 该 run 下至少已有一个任务，以便产生非空 `task_counts`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_001 --goal "Build blog MVP" --summary "Public blog plus admin CRUD"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_001 --task T1 --title "Implement retry policy" --summary "Add retry policy to HTTP client" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json run show --run run_blog_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `run show` 退出码为 `0`
|
||||
- `data.run.run_id == "run_blog_001"`
|
||||
- `data.run.status == "active"`
|
||||
- `data.task_counts.ready >= 1`
|
||||
- 返回值不包含 `tasks` 数组
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `run show` 提供的是聚合视图，而不是完整任务明细
|
||||
- run 级状态和任务计数可以在不调用 `status` 的情况下被读取
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 当 `--run` 指向不存在的 run 时，应返回 `not_found`
|
||||
@@ -1,7 +1,7 @@
|
||||
# Orch `status` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `status-returns-run-summary-and-task-list` | [status-returns-run-summary-and-task-list.md](./status-returns-run-summary-and-task-list.md) | returns aggregate run status plus the per-task status list |
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# Case: `status-returns-run-summary-and-task-list`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `status` 会返回 run 聚合视图以及任务明细列表，是 leader 端的完整状态检查入口。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_001`
|
||||
- 任务 `T1` 已经过完整的 dispatch -> worker done -> reconcile 流程
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_001 --goal "Build blog MVP"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_001 --task T1 --title "Implement retry policy" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_001 --task T1 --body "Implement retry handling for the HTTP client."
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-a --thread THREAD_ID
|
||||
inbox --db TMPDIR/coord.db --json done --agent worker-a --thread THREAD_ID --summary "Retry policy implemented" --body "The HTTP client now retries transient failures."
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_001
|
||||
orch --db TMPDIR/coord.db --json status --run run_blog_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `status` 退出码为 `0`
|
||||
- `data.run.run_id == "run_blog_001"`
|
||||
- `data.run.status == "done"`
|
||||
- 返回 `data.task_counts`
|
||||
- 返回 `data.tasks` 数组
|
||||
- `data.tasks[0].task_id == "T1"`
|
||||
- `data.tasks[0].status == "done"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `status` 比 `run show` 更完整，适合做 run 级收口检查
|
||||
- 任务清单与 run 聚合状态应保持一致，不应出现 run 已完成而任务仍显示旧状态的结果
|
||||
@@ -1,7 +1,9 @@
|
||||
# Orch `task add` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `task-add-creates-ready-root-task` | [task-add-creates-ready-root-task.md](./task-add-creates-ready-root-task.md) | creates a dependency-free task that becomes ready immediately |
|
||||
| `task-add-rejects-invalid-acceptance-json` | [task-add-rejects-invalid-acceptance-json.md](./task-add-rejects-invalid-acceptance-json.md) | rejects malformed `--acceptance-json` with `invalid_input` |
|
||||
| `task-add-rejects-invalid-priority` | [task-add-rejects-invalid-priority.md](./task-add-rejects-invalid-priority.md) | rejects priorities outside `low|normal|high` |
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
# Case: `task-add-creates-ready-root-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `task add` 为无依赖任务创建记录时，会在同一事务里把任务推进为 `ready`。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_001`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_001 --goal "Build blog MVP"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_001 --task T1 --title "Implement retry policy" --summary "Add retry policy to HTTP client" --default-to worker-a
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `task add` 退出码为 `0`
|
||||
- `data.task.task_id == "T1"`
|
||||
- `data.task.title == "Implement retry policy"`
|
||||
- `data.task.status == "ready"`
|
||||
- `data.task.default_to == "worker-a"`
|
||||
- `data.task.priority == "normal"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `task add` 不只是插入 `planned` 任务；对无依赖任务会立即刷新为 `ready`
|
||||
- 默认优先级会稳定回退到 `normal`
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `--run`、`--task`、`--title` 是必填
|
||||
- 未显式传 `--acceptance-json` 时，会回退为合法 JSON 默认值，而不是空字符串
|
||||
- 同一 run 下重复的 `task_id` 应返回 `invalid_state`
|
||||
@@ -0,0 +1,27 @@
|
||||
# Case: `task-add-rejects-invalid-acceptance-json`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `task add` 会拒绝格式非法的 `--acceptance-json`，并返回稳定的 `invalid_input` 错误契约。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_003`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_003 --goal "Validate task add input guards"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_003 --task T1 --title "Implement retry policy" --acceptance-json '{"done":true'
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `task add` 退出码为 `30`
|
||||
- JSON 错误码为 `invalid_input`
|
||||
- 错误消息指出 `acceptance-json` 必须是合法 JSON
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `task add` 不会把格式错误的 acceptance 条件静默写入数据库
|
||||
- `--acceptance-json` 的校验属于稳定的 CLI 输入契约，而不是存储层偶然失败
|
||||
@@ -0,0 +1,27 @@
|
||||
# Case: `task-add-rejects-invalid-priority`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `task add` 只接受 `low|normal|high` 三种优先级值，并在其他输入下返回稳定错误。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_004`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_004 --goal "Validate task priority input"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_004 --task T1 --title "Implement retry policy" --priority urgent
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `task add` 退出码为 `30`
|
||||
- JSON 错误码为 `invalid_input`
|
||||
- 错误消息指出 `priority` 必须是 `low`、`normal` 或 `high`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `task add` 的优先级枚举是明确而稳定的 CLI 契约
|
||||
- 非法优先级会在任务写入前被拒绝，而不是退回到默认值或被静默接受
|
||||
@@ -1,7 +1,8 @@
|
||||
# Orch `wait` Test Plan Index
|
||||
|
||||
## Status
|
||||
## Case Files
|
||||
|
||||
No command case files are authored yet.
|
||||
|
||||
Use [../ROADMAP.md](../ROADMAP.md) for planned case slugs and document progress.
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `wait-wakes-on-matching-run-event` | [wait-wakes-on-matching-run-event.md](./wait-wakes-on-matching-run-event.md) | wakes on a later matching task event and returns that event payload |
|
||||
| `wait-times-out-without-matching-event` | [wait-times-out-without-matching-event.md](./wait-times-out-without-matching-event.md) | returns a stable timeout payload when no later matching event appears |
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
# Case: `wait-times-out-without-matching-event`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `wait` 在没有后续匹配事件时返回稳定的超时结果，而不是把超时视为命令失败。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 空数据库已初始化
|
||||
- 已创建运行 `run_blog_wait_002`
|
||||
- 当前没有会产生 `task_done` 的后续事件
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json wait --run run_blog_wait_002 --for task_done --after-event 0 --timeout-seconds 1
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 退出码为 `0`
|
||||
- `wait.data.woke == false`
|
||||
- `wait.data.next_event_id == 0`
|
||||
- `wait.data.events` 为空或缺省
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `wait` 的超时是可消费的正常结果，不是错误态
|
||||
- leader 可以基于 `woke=false` 决定继续轮询、切换过滤条件，或退出当前控制循环
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 该用例强调超时契约，不要求系统中存在任何任务
|
||||
@@ -0,0 +1,40 @@
|
||||
# Case: `wait-wakes-on-matching-run-event`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `wait` 能在后续匹配事件出现时被唤醒，并返回稳定的事件载荷。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 空数据库已初始化
|
||||
- 已创建运行 `run_blog_wait_001`
|
||||
- 已添加任务 `T1` 并完成一次 `dispatch`
|
||||
- 已知当前尝试线程为 `THREAD_ID`
|
||||
- `wait` 在工作线程写入阻塞状态前启动
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json wait --run run_blog_wait_001 --for task_blocked --after-event 0 --timeout-seconds 2
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-a --thread THREAD_ID
|
||||
inbox --db TMPDIR/coord.db --json update --agent worker-a --thread THREAD_ID --status blocked --summary "Need logging decision" --payload-json '{"question":"stdout or stderr?"}'
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `wait` 退出码为 `0`
|
||||
- `wait.data.woke == true`
|
||||
- `wait.data.events` 长度为 `1`
|
||||
- 唯一事件的 `type == "task_blocked"`
|
||||
- 事件 `summary == "Need logging decision"`
|
||||
- 事件 `payload.question == "stdout or stderr?"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `wait` 不是简单睡眠，而是面向 run 事件流的阻塞读取接口
|
||||
- `task_blocked` 事件会把 worker 提问摘要和结构化 payload 暴露给 leader
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `--for` 支持逗号分隔的事件类型列表；该用例验证的是单事件过滤
|
||||
- `wait` 返回成功时也会给出 `next_event_id`，便于后续增量等待
|
||||
@@ -10,13 +10,166 @@ All examples assume:
|
||||
- `orch --db TMPDIR/coord.db --json` is used consistently
|
||||
- assertions follow the shared rules in [../_shared/README.md](../_shared/README.md)
|
||||
|
||||
## Current Status
|
||||
## case: run-dispatch-reconcile-status-happy-path
|
||||
|
||||
No workflow case documents are authored yet.
|
||||
### 用例意义
|
||||
|
||||
Planned first workflow cases live in [../ROADMAP.md](../ROADMAP.md), starting with:
|
||||
验证 `orch` 的主干领导者流程可用：创建 run、加入 task、查看 ready、dispatch、通过 `inbox` 推进 worker 状态、reconcile，再用 `status` 看到最终完成态。
|
||||
|
||||
- `run-dispatch-reconcile-status-happy-path`
|
||||
- `dependency-blocked-answer-resume-flow`
|
||||
- `strict-worktree-dispatch-to-cleanup`
|
||||
- `council-review-end-to-end`
|
||||
### 前置条件
|
||||
|
||||
- 空数据库路径 `TMPDIR/coord.db`
|
||||
- 执行者为 `worker-a`
|
||||
|
||||
### 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_001 --goal "Build blog MVP" --summary "Public blog plus admin CRUD"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_001 --task T1 --title "Implement retry policy" --summary "Add retry policy to HTTP client" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json ready --run run_blog_001
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_001 --task T1 --body "Implement retry handling for the HTTP client."
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-a --thread THREAD_ID
|
||||
inbox --db TMPDIR/coord.db --json update --agent worker-a --thread THREAD_ID --status in_progress --summary "Implementation started"
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_001
|
||||
inbox --db TMPDIR/coord.db --json done --agent worker-a --thread THREAD_ID --summary "Retry policy implemented" --body "The HTTP client now retries transient failures."
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_001
|
||||
orch --db TMPDIR/coord.db --json status --run run_blog_001
|
||||
```
|
||||
|
||||
### 预期输出
|
||||
|
||||
- `run init` 成功创建 `run_blog_001`
|
||||
- `task add` 返回的新 task 初始状态为 `ready`
|
||||
- `ready` 只返回 `T1`
|
||||
- `dispatch` 创建 attempt 与 inbox thread，并将 task 推进到 `dispatched`
|
||||
- 第一次 `reconcile` 后 task 状态变为 `running`
|
||||
- 第二次 `reconcile` 后 task 状态变为 `done`
|
||||
- `status` 返回 `run.status == "done"`
|
||||
|
||||
### 断言结论
|
||||
|
||||
- `orch` 的主干 happy path 不是单命令行为，而是 `orch` 与 `inbox` 共同完成的闭环
|
||||
- `reconcile` 是把 worker-side 线程状态折叠回 leader-side task 状态的关键步骤
|
||||
|
||||
## case: dependency-blocked-answer-resume-flow
|
||||
|
||||
### 用例意义
|
||||
|
||||
验证依赖门控、blocked 列表、`answer` 反馈以及最终恢复到完成态的完整交互链路。
|
||||
|
||||
### 前置条件
|
||||
|
||||
- 空数据库路径 `TMPDIR/coord.db`
|
||||
- `worker-a` 负责 `T1`
|
||||
- `worker-b` 负责 `T2`
|
||||
|
||||
### 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_002 --goal "Build dependency-aware workflow"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_002 --task T1 --title "Build backend" --summary "Implement backend APIs" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_002 --task T2 --title "Build frontend" --summary "Implement frontend flows" --default-to worker-b
|
||||
orch --db TMPDIR/coord.db --json dep add --run run_blog_002 --task T2 --depends-on T1
|
||||
orch --db TMPDIR/coord.db --json ready --run run_blog_002
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_002 --task T1
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-a --thread THREAD_BACKEND
|
||||
inbox --db TMPDIR/coord.db --json done --agent worker-a --thread THREAD_BACKEND --summary "Backend complete"
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_002
|
||||
orch --db TMPDIR/coord.db --json ready --run run_blog_002
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_002 --task T2
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-b --thread THREAD_FRONTEND
|
||||
inbox --db TMPDIR/coord.db --json update --agent worker-b --thread THREAD_FRONTEND --status blocked --summary "Need logging decision" --payload-json '{"question":"stdout or stderr?"}'
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_002
|
||||
orch --db TMPDIR/coord.db --json blocked --run run_blog_002
|
||||
orch --db TMPDIR/coord.db --json answer --run run_blog_002 --task T2 --body "Use stdout for MVP."
|
||||
inbox --db TMPDIR/coord.db --json update --agent worker-b --thread THREAD_FRONTEND --status in_progress --summary "Decision applied"
|
||||
inbox --db TMPDIR/coord.db --json done --agent worker-b --thread THREAD_FRONTEND --summary "Frontend complete"
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_002
|
||||
orch --db TMPDIR/coord.db --json status --run run_blog_002
|
||||
```
|
||||
|
||||
### 预期输出
|
||||
|
||||
- 初始 `ready` 仅包含 `T1`
|
||||
- `T1` 完成并 `reconcile` 后，`T2` 才出现在 `ready`
|
||||
- `blocked` 返回 `T2` 与最新 question
|
||||
- `answer` 向活跃 thread 追加一条 `kind=answer` 消息
|
||||
- 最终 `status` 中 run 进入 `done`
|
||||
|
||||
### 断言结论
|
||||
|
||||
- 依赖门控和 blocked-answer 机制在同一个 run 中可以顺序衔接
|
||||
- `answer` 不直接改 task 状态；真正的状态恢复仍依赖 worker 继续推进线程并由 `reconcile` 采集
|
||||
|
||||
## case: strict-worktree-dispatch-to-cleanup
|
||||
|
||||
### 用例意义
|
||||
|
||||
验证代码任务的 strict worktree 路径能从 dispatch 一直走到 cleanup，确保隔离工作区既会被创建，也能在完成后被移除。
|
||||
|
||||
### 前置条件
|
||||
|
||||
- `TMPDIR/repo` 是一个已提交初始内容的 Git 仓库
|
||||
- `worker-a` 负责代码任务
|
||||
|
||||
### 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_worktree_001 --goal "Validate strict worktree dispatch"
|
||||
orch --db TMPDIR/coord.db --json task add --run run_blog_worktree_001 --task T1 --title "Implement backend" --default-to worker-a
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_blog_worktree_001 --task T1 --repo-path TMPDIR/repo --workspace-root .orch/worktrees --strict-worktree --body "Implement inside isolated worktree."
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-a --thread THREAD_ID
|
||||
inbox --db TMPDIR/coord.db --json done --agent worker-a --thread THREAD_ID --summary "Backend complete"
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_blog_worktree_001
|
||||
orch --db TMPDIR/coord.db --json cleanup --run run_blog_worktree_001 --task T1 --attempt 1
|
||||
```
|
||||
|
||||
### 预期输出
|
||||
|
||||
- `dispatch` 返回非空的 `attempt.base_ref`、`attempt.base_commit`、`attempt.branch_name`、`attempt.worktree_path`
|
||||
- `attempt.workspace_status == "created"`
|
||||
- `cleanup` 返回被清理的 attempt 记录
|
||||
- 清理后 `worktree_path` 不再存在于文件系统
|
||||
|
||||
### 断言结论
|
||||
|
||||
- strict worktree 不是单次 dispatch 细节，而是完整 attempt 生命周期的一部分
|
||||
- `cleanup` 的目标是已完成或废弃的工作区，不应误删仍在活动中的执行目录
|
||||
|
||||
## case: council-review-end-to-end
|
||||
|
||||
### 用例意义
|
||||
|
||||
验证 `orch council` 高层工作流可从 reviewer dispatch 一直走到 final report，且 grouped recommendations 与最终输出衔接一致。
|
||||
|
||||
### 前置条件
|
||||
|
||||
- 空数据库路径 `TMPDIR/coord.db`
|
||||
- 三个固定 reviewer 分别为 `architecture-reviewer`、`implementation-reviewer`、`risk-reviewer`
|
||||
|
||||
### 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json council start --run council_blog_001 --target "Review the current blog architecture."
|
||||
inbox --db TMPDIR/coord.db --json claim --agent architecture-reviewer --thread THREAD_CR1
|
||||
inbox --db TMPDIR/coord.db --json done --agent architecture-reviewer --thread THREAD_CR1 --summary "Review complete" --body '{"reviewer_role":"architecture-reviewer","findings":[...]}'
|
||||
inbox --db TMPDIR/coord.db --json claim --agent implementation-reviewer --thread THREAD_CR2
|
||||
inbox --db TMPDIR/coord.db --json done --agent implementation-reviewer --thread THREAD_CR2 --summary "Review complete" --body '{"reviewer_role":"implementation-reviewer","findings":[...]}'
|
||||
inbox --db TMPDIR/coord.db --json claim --agent risk-reviewer --thread THREAD_CR3
|
||||
inbox --db TMPDIR/coord.db --json done --agent risk-reviewer --thread THREAD_CR3 --summary "Review complete" --body '{"reviewer_role":"risk-reviewer","findings":[...]}'
|
||||
orch --db TMPDIR/coord.db --json council wait --run council_blog_001 --timeout-seconds 2
|
||||
orch --db TMPDIR/coord.db --json council tally --run council_blog_001 --similarity normal
|
||||
orch --db TMPDIR/coord.db --json council report --run council_blog_001
|
||||
```
|
||||
|
||||
### 预期输出
|
||||
|
||||
- `council start` 创建 3 个 reviewer task 并完成 dispatch
|
||||
- `council wait` 在 3 个 reviewer 全部完成后返回 `all_complete == true`
|
||||
- `council tally` 返回 grouped recommendations，并按 `consensus|majority|minority` 分桶
|
||||
- `council report` 返回默认 `show == ["consensus","majority"]`，并产出 markdown artifact
|
||||
|
||||
### 断言结论
|
||||
|
||||
- council workflow 是建立在 `orch` 调度面之上的高层流程，而不是独立基础设施
|
||||
- final report 依赖已持久化的 grouped recommendations，因此 `tally` 与 `report` 必须在契约上连续
|
||||
|
||||
Reference in New Issue
Block a user