Add spec-aware orch tasks and verification gates
This commit is contained in:
@@ -8,6 +8,7 @@ The design target is a local, file-portable agent coordination stack:
|
||||
|
||||
- `inbox`: durable communication bus
|
||||
- `orch`: task graph and scheduling control plane
|
||||
- spec-aware tasks and verification gates owned by `orch`
|
||||
- worktree-backed task execution for code-writing workers
|
||||
- optional user-facing council review workflow on top of `orch`
|
||||
- shared SQLite database file
|
||||
@@ -25,7 +26,7 @@ If `inbox` is reduced to pure chat storage, the scheduler must reconstruct state
|
||||
## Role Model
|
||||
|
||||
- `user`: talks only to the leader
|
||||
- `leader`: owns the overall goal, task graph, acceptance criteria, and final integration
|
||||
- `leader`: owns the overall goal, task graph, task specs, acceptance criteria, verification policy, and final integration
|
||||
- `worker`: executes one assigned task at a time and reports through `inbox`
|
||||
- `inbox`: durable thread/message/lease/artifact store
|
||||
- `orch`: run/task/dependency/dispatch state machine built on top of `inbox`
|
||||
@@ -45,7 +46,7 @@ If `inbox` is reduced to pure chat storage, the scheduler must reconstruct state
|
||||
Both CLIs should point at the same SQLite file.
|
||||
|
||||
- `inbox` owns communication tables such as threads, messages, leases, and artifacts.
|
||||
- `orch` owns scheduling tables such as runs, tasks, dependencies, and attempts.
|
||||
- `orch` owns scheduling tables such as runs, tasks, dependencies, attempts, task specs, and verification check results.
|
||||
- both layers append to a shared event stream for blocking waits
|
||||
- `orch dispatch` creates or updates `inbox` threads.
|
||||
- `orch reconcile` reads `inbox` state and updates task state.
|
||||
@@ -135,9 +136,11 @@ If packaging later favors a single binary, the same model can be exposed as comm
|
||||
|
||||
- runs
|
||||
- task graph and dependencies
|
||||
- task spec snapshots and task-level policy metadata
|
||||
- ready queue calculation
|
||||
- dispatch decisions
|
||||
- task-attempt worktree allocation
|
||||
- verification gates and check-result aggregation
|
||||
- blocked queue review for the leader
|
||||
- retries, reassignment, and cancellation
|
||||
- mapping task attempts to inbox threads
|
||||
|
||||
+66
-10
@@ -2,7 +2,7 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
`orch` is the leader-facing scheduler and control plane. It owns the run, task graph, dependencies, ready queue, dispatch decisions, retries, and reassignment logic.
|
||||
`orch` is the leader-facing scheduler and control plane. It owns the run, task graph, task specs, dependencies, verification gates, ready queue, dispatch decisions, retries, and reassignment logic.
|
||||
|
||||
`orch` does not replace `inbox`. It uses `inbox` as the durable transport and execution record.
|
||||
|
||||
@@ -20,10 +20,12 @@ In normal operation:
|
||||
|
||||
- creating a run for one user request or project
|
||||
- defining tasks and dependencies
|
||||
- snapshotting task specs and per-task verification policy
|
||||
- calculating which tasks are ready
|
||||
- dispatching ready tasks to workers
|
||||
- tracking attempts and mapping them to inbox threads
|
||||
- allocating attempt worktrees for code tasks
|
||||
- aggregating post-implementation check results into a task verification gate
|
||||
- surfacing blocked tasks to the leader
|
||||
- sending answers back into the active inbox thread
|
||||
- reconciling thread state into task state
|
||||
@@ -71,6 +73,7 @@ See [worktree-execution.md](/home/kurihada/project/ai-workflow-skill/docs/worktr
|
||||
- `dispatched`: an inbox thread exists but the worker has not started yet
|
||||
- `running`: the task has been claimed and is actively executing
|
||||
- `blocked`: the active attempt needs clarification or an external dependency
|
||||
- `verifying`: the worker reported completion, but required verification checks have not all passed yet
|
||||
- `done`: task completed and passed its current acceptance gate
|
||||
- `failed`: task completed unsuccessfully
|
||||
- `cancelled`: task was cancelled and should not continue
|
||||
@@ -82,7 +85,10 @@ Suggested transitions:
|
||||
- `dispatched -> running`
|
||||
- `running -> blocked`
|
||||
- `blocked -> running`
|
||||
- `running -> done`
|
||||
- `running -> verifying` when the worker reports `done` and the task has required checks
|
||||
- `running -> done` when the worker reports `done` and the task has no required checks
|
||||
- `verifying -> done`
|
||||
- `verifying -> failed`
|
||||
- `running -> failed`
|
||||
- `failed -> ready` through explicit retry
|
||||
- `* -> cancelled` by leader action
|
||||
@@ -97,12 +103,13 @@ The normal leader loop is:
|
||||
4. inspect `ready`
|
||||
5. `dispatch` tasks
|
||||
6. arrange or launch a separate worker runtime that consumes the assigned inbox threads
|
||||
7. use `status` for the current operational view; it reconciles first and includes latest attempt and message context
|
||||
8. inspect `blocked`
|
||||
9. answer blocked questions
|
||||
10. if nothing is actionable, call `wait`
|
||||
11. retry or reassign failures when needed
|
||||
12. finish when all required tasks are `done`
|
||||
7. use `status` for the current operational view; it reconciles first and includes latest attempt, message, and gate context
|
||||
8. if a task enters `verifying`, record check results with `verify record` and inspect gate state with `verify status`
|
||||
9. inspect `blocked`
|
||||
10. answer blocked questions
|
||||
11. if nothing is actionable, call `wait`
|
||||
12. retry or reassign failures when needed
|
||||
13. finish when all required tasks are `done`
|
||||
|
||||
The leader should block on `orch wait`, not on ad hoc `sleep`.
|
||||
|
||||
@@ -162,6 +169,13 @@ Suggested flags:
|
||||
- `--default-to AGENT`
|
||||
- `--acceptance-json STRING`
|
||||
- `--priority low|normal|high`
|
||||
- `--spec-file PATH`
|
||||
- `--spec-sha SHA256`
|
||||
- `--check-profile NAME`
|
||||
- `--required-check NAME` repeatable
|
||||
- `--allowed-path PATH` repeatable
|
||||
- `--blocked-path PATH` repeatable
|
||||
- `--metadata-json STRING`
|
||||
|
||||
### `orch dep add`
|
||||
|
||||
@@ -207,7 +221,7 @@ Behavior:
|
||||
- in `code` mode, resolves a committed base revision
|
||||
- in `code` mode, creates a branch and worktree for the attempt
|
||||
- creates or links an `inbox` thread
|
||||
- writes `execution_mode` into the inbox task payload and writes workspace metadata for code tasks into attempt storage and task payload
|
||||
- writes `execution_mode` into the inbox task payload, includes the task spec snapshot and verification policy in the dispatch payload, and writes workspace metadata for code tasks into attempt storage and task payload
|
||||
- moves the task to `dispatched`
|
||||
- does not start a worker runtime on its own
|
||||
|
||||
@@ -235,7 +249,8 @@ Behavior:
|
||||
|
||||
- maps inbox `claimed` or `in_progress` to `running`
|
||||
- maps inbox `blocked` to `blocked`
|
||||
- maps inbox `done` to `done`
|
||||
- maps inbox `done` to `verifying` when the task has required checks
|
||||
- maps inbox `done` to `done` when the task has no required checks
|
||||
- maps inbox `failed` to `failed`
|
||||
|
||||
### `orch blocked`
|
||||
@@ -246,6 +261,47 @@ Suggested flags:
|
||||
|
||||
- `--run RUN_ID`
|
||||
|
||||
### `orch verify record`
|
||||
|
||||
Record or update one verification check result for the latest task attempt.
|
||||
|
||||
Suggested flags:
|
||||
|
||||
- `--run RUN_ID`
|
||||
- `--task TASK_ID`
|
||||
- `--attempt N` optional; defaults to latest
|
||||
- `--check NAME`
|
||||
- `--status passed|failed|skipped`
|
||||
- `--summary TEXT`
|
||||
- `--body TEXT`
|
||||
- `--body-file PATH`
|
||||
- `--metadata-json STRING`
|
||||
- `--recorded-by NAME`
|
||||
|
||||
Behavior:
|
||||
|
||||
- upserts one named check result for the selected attempt
|
||||
- emits a verification-recorded event
|
||||
- recomputes the gate for the task
|
||||
- keeps the task in `verifying` while required checks are still pending
|
||||
- moves the task to `done` when all required checks pass
|
||||
- moves the task to `failed` when one or more required checks fail
|
||||
|
||||
### `orch verify status`
|
||||
|
||||
Show the current verification state for one task.
|
||||
|
||||
Suggested flags:
|
||||
|
||||
- `--run RUN_ID`
|
||||
- `--task TASK_ID`
|
||||
- `--attempt N` optional; defaults to latest
|
||||
|
||||
Behavior:
|
||||
|
||||
- returns the task, selected attempt, task spec snapshot, and current gate state
|
||||
- helps the leader inspect which required checks are still pending or failed
|
||||
|
||||
### `orch wait`
|
||||
|
||||
Block until one or more run-scoped events become available.
|
||||
|
||||
@@ -51,6 +51,7 @@ Unless a case says otherwise:
|
||||
- `_shared/README.md`: reusable fixtures, JSON assertions, exit-code rules, and worktree conventions
|
||||
- `workflows/README.md`: cross-command end-to-end scenarios
|
||||
- per-command folders: one leaf-command directory per implemented `orch` command surface
|
||||
- `verify/`: verification-gate command cases
|
||||
|
||||
## Glossary
|
||||
|
||||
@@ -60,6 +61,8 @@ Unless a case says otherwise:
|
||||
- `attempt`: one execution try for a task
|
||||
- `dispatch`: the act of materializing a task into an inbox thread
|
||||
- `workspace`: the branch and worktree assigned to a code-writing attempt
|
||||
- `verification gate`: the check aggregation state between worker `done` and final task completion
|
||||
- `verifying`: the task state used while required checks are still pending or being recorded
|
||||
- `blocked task`: a task whose active attempt requires clarification or another external decision
|
||||
- `council review`: a higher-level workflow built on top of `orch` that dispatches fixed reviewer roles and tallies recommendations
|
||||
|
||||
|
||||
+28
-10
@@ -18,13 +18,13 @@ It is not a replacement for automated Go tests.
|
||||
|
||||
Snapshot date:
|
||||
|
||||
- `2026-03-20`
|
||||
- `2026-03-23`
|
||||
|
||||
Current state:
|
||||
|
||||
- `orch` CLI is implemented for the current scheduler, explicit execution-mode dispatch, wait, and council review surfaces
|
||||
- automated Go tests now cover every currently documented `orch` command case and workflow case, combining the original integration suite with focused contract tests for run/task/ready/dispatch/blocked/answer/cleanup/status/reconcile/workflow/council-report edges
|
||||
- `status` coverage now also documents the richer leader view: auto-reconcile plus latest attempt, latest message, and blocked-question context
|
||||
- `orch` CLI now covers scheduler control, explicit execution-mode dispatch, verification gates, wait, and council review surfaces
|
||||
- automated Go tests now cover every currently documented `orch` command case and workflow case, combining the original integration suite with focused contract tests for run/task/ready/dispatch/verify/blocked/answer/cleanup/status/reconcile/workflow/council-report edges
|
||||
- `status` coverage now also documents the richer leader view: auto-reconcile plus latest attempt, latest message, blocked-question context, and task gate context
|
||||
- this roadmap now exists under `docs/tests/orch/ROADMAP.md`
|
||||
- all planned global, shared, workflow, command-index, and command-case Markdown documents in the current `orch` test-plan set have been authored
|
||||
- every implemented `orch` leaf-command folder now uses `README.md` as an index plus one Markdown file per planned case
|
||||
@@ -32,10 +32,10 @@ Current state:
|
||||
|
||||
Progress summary for planned test-plan documents, excluding `ROADMAP.md`:
|
||||
|
||||
- planned document files: `65`
|
||||
- authored document files: `65`
|
||||
- planned case slugs in this roadmap: `47`
|
||||
- authored case slugs in this roadmap: `47`
|
||||
- planned document files: `71`
|
||||
- authored document files: `71`
|
||||
- planned case slugs in this roadmap: `52`
|
||||
- authored case slugs in this roadmap: `52`
|
||||
|
||||
## Scope
|
||||
|
||||
@@ -48,6 +48,7 @@ In scope:
|
||||
- `orch ready`
|
||||
- `orch dispatch`
|
||||
- `orch reconcile`
|
||||
- `orch verify`
|
||||
- `orch wait`
|
||||
- `orch blocked`
|
||||
- `orch answer`
|
||||
@@ -150,7 +151,10 @@ The Markdown test-plan set starts at zero, but these automated tests already exi
|
||||
- [command_contracts_core_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_core_test.go) `TestOrchRunShowRejectsMissingRun`
|
||||
- [command_contracts_core_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_core_test.go) `TestOrchTaskAddRejectsInvalidAcceptanceJSON`
|
||||
- [command_contracts_core_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_core_test.go) `TestOrchTaskAddRejectsInvalidPriority`
|
||||
- [command_contracts_core_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_core_test.go#L147) `TestOrchTaskAddSnapshotsSpecAndVerificationPolicy`
|
||||
- [command_contracts_core_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_core_test.go#L203) `TestOrchTaskAddRejectsSpecSHAMismatch`
|
||||
- [command_contracts_core_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_core_test.go) `TestOrchReadyOrdersByPriorityAndRespectsLimit`
|
||||
- [integration_test.go](../../../packages/orch-runtime/internal/cli/orch/integration_test.go#L185) `TestOrchVerificationGateLifecycle`
|
||||
- [command_contracts_edges_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_edges_test.go) `TestOrchAnswerAcceptsPayloadJSONWithoutBody`
|
||||
- [command_contracts_edges_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_edges_test.go) `TestOrchAnswerRejectsEmptyBodyAndPayload`
|
||||
- [command_contracts_edges_test.go](../../../packages/orch-runtime/internal/cli/orch/command_contracts_edges_test.go) `TestOrchCleanupRejectsAttemptWithoutTask`
|
||||
@@ -192,6 +196,8 @@ docs/tests/orch/
|
||||
README.md
|
||||
reconcile/
|
||||
README.md
|
||||
verify/
|
||||
README.md
|
||||
wait/
|
||||
README.md
|
||||
blocked/
|
||||
@@ -233,6 +239,8 @@ docs/tests/orch/
|
||||
| `docs/tests/orch/task-add/task-add-creates-ready-root-task.md` | `task add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-invalid-acceptance-json.md` | `task add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-invalid-priority.md` | `task add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/task-add/task-add-snapshots-spec-and-verification-policy.md` | `task add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-spec-sha-mismatch.md` | `task add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/dep-add/README.md` | `dep add` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/dep-add/dep-add-blocks-dependent-task-until-prerequisite-completes.md` | `dep add` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/ready/README.md` | `ready` command case index | 0 | 0 | done |
|
||||
@@ -249,6 +257,10 @@ docs/tests/orch/
|
||||
| `docs/tests/orch/reconcile/README.md` | `reconcile` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-claimed-or-in-progress-thread-to-running.md` | `reconcile` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-done-or-failed-thread-to-terminal-task-state.md` | `reconcile` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-done-thread-to-verifying-when-task-has-required-checks.md` | `reconcile` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/verify/README.md` | `verify` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/verify/verify-status-returns-spec-and-gate-for-task.md` | `verify` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/verify/verify-record-updates-gate-and-marks-task-done-when-required-checks-pass.md` | `verify` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/wait/README.md` | `wait` command case index | 0 | 0 | done |
|
||||
| `docs/tests/orch/wait/wait-wakes-on-matching-run-event.md` | `wait` command case | 1 | 1 | done |
|
||||
| `docs/tests/orch/wait/wait-times-out-without-matching-event.md` | `wait` command case | 1 | 1 | done |
|
||||
@@ -294,8 +306,9 @@ docs/tests/orch/
|
||||
2. shared fixtures and assertion helpers in `docs/tests/orch/_shared/README.md`
|
||||
3. workflow cases in `docs/tests/orch/workflows/README.md`
|
||||
4. core scheduler command docs: `run-init`, `task-add`, `dep-add`, `ready`, `dispatch`, `reconcile`, `status`
|
||||
5. interactive leader command docs: `wait`, `blocked`, `answer`, `retry`, `reassign`, `cancel`, `cleanup`
|
||||
6. council workflow docs: `council-start`, `council-wait`, `council-tally`, `council-report`
|
||||
5. verification command docs: `verify`
|
||||
6. interactive leader command docs: `wait`, `blocked`, `answer`, `retry`, `reassign`, `cancel`, `cleanup`
|
||||
7. council workflow docs: `council-start`, `council-wait`, `council-tally`, `council-report`
|
||||
|
||||
## Authored Case Register
|
||||
|
||||
@@ -310,6 +323,8 @@ docs/tests/orch/
|
||||
| `docs/tests/orch/task-add/task-add-creates-ready-root-task.md` | `task-add-creates-ready-root-task` | dependency-free task becomes ready immediately | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-invalid-acceptance-json.md` | `task-add-rejects-invalid-acceptance-json` | malformed `--acceptance-json` returns stable invalid_input | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-invalid-priority.md` | `task-add-rejects-invalid-priority` | unsupported priorities are rejected with invalid_input | done |
|
||||
| `docs/tests/orch/task-add/task-add-snapshots-spec-and-verification-policy.md` | `task-add-snapshots-spec-and-verification-policy` | task add snapshots spec content, verification profile, and scope policy onto the task | done |
|
||||
| `docs/tests/orch/task-add/task-add-rejects-spec-sha-mismatch.md` | `task-add-rejects-spec-sha-mismatch` | explicit spec hash mismatch returns invalid_input | done |
|
||||
| `docs/tests/orch/dep-add/dep-add-blocks-dependent-task-until-prerequisite-completes.md` | `dep-add-blocks-dependent-task-until-prerequisite-completes` | dependency edge prevents immediate readiness | done |
|
||||
| `docs/tests/orch/ready/ready-lists-only-eligible-tasks.md` | `ready-lists-only-eligible-tasks` | ready list excludes dependency-gated tasks | done |
|
||||
| `docs/tests/orch/ready/ready-orders-by-priority-and-respects-limit.md` | `ready-orders-by-priority-and-respects-limit` | ready output orders by priority and applies explicit limit truncation | done |
|
||||
@@ -322,6 +337,9 @@ docs/tests/orch/
|
||||
| `docs/tests/orch/dispatch/dispatch-analysis-mode-skips-worktree.md` | `dispatch-analysis-mode-skips-worktree` | analysis mode stays on the normal non-worktree path | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-claimed-or-in-progress-thread-to-running.md` | `reconcile-maps-claimed-or-in-progress-thread-to-running` | reconcile maps active inbox execution to running task state | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-done-or-failed-thread-to-terminal-task-state.md` | `reconcile-maps-done-or-failed-thread-to-terminal-task-state` | reconcile maps terminal inbox states to terminal task states | done |
|
||||
| `docs/tests/orch/reconcile/reconcile-maps-done-thread-to-verifying-when-task-has-required-checks.md` | `reconcile-maps-done-thread-to-verifying-when-task-has-required-checks` | reconcile routes worker done into verifying when the task has required checks | done |
|
||||
| `docs/tests/orch/verify/verify-status-returns-spec-and-gate-for-task.md` | `verify-status-returns-spec-and-gate-for-task` | verify status returns the task spec snapshot, selected attempt, and current gate state | done |
|
||||
| `docs/tests/orch/verify/verify-record-updates-gate-and-marks-task-done-when-required-checks-pass.md` | `verify-record-updates-gate-and-marks-task-done-when-required-checks-pass` | verify record recomputes the gate and promotes the task to done when all required checks pass | done |
|
||||
| `docs/tests/orch/wait/wait-wakes-on-matching-run-event.md` | `wait-wakes-on-matching-run-event` | wait wakes on a later matching run-scoped event | done |
|
||||
| `docs/tests/orch/wait/wait-times-out-without-matching-event.md` | `wait-times-out-without-matching-event` | wait timeout returns a normal non-woken result | done |
|
||||
| `docs/tests/orch/blocked/blocked-lists-latest-question-for-blocked-task.md` | `blocked-lists-latest-question-for-blocked-task` | blocked view includes latest question payload for the task | done |
|
||||
|
||||
@@ -5,4 +5,5 @@
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `reconcile-maps-claimed-or-in-progress-thread-to-running` | [reconcile-maps-claimed-or-in-progress-thread-to-running.md](./reconcile-maps-claimed-or-in-progress-thread-to-running.md) | maps worker claim or in-progress inbox state back into a running orch task |
|
||||
| `reconcile-maps-done-or-failed-thread-to-terminal-task-state` | [reconcile-maps-done-or-failed-thread-to-terminal-task-state.md](./reconcile-maps-done-or-failed-thread-to-terminal-task-state.md) | maps terminal inbox states into terminal task states and updates run aggregates |
|
||||
| `reconcile-maps-done-or-failed-thread-to-terminal-task-state` | [reconcile-maps-done-or-failed-thread-to-terminal-task-state.md](./reconcile-maps-done-or-failed-thread-to-terminal-task-state.md) | maps `done` without a gate or `failed` inbox states into terminal task states and updates run aggregates |
|
||||
| `reconcile-maps-done-thread-to-verifying-when-task-has-required-checks` | [reconcile-maps-done-thread-to-verifying-when-task-has-required-checks.md](./reconcile-maps-done-thread-to-verifying-when-task-has-required-checks.md) | routes worker `done` into `verifying` when the task has required checks |
|
||||
|
||||
+9
@@ -3,10 +3,15 @@
|
||||
## 用例意义
|
||||
|
||||
验证 `reconcile` 会把 worker 侧 thread 的终态同步到 `orch` 任务,并刷新 run 聚合状态。
|
||||
这个 case 只覆盖两类终态:
|
||||
|
||||
- worker `done` 且 task 没有 required checks
|
||||
- worker `fail`
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run 和已 dispatch 的任务
|
||||
- 该任务没有 configured verification gate,或者输入使用的是 `fail`
|
||||
- worker 已对该 thread 完成 `done` 或 `fail`
|
||||
|
||||
## 输入
|
||||
@@ -32,3 +37,7 @@ orch --db TMPDIR/coord.db --json status --run run_blog_001
|
||||
|
||||
- 任务终态依赖 `reconcile` 落回 `orch`,而不是由 worker 直接改写 task 表
|
||||
- run 级聚合状态会随终态任务一并刷新
|
||||
|
||||
## 补充约束
|
||||
|
||||
- 如果 task 声明了 required checks,worker `done` 不应再直接进入 `done`;那条分支由 `reconcile-maps-done-thread-to-verifying-when-task-has-required-checks.md` 覆盖
|
||||
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
# Case: `reconcile-maps-done-thread-to-verifying-when-task-has-required-checks`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `reconcile` 在 worker 报 `done` 之后,如果任务声明了 required checks,不会直接把 task 置为 `done`,而是先推进到 `verifying`。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在带 required checks 的任务
|
||||
- 该任务已经 dispatch 并被 worker claim
|
||||
- worker 已对该 thread 执行 `done`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_verify_001 --goal "Exercise verification gates"
|
||||
orch --db TMPDIR/coord.db --json task add \
|
||||
--run run_verify_001 \
|
||||
--task T1 \
|
||||
--title "Implement verifier-backed task" \
|
||||
--default-to worker-a \
|
||||
--spec-file TMPDIR/task.md \
|
||||
--check-profile cadence_component \
|
||||
--required-check lint \
|
||||
--required-check test
|
||||
orch --db TMPDIR/coord.db --json dispatch --run run_verify_001 --task T1 --execution-mode analysis --body "Implement the gated task."
|
||||
inbox --db TMPDIR/coord.db --json claim --agent worker-a --thread THREAD_ID
|
||||
inbox --db TMPDIR/coord.db --json done --agent worker-a --thread THREAD_ID --summary "Implementation finished" --body "Ready for verification."
|
||||
orch --db TMPDIR/coord.db --json reconcile --run run_verify_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `reconcile` 退出码为 `0`
|
||||
- `data.updated_tasks` 包含 `T1`
|
||||
- `T1.status == "verifying"`
|
||||
- 后续 `orch verify status --run run_verify_001 --task T1` 返回 `data.gate.status == "pending"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- worker 的 `done` 不再自动等同于 task `done`
|
||||
- 一旦 task 定义了 required checks,`reconcile` 的职责是把它送入验证门,而不是直接宣布完成
|
||||
@@ -7,3 +7,5 @@
|
||||
| `task-add-creates-ready-root-task` | [task-add-creates-ready-root-task.md](./task-add-creates-ready-root-task.md) | creates a dependency-free task that becomes ready immediately |
|
||||
| `task-add-rejects-invalid-acceptance-json` | [task-add-rejects-invalid-acceptance-json.md](./task-add-rejects-invalid-acceptance-json.md) | rejects malformed `--acceptance-json` with `invalid_input` |
|
||||
| `task-add-rejects-invalid-priority` | [task-add-rejects-invalid-priority.md](./task-add-rejects-invalid-priority.md) | rejects priorities outside `low|normal|high` |
|
||||
| `task-add-snapshots-spec-and-verification-policy` | [task-add-snapshots-spec-and-verification-policy.md](./task-add-snapshots-spec-and-verification-policy.md) | snapshots spec file content, verification profile, and scope policy onto the task |
|
||||
| `task-add-rejects-spec-sha-mismatch` | [task-add-rejects-spec-sha-mismatch.md](./task-add-rejects-spec-sha-mismatch.md) | rejects explicit spec hashes that do not match the task spec file content |
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# Case: `task-add-rejects-spec-sha-mismatch`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `task add` 在接收 `--spec-file` 与 `--spec-sha` 时,会拒绝内容摘要不匹配的任务定义,避免 task spec 漂移。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_007`
|
||||
- 临时目录内存在可读取的 spec 文件 `TMPDIR/task.md`
|
||||
- 调用时传入的 `--spec-sha` 与文件实际 SHA256 不一致
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_007 --goal "Validate spec sha mismatch"
|
||||
orch --db TMPDIR/coord.db --json task add \
|
||||
--run run_blog_007 \
|
||||
--task T1 \
|
||||
--title "Implement verifier" \
|
||||
--spec-file TMPDIR/task.md \
|
||||
--spec-sha deadbeef
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `task add` 退出码为 `30`
|
||||
- JSON error payload 的 `error.code == "invalid_input"`
|
||||
- `error.message` 包含 `spec-sha does not match spec-file contents`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- task spec 快照不是“尽力而为”的附带字段;当显式声明 SHA 时,CLI 会把它当成契约校验
|
||||
- leader 不能在 spec 内容与预期摘要不一致时继续创建 task
|
||||
@@ -0,0 +1,49 @@
|
||||
# Case: `task-add-snapshots-spec-and-verification-policy`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `task add` 在创建任务时,不只是写入基础调度字段,还会快照 task spec 与验证策略。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在 run `run_blog_006`
|
||||
- 临时目录内存在可读取的 spec 文件 `TMPDIR/task.md`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json run init --run run_blog_006 --goal "Validate spec-aware task add"
|
||||
orch --db TMPDIR/coord.db --json task add \
|
||||
--run run_blog_006 \
|
||||
--task T1 \
|
||||
--title "Implement verifier" \
|
||||
--spec-file TMPDIR/task.md \
|
||||
--check-profile cadence_component \
|
||||
--required-check lint \
|
||||
--required-check test \
|
||||
--allowed-path packages/ui \
|
||||
--blocked-path scripts/release-metadata.mjs \
|
||||
--metadata-json '{"repo":"cadence-ui"}'
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `task add` 退出码为 `0`
|
||||
- `data.task.status == "ready"`
|
||||
- `data.task.spec.spec_file == "TMPDIR/task.md"`
|
||||
- `data.task.spec.check_profile == "cadence_component"`
|
||||
- `data.task.spec.required_checks` 包含 `lint` 与 `test`
|
||||
- `data.task.spec.allowed_paths` 包含 `packages/ui`
|
||||
- `data.task.spec.blocked_paths` 包含 `scripts/release-metadata.mjs`
|
||||
- `data.task.gate.status == "pending"`
|
||||
- `data.task.gate.required_checks` 与 spec 中的 required checks 一致
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `task add` 现在会把任务说明和验证策略一起固化到 task spec,而不是只保存 `title`/`summary`
|
||||
- required checks 一旦存在,task 会立即带上 `pending` gate,而不是等到 worker 完成后才临时推断
|
||||
|
||||
## 补充约束
|
||||
|
||||
- `spec_file` 对应内容应作为快照随 task 保存,而不是只保存路径引用
|
||||
- `check_profile` 目前只是任务策略名,后续 profile/adapter 机制会负责把它解释成真正的执行计划
|
||||
@@ -0,0 +1,8 @@
|
||||
# Orch `verify` Test Plan Index
|
||||
|
||||
## Case Files
|
||||
|
||||
| Case Slug | File | Coverage Note |
|
||||
| --- | --- | --- |
|
||||
| `verify-status-returns-spec-and-gate-for-task` | [verify-status-returns-spec-and-gate-for-task.md](./verify-status-returns-spec-and-gate-for-task.md) | returns the selected task, latest attempt, spec snapshot, and current gate state |
|
||||
| `verify-record-updates-gate-and-marks-task-done-when-required-checks-pass` | [verify-record-updates-gate-and-marks-task-done-when-required-checks-pass.md](./verify-record-updates-gate-and-marks-task-done-when-required-checks-pass.md) | records named checks, recomputes the gate, and promotes the task to `done` when all required checks pass |
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
# Case: `verify-record-updates-gate-and-marks-task-done-when-required-checks-pass`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `verify record` 在逐个记录 required checks 后,会重新计算 gate,并在所有必过项通过时把 task 从 `verifying` 推进到 `done`。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在处于 `verifying` 的任务 `T1`
|
||||
- 该任务的 required checks 为 `lint` 与 `test`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json verify record --run run_verify_001 --task T1 --check lint --status passed --summary "lint clean"
|
||||
orch --db TMPDIR/coord.db --json verify record --run run_verify_001 --task T1 --check test --status passed --summary "tests clean"
|
||||
orch --db TMPDIR/coord.db --json status --run run_verify_001
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- 第一次 `verify record` 后:
|
||||
- `data.task.status == "verifying"`
|
||||
- `data.gate.status == "pending"`
|
||||
- `data.gate.pending_checks` 仍包含 `test`
|
||||
- 第二次 `verify record` 后:
|
||||
- `data.task.status == "done"`
|
||||
- `data.gate.status == "passed"`
|
||||
- `data.gate.pending_checks` 为空
|
||||
- 后续 `status.data.run.status == "done"`
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `verify record` 不是单纯写一条 check 日志;它会驱动 gate 和 task 状态机前进
|
||||
- 只有所有 required checks 通过,task 才会真正完成
|
||||
@@ -0,0 +1,32 @@
|
||||
# Case: `verify-status-returns-spec-and-gate-for-task`
|
||||
|
||||
## 用例意义
|
||||
|
||||
验证 `verify status` 能把 task 的验证上下文一次性展示出来,而不是要求 leader 手工拼装 task、attempt、spec 与 check 结果。
|
||||
|
||||
## 前置条件
|
||||
|
||||
- 已存在带 required checks 的任务
|
||||
- 该任务已经经过 `reconcile` 进入 `verifying`
|
||||
|
||||
## 输入
|
||||
|
||||
```bash
|
||||
orch --db TMPDIR/coord.db --json verify status --run run_verify_001 --task T1
|
||||
```
|
||||
|
||||
## 预期输出
|
||||
|
||||
- `verify status` 退出码为 `0`
|
||||
- `data.task.task_id == "T1"`
|
||||
- `data.attempt.attempt_no == 1`
|
||||
- `data.spec.spec_file` 非空
|
||||
- `data.spec.check_profile == "cadence_component"`
|
||||
- `data.gate.status == "pending"`
|
||||
- `data.gate.required_checks` 包含 `lint` 与 `test`
|
||||
- `data.gate.pending_checks` 在首次查询时仍包含所有未通过检查
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `verify status` 是 leader 查看 gate 的主入口,而不是只返回 task 表里的裸状态
|
||||
- gate 是否仍在等待检查、已经失败、还是已经通过,都应在一个响应里可见
|
||||
Reference in New Issue
Block a user