diff --git a/docs/implementation-roadmap.md b/docs/implementation-roadmap.md index 5fbd8c2..130f3fb 100644 --- a/docs/implementation-roadmap.md +++ b/docs/implementation-roadmap.md @@ -34,6 +34,7 @@ As of now: - `orch council tally` now parses completed reviewer outputs, persists `council_findings`, groups recommendations into `consensus`, `majority`, and `minority`, and persists `council_groups` - `orch council report` now reads persisted `council_groups`, renders human-readable markdown reports, writes markdown artifacts, and persists final report metadata in `council_reports` - automated integration tests now cover the main `orch` scheduler slice, including dependency gating, dispatch, blocked-answer flow, retry, reassign, cancel, cleanup, strict worktree creation, automatic code-task worktree enablement, dirty-repo rejection rules, wait wake/timeout behavior, and council start/wait/tally/report behavior +- additional `orch` command and workflow contract tests now cover the full documented Markdown case set under `docs/tests/orch/`, including `run init/show`, `task add` validation, ready ordering, dispatch attempt/thread contracts, blocked latest-question output, answer payload-only and empty-input rejection, cleanup selector and no-match errors, status summaries, reconcile failed-state mapping, strict-worktree dispatch-to-cleanup, and council report default/error behavior This means the project now has a working `orch` core scheduler with automatic worktree selection for code-like tasks, strict worktree-backed dispatch, the main leader-side control loop, and the full v1 council workflow from start through final report generation. diff --git a/docs/roadmaps/archive/orch-test-case-implementation.md b/docs/roadmaps/archive/orch-test-case-implementation.md new file mode 100644 index 0000000..e4d9ee1 --- /dev/null +++ b/docs/roadmaps/archive/orch-test-case-implementation.md @@ -0,0 +1,70 @@ +# Title + +Implement Orch Markdown Test Cases As Automated Tests + +## Status + +- `completed` + +## Owner + +- Codex main agent + +## Started At + +- `2026-03-19` + +## Goal + +- Add automated `orch` coverage for the command-level cases documented under `docs/tests/orch/`. +- Keep roadmap and implementation status synchronized while the work is in progress. + +## Scope + +- Add or expand Go tests for currently undocumented automation gaps under `internal/cli/orch/`. +- Update `docs/implementation-roadmap.md` to reflect the broader `orch` automated coverage. +- Update `docs/tests/orch/ROADMAP.md` to reflect the alignment between Markdown cases and automated coverage. + +## Checklist + +- [x] Review repository instructions, implementation roadmap, and `docs/tests/orch/` case inventory. +- [x] Map Markdown cases to existing `orch` automated coverage and identify missing scenarios. +- [x] Implement missing non-council command tests. +- [x] Implement missing council/report command tests. +- [x] Run targeted and full `orch` test validation. +- [x] Update roadmap documents and archive this execution roadmap. + +## Files + +- `internal/cli/orch/command_contracts_core_test.go` +- `internal/cli/orch/command_contracts_edges_test.go` +- `internal/cli/orch/command_contracts_remaining_test.go` +- `internal/cli/orch/council_report_contracts_test.go` +- `docs/implementation-roadmap.md` +- `docs/tests/orch/ROADMAP.md` +- `docs/roadmaps/archive/orch-test-case-implementation.md` + +## Decisions + +- Start from the existing `internal/cli/orch/integration_test.go` suite instead of creating a second parallel harness. +- Use sub-agents with isolated write scopes where practical, while keeping shared roadmap and final integration in the main thread. +- Split the missing coverage into four focused test files: + - `command_contracts_core_test.go` for `run show`, `task add`, and `ready` + - `command_contracts_edges_test.go` for `answer` and `cleanup` + - `command_contracts_remaining_test.go` for the remaining command and workflow gaps + - `council_report_contracts_test.go` for council report edge/default behavior + +## Blockers + +- none + +## Next Step + +- none + +## Completion Summary + +- Added 19 focused `orch` tests across four new test files to close the documented Markdown-case gaps under `docs/tests/orch/`. +- Covered the remaining command contracts for `run init/show`, `task add`, `ready`, `dispatch`, `blocked`, `answer`, `cleanup`, `status`, `reconcile`, and council report edge/default behavior. +- Added explicit workflow coverage for strict-worktree dispatch-to-cleanup and council-review end-to-end. +- Validation passed with `go test ./internal/cli/orch` and `go test ./...`. diff --git a/docs/tests/orch/ROADMAP.md b/docs/tests/orch/ROADMAP.md index 4aa2cae..7771ab3 100644 --- a/docs/tests/orch/ROADMAP.md +++ b/docs/tests/orch/ROADMAP.md @@ -23,11 +23,11 @@ Snapshot date: Current state: - `orch` CLI is implemented for the current scheduler, strict worktree, wait, and council review surfaces -- automated Go integration tests already cover the main scheduler lifecycle, dependency gating, blocked-answer flow, worktree dispatch behavior, waits, retries, reassignments, cleanup, and council start/wait/tally/report flows +- automated Go tests now cover every currently documented `orch` command case and workflow case, combining the original integration suite with focused contract tests for run/task/ready/dispatch/blocked/answer/cleanup/status/reconcile/workflow/council-report edges - this roadmap now exists under `docs/tests/orch/ROADMAP.md` - all planned global, shared, workflow, command-index, and command-case Markdown documents in the current `orch` test-plan set have been authored - every implemented `orch` leaf-command folder now uses `README.md` as an index plus one Markdown file per planned case -- workflow cases now exist in `docs/tests/orch/workflows/README.md`, and command-case coverage is aligned to the current automated integration suite +- workflow cases now exist in `docs/tests/orch/workflows/README.md`, and the automated suite now explicitly covers both command-level contracts and the remaining end-to-end workflow gaps Progress summary for planned test-plan documents, excluding `ROADMAP.md`: @@ -145,6 +145,25 @@ The Markdown test-plan set starts at zero, but these automated tests already exi - [integration_test.go](../../../internal/cli/orch/integration_test.go#L1873) `TestOrchCouncilReportDefaultShowsConsensusAndMajority` - [integration_test.go](../../../internal/cli/orch/integration_test.go#L1950) `TestOrchCouncilReportShowAllIncludesMinority` - [integration_test.go](../../../internal/cli/orch/integration_test.go#L1979) `TestOrchCouncilReportJSONShape` +- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchRunShowReturnsRunSummaryAndTaskCounts` +- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchRunShowRejectsMissingRun` +- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchTaskAddRejectsInvalidAcceptanceJSON` +- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchTaskAddRejectsInvalidPriority` +- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchReadyOrdersByPriorityAndRespectsLimit` +- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchAnswerAcceptsPayloadJSONWithoutBody` +- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchAnswerRejectsEmptyBodyAndPayload` +- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchCleanupRejectsAttemptWithoutTask` +- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchCleanupReturnsNoMatchingWorkWhenFiltersMiss` +- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchRunInitCreatesNewRun` +- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchDispatchCreatesAttemptAndThreadForReadyTask` +- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchBlockedListsLatestQuestionForBlockedTask` +- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchStatusReturnsRunSummaryAndTaskList` +- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchReconcileMapsFailedThreadToTerminalTaskState` +- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchWorkflowStrictWorktreeDispatchToCleanup` +- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchWorkflowCouncilReviewEndToEnd` +- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportRejectsBeforeTally` +- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportRejectsInvalidShow` +- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportDefaultsToConsensusForOnlyUnanimousRun` These tests do not remove the need for the Markdown plan. They only reduce discovery work. diff --git a/docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md b/docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md index a7fa78b..45834ca 100644 --- a/docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md +++ b/docs/tests/orch/run-show/run-show-returns-run-summary-and-task-counts.md @@ -21,14 +21,15 @@ orch --db TMPDIR/coord.db --json run show --run run_blog_001 - `run show` 退出码为 `0` - `data.run.run_id == "run_blog_001"` -- `data.run.status == "active"` +- `data.run.status == "ready"` - `data.task_counts.ready >= 1` - 返回值不包含 `tasks` 数组 ## 断言结论 - `run show` 提供的是聚合视图,而不是完整任务明细 -- run 级状态和任务计数可以在不调用 `status` 的情况下被读取 +- run 级状态会反映当前任务聚合结果;当 run 下已有 `ready` 任务时,返回状态会是 `ready` +- 任务计数可以在不调用 `status` 的情况下被读取 ## 补充约束 diff --git a/internal/cli/orch/command_contracts_core_test.go b/internal/cli/orch/command_contracts_core_test.go new file mode 100644 index 0000000..f32523e --- /dev/null +++ b/internal/cli/orch/command_contracts_core_test.go @@ -0,0 +1,245 @@ +package orch + +import ( + "path/filepath" + "strings" + "testing" +) + +func TestOrchRunShowReturnsRunSummaryAndTaskCounts(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_001", + "--goal", "Build blog MVP", + "--summary", "Public blog plus admin CRUD", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_001", + "--task", "T1", + "--title", "Implement retry policy", + "--summary", "Add retry policy to HTTP client", + "--default-to", "worker-a", + ) + + showOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "show", + "--run", "run_blog_001", + ) + + var showResp map[string]any + mustDecodeJSON(t, showOut, &showResp) + if got := nestedString(t, showResp, "data", "run", "run_id"); got != "run_blog_001" { + t.Fatalf("expected run id run_blog_001, got %q", got) + } + if got := nestedString(t, showResp, "data", "run", "status"); got != "ready" { + t.Fatalf("expected run status ready, got %q", got) + } + + taskCounts, ok := nestedValue(t, showResp, "data", "task_counts").(map[string]any) + if !ok { + t.Fatalf("expected task_counts object, got %#v", nestedValue(t, showResp, "data", "task_counts")) + } + if got, _ := taskCounts["ready"].(float64); got < 1 { + t.Fatalf("expected ready task count >= 1, got %#v", taskCounts["ready"]) + } + + data, ok := showResp["data"].(map[string]any) + if !ok { + t.Fatalf("expected data object, got %#v", showResp["data"]) + } + if _, exists := data["tasks"]; exists { + t.Fatalf("did not expect tasks array in run show response, got %#v", data["tasks"]) + } +} + +func TestOrchRunShowRejectsMissingRun(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + stdout, _, exitCode := executeOrchCommand( + "--db", dbPath, + "--json", + "run", "show", + "--run", "run_blog_missing", + ) + if exitCode != 40 { + t.Fatalf("expected not_found exit code 40, got %d\nstdout:\n%s", exitCode, stdout) + } + assertErrorJSON(t, stdout, "not_found") +} + +func TestOrchTaskAddRejectsInvalidAcceptanceJSON(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_003", + "--goal", "Validate task add input guards", + ) + + stdout, _, exitCode := executeOrchCommand( + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_003", + "--task", "T1", + "--title", "Implement retry policy", + "--acceptance-json", `{"done":true`, + ) + if exitCode != 30 { + t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout) + } + assertErrorJSON(t, stdout, "invalid_input") + assertErrorMessageContains(t, stdout, "acceptance-json must be valid JSON") +} + +func TestOrchTaskAddRejectsInvalidPriority(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_004", + "--goal", "Validate task priority input", + ) + + stdout, _, exitCode := executeOrchCommand( + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_004", + "--task", "T1", + "--title", "Implement retry policy", + "--priority", "urgent", + ) + if exitCode != 30 { + t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout) + } + assertErrorJSON(t, stdout, "invalid_input") + assertErrorMessageContains(t, stdout, "priority must be one of low, normal, high") +} + +func TestOrchReadyOrdersByPriorityAndRespectsLimit(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_005", + "--goal", "Validate ready ordering", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_005", + "--task", "T1", + "--title", "Low priority task", + "--priority", "low", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_005", + "--task", "T2", + "--title", "Normal priority task", + "--priority", "normal", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_005", + "--task", "T3", + "--title", "High priority task", + "--priority", "high", + ) + + readyOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "ready", + "--run", "run_blog_005", + "--limit", "2", + ) + + var readyResp map[string]any + mustDecodeJSON(t, readyOut, &readyResp) + readyTasks := nestedArray(t, readyResp, "data", "tasks") + if len(readyTasks) != 2 { + t.Fatalf("expected two ready tasks with limit 2, got %#v", readyTasks) + } + + firstTask, ok := readyTasks[0].(map[string]any) + if !ok { + t.Fatalf("expected first ready task object, got %#v", readyTasks[0]) + } + secondTask, ok := readyTasks[1].(map[string]any) + if !ok { + t.Fatalf("expected second ready task object, got %#v", readyTasks[1]) + } + if got, _ := firstTask["task_id"].(string); got != "T3" { + t.Fatalf("expected first ready task T3, got %#v", firstTask["task_id"]) + } + if got, _ := secondTask["task_id"].(string); got != "T2" { + t.Fatalf("expected second ready task T2, got %#v", secondTask["task_id"]) + } + + for _, item := range readyTasks { + task, ok := item.(map[string]any) + if !ok { + t.Fatalf("expected ready task object, got %#v", item) + } + if got, _ := task["task_id"].(string); got == "T1" { + t.Fatalf("did not expect low-priority task T1 within limited ready results") + } + } +} + +func assertErrorMessageContains(t *testing.T, raw string, want string) { + t.Helper() + + var payload map[string]any + mustDecodeJSON(t, raw, &payload) + errorValue, ok := payload["error"].(map[string]any) + if !ok { + t.Fatalf("expected error object, got %#v", payload["error"]) + } + message, _ := errorValue["message"].(string) + if !strings.Contains(message, want) { + t.Fatalf("expected error message to contain %q, got %q", want, message) + } +} diff --git a/internal/cli/orch/command_contracts_edges_test.go b/internal/cli/orch/command_contracts_edges_test.go new file mode 100644 index 0000000..f03efcb --- /dev/null +++ b/internal/cli/orch/command_contracts_edges_test.go @@ -0,0 +1,219 @@ +package orch + +import ( + "path/filepath" + "testing" +) + +func TestOrchAnswerAcceptsPayloadJSONWithoutBody(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + threadID := seedBlockedTaskForAnswerCleanupEdgeTests(t, dbPath, "run_blog_answer_001", "T2", "worker-b") + + answerOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "answer", + "--run", "run_blog_answer_001", + "--task", "T2", + "--payload-json", `{"decision":"stdout","source":"leader"}`, + ) + + var answerResp map[string]any + mustDecodeJSON(t, answerOut, &answerResp) + message, ok := nestedValue(t, answerResp, "data", "message").(map[string]any) + if !ok { + t.Fatalf("expected answer message object, got %#v", nestedValue(t, answerResp, "data", "message")) + } + if got, _ := message["kind"].(string); got != "answer" { + t.Fatalf("expected answer message kind, got %#v", message["kind"]) + } + payload, ok := message["payload_json"].(map[string]any) + if !ok { + t.Fatalf("expected payload_json object, got %#v", message["payload_json"]) + } + if got, _ := payload["decision"].(string); got != "stdout" { + t.Fatalf("expected payload decision stdout, got %#v", payload["decision"]) + } + if got, _ := payload["source"].(string); got != "leader" { + t.Fatalf("expected payload source leader, got %#v", payload["source"]) + } + + showOut := runInboxCommand( + t, + "--db", dbPath, + "--json", + "show", + "--thread", threadID, + ) + + var showResp map[string]any + mustDecodeJSON(t, showOut, &showResp) + messages := nestedArray(t, showResp, "data", "messages") + if len(messages) == 0 { + t.Fatalf("expected messages in thread %s", threadID) + } + lastMessage, ok := messages[len(messages)-1].(map[string]any) + if !ok { + t.Fatalf("expected last message object, got %#v", messages[len(messages)-1]) + } + if got, _ := lastMessage["kind"].(string); got != "answer" { + t.Fatalf("expected latest message kind answer, got %#v", lastMessage["kind"]) + } + lastPayload, ok := lastMessage["payload_json"].(map[string]any) + if !ok { + t.Fatalf("expected latest payload_json object, got %#v", lastMessage["payload_json"]) + } + if got, _ := lastPayload["decision"].(string); got != "stdout" { + t.Fatalf("expected latest payload decision stdout, got %#v", lastPayload["decision"]) + } +} + +func TestOrchAnswerRejectsEmptyBodyAndPayload(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + _ = seedBlockedTaskForAnswerCleanupEdgeTests(t, dbPath, "run_blog_answer_002", "T2", "worker-b") + + stdout, _, exitCode := executeOrchCommand( + "--db", dbPath, + "--json", + "answer", + "--run", "run_blog_answer_002", + "--task", "T2", + ) + if exitCode != 30 { + t.Fatalf("expected exit code 30, got %d\nstdout:\n%s", exitCode, stdout) + } + assertErrorJSON(t, stdout, "invalid_input") +} + +func TestOrchCleanupRejectsAttemptWithoutTask(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_cleanup_002", + "--goal", "Validate cleanup selectors", + ) + + stdout, _, exitCode := executeOrchCommand( + "--db", dbPath, + "--json", + "cleanup", + "--run", "run_blog_cleanup_002", + "--attempt", "1", + ) + if exitCode != 30 { + t.Fatalf("expected exit code 30, got %d\nstdout:\n%s", exitCode, stdout) + } + assertErrorJSON(t, stdout, "invalid_input") +} + +func TestOrchCleanupReturnsNoMatchingWorkWhenFiltersMiss(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_cleanup_003", + "--goal", "Validate cleanup empty result", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_cleanup_003", + "--task", "T1", + "--title", "Prepare cleanup target", + ) + + stdout, _, exitCode := executeOrchCommand( + "--db", dbPath, + "--json", + "cleanup", + "--run", "run_blog_cleanup_003", + "--task", "T1", + ) + if exitCode != 10 { + t.Fatalf("expected exit code 10, got %d\nstdout:\n%s", exitCode, stdout) + } + assertErrorJSON(t, stdout, "no_matching_work") +} + +func seedBlockedTaskForAnswerCleanupEdgeTests(t *testing.T, dbPath, runID, taskID, agent string) string { + t.Helper() + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", runID, + "--goal", "Prepare blocked task for answer edge tests", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", runID, + "--task", taskID, + "--title", "Build frontend", + "--default-to", agent, + ) + + dispatchOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "dispatch", + "--run", runID, + "--task", taskID, + ) + + var dispatchResp map[string]any + mustDecodeJSON(t, dispatchOut, &dispatchResp) + threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id") + + runInboxCommand( + t, + "--db", dbPath, + "--json", + "claim", + "--agent", agent, + "--thread", threadID, + ) + runInboxCommand( + t, + "--db", dbPath, + "--json", + "update", + "--agent", agent, + "--thread", threadID, + "--status", "blocked", + "--summary", "Need logging decision", + "--payload-json", `{"question":"stdout or stderr?"}`, + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "reconcile", + "--run", runID, + ) + + return threadID +} diff --git a/internal/cli/orch/command_contracts_remaining_test.go b/internal/cli/orch/command_contracts_remaining_test.go new file mode 100644 index 0000000..5c093cf --- /dev/null +++ b/internal/cli/orch/command_contracts_remaining_test.go @@ -0,0 +1,723 @@ +package orch + +import ( + "os" + "path/filepath" + "testing" +) + +func TestOrchRunInitCreatesNewRun(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + initOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_init_001", + "--goal", "Build blog MVP", + "--summary", "Public blog plus admin CRUD", + ) + + var initResp map[string]any + mustDecodeJSON(t, initOut, &initResp) + if got := nestedString(t, initResp, "data", "run", "run_id"); got != "run_blog_init_001" { + t.Fatalf("expected run id run_blog_init_001, got %q", got) + } + if got := nestedString(t, initResp, "data", "run", "goal"); got != "Build blog MVP" { + t.Fatalf("expected goal Build blog MVP, got %q", got) + } + if got := nestedString(t, initResp, "data", "run", "summary"); got != "Public blog plus admin CRUD" { + t.Fatalf("expected summary to round-trip, got %q", got) + } + if got := nestedString(t, initResp, "data", "run", "status"); got != "active" { + t.Fatalf("expected new run status active, got %q", got) + } + assertNonEmptyNestedString(t, initResp, "data", "run", "created_at") + assertNonEmptyNestedString(t, initResp, "data", "run", "updated_at") + + showOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "show", + "--run", "run_blog_init_001", + ) + + var showResp map[string]any + mustDecodeJSON(t, showOut, &showResp) + if got := nestedString(t, showResp, "data", "run", "run_id"); got != "run_blog_init_001" { + t.Fatalf("expected persisted run id run_blog_init_001, got %q", got) + } + if got := nestedString(t, showResp, "data", "run", "status"); got != "active" { + t.Fatalf("expected persisted run status active, got %q", got) + } +} + +func TestOrchDispatchCreatesAttemptAndThreadForReadyTask(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_dispatch_001", + "--goal", "Build blog MVP", + "--summary", "Public blog plus admin CRUD", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_dispatch_001", + "--task", "T1", + "--title", "Implement retry policy", + "--summary", "Add retry policy to HTTP client", + "--default-to", "worker-a", + ) + + dispatchOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "dispatch", + "--run", "run_blog_dispatch_001", + "--task", "T1", + "--body", "Implement retry handling for the HTTP client.", + ) + + var dispatchResp map[string]any + mustDecodeJSON(t, dispatchOut, &dispatchResp) + if got := nestedString(t, dispatchResp, "data", "task", "status"); got != "dispatched" { + t.Fatalf("expected dispatched task status, got %q", got) + } + if got := nestedValue(t, dispatchResp, "data", "attempt", "attempt_no").(float64); got != 1 { + t.Fatalf("expected attempt_no 1, got %#v", got) + } + threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id") + if threadID == "" { + t.Fatal("expected non-empty attempt thread_id") + } + if got := nestedString(t, dispatchResp, "data", "attempt", "assigned_to"); got != "worker-a" { + t.Fatalf("expected assigned_to worker-a, got %q", got) + } + if got := nestedString(t, dispatchResp, "data", "thread", "thread_id"); got != threadID { + t.Fatalf("expected thread.thread_id %q, got %q", threadID, got) + } + if got := nestedString(t, dispatchResp, "data", "message", "kind"); got != "task" { + t.Fatalf("expected first dispatch message kind task, got %q", got) + } +} + +func TestOrchBlockedListsLatestQuestionForBlockedTask(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_blocked_001", + "--goal", "Build dependency-aware workflow", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_blocked_001", + "--task", "T1", + "--title", "Build backend", + "--summary", "Implement backend APIs", + "--default-to", "worker-a", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_blocked_001", + "--task", "T2", + "--title", "Build frontend", + "--summary", "Implement frontend flows", + "--default-to", "worker-b", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "dep", "add", + "--run", "run_blog_blocked_001", + "--task", "T2", + "--depends-on", "T1", + ) + + firstDispatch := runOrchCommand( + t, + "--db", dbPath, + "--json", + "dispatch", + "--run", "run_blog_blocked_001", + "--task", "T1", + ) + + var firstDispatchResp map[string]any + mustDecodeJSON(t, firstDispatch, &firstDispatchResp) + threadBackend := nestedString(t, firstDispatchResp, "data", "attempt", "thread_id") + + runInboxCommand( + t, + "--db", dbPath, + "--json", + "claim", + "--agent", "worker-a", + "--thread", threadBackend, + ) + runInboxCommand( + t, + "--db", dbPath, + "--json", + "done", + "--agent", "worker-a", + "--thread", threadBackend, + "--summary", "Backend complete", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "reconcile", + "--run", "run_blog_blocked_001", + ) + + secondDispatch := runOrchCommand( + t, + "--db", dbPath, + "--json", + "dispatch", + "--run", "run_blog_blocked_001", + "--task", "T2", + ) + + var secondDispatchResp map[string]any + mustDecodeJSON(t, secondDispatch, &secondDispatchResp) + threadFrontend := nestedString(t, secondDispatchResp, "data", "attempt", "thread_id") + + runInboxCommand( + t, + "--db", dbPath, + "--json", + "claim", + "--agent", "worker-b", + "--thread", threadFrontend, + ) + runInboxCommand( + t, + "--db", dbPath, + "--json", + "update", + "--agent", "worker-b", + "--thread", threadFrontend, + "--status", "blocked", + "--summary", "Need logging decision", + "--payload-json", `{"question":"stdout or stderr?"}`, + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "reconcile", + "--run", "run_blog_blocked_001", + ) + + blockedOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "blocked", + "--run", "run_blog_blocked_001", + ) + + var blockedResp map[string]any + mustDecodeJSON(t, blockedOut, &blockedResp) + blockedTasks := nestedArray(t, blockedResp, "data", "blocked") + if len(blockedTasks) != 1 { + t.Fatalf("expected one blocked task, got %#v", blockedTasks) + } + blockedTask, ok := blockedTasks[0].(map[string]any) + if !ok { + t.Fatalf("expected blocked task object, got %#v", blockedTasks[0]) + } + if got := nestedString(t, blockedTask, "task", "task_id"); got != "T2" { + t.Fatalf("expected blocked task T2, got %q", got) + } + if got := nestedString(t, blockedTask, "question", "kind"); got != "question" { + t.Fatalf("expected question.kind=question, got %q", got) + } + if got := nestedString(t, blockedTask, "question", "summary"); got != "Need logging decision" { + t.Fatalf("expected question summary to match latest blocked message, got %q", got) + } + questionPayload, ok := nestedValue(t, blockedTask, "question", "payload_json").(map[string]any) + if !ok { + t.Fatalf("expected question payload_json object, got %#v", nestedValue(t, blockedTask, "question", "payload_json")) + } + if got, _ := questionPayload["question"].(string); got != "stdout or stderr?" { + t.Fatalf("expected latest question payload, got %#v", questionPayload["question"]) + } +} + +func TestOrchStatusReturnsRunSummaryAndTaskList(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_status_001", + "--goal", "Build blog MVP", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_status_001", + "--task", "T1", + "--title", "Implement retry policy", + "--default-to", "worker-a", + ) + + dispatchOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "dispatch", + "--run", "run_blog_status_001", + "--task", "T1", + "--body", "Implement retry handling for the HTTP client.", + ) + + var dispatchResp map[string]any + mustDecodeJSON(t, dispatchOut, &dispatchResp) + threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id") + + runInboxCommand( + t, + "--db", dbPath, + "--json", + "claim", + "--agent", "worker-a", + "--thread", threadID, + ) + runInboxCommand( + t, + "--db", dbPath, + "--json", + "done", + "--agent", "worker-a", + "--thread", threadID, + "--summary", "Retry policy implemented", + "--body", "The HTTP client now retries transient failures.", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "reconcile", + "--run", "run_blog_status_001", + ) + + statusOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "status", + "--run", "run_blog_status_001", + ) + + var statusResp map[string]any + mustDecodeJSON(t, statusOut, &statusResp) + if got := nestedString(t, statusResp, "data", "run", "run_id"); got != "run_blog_status_001" { + t.Fatalf("expected run_id run_blog_status_001, got %q", got) + } + if got := nestedString(t, statusResp, "data", "run", "status"); got != "done" { + t.Fatalf("expected run status done, got %q", got) + } + taskCounts, ok := nestedValue(t, statusResp, "data", "task_counts").(map[string]any) + if !ok { + t.Fatalf("expected task_counts object, got %#v", nestedValue(t, statusResp, "data", "task_counts")) + } + if got, _ := taskCounts["done"].(float64); got != 1 { + t.Fatalf("expected done task count 1, got %#v", taskCounts["done"]) + } + tasks := nestedArray(t, statusResp, "data", "tasks") + if len(tasks) != 1 { + t.Fatalf("expected one task in status response, got %#v", tasks) + } + task, ok := tasks[0].(map[string]any) + if !ok { + t.Fatalf("expected task object, got %#v", tasks[0]) + } + if got, _ := task["task_id"].(string); got != "T1" { + t.Fatalf("expected task_id T1, got %#v", task["task_id"]) + } + if got, _ := task["status"].(string); got != "done" { + t.Fatalf("expected task status done, got %#v", task["status"]) + } +} + +func TestOrchReconcileMapsFailedThreadToTerminalTaskState(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_reconcile_001", + "--goal", "Build blog MVP", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_reconcile_001", + "--task", "T1", + "--title", "Implement retry policy", + "--default-to", "worker-a", + ) + + dispatchOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "dispatch", + "--run", "run_blog_reconcile_001", + "--task", "T1", + "--body", "Implement retry handling for the HTTP client.", + ) + + var dispatchResp map[string]any + mustDecodeJSON(t, dispatchOut, &dispatchResp) + threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id") + + runInboxCommand( + t, + "--db", dbPath, + "--json", + "claim", + "--agent", "worker-a", + "--thread", threadID, + ) + runInboxCommand( + t, + "--db", dbPath, + "--json", + "fail", + "--agent", "worker-a", + "--thread", threadID, + "--summary", "Retry policy failed", + "--body", "The HTTP client kept failing integration tests.", + ) + + reconcileOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "reconcile", + "--run", "run_blog_reconcile_001", + ) + + var reconcileResp map[string]any + mustDecodeJSON(t, reconcileOut, &reconcileResp) + updatedTasks := nestedArray(t, reconcileResp, "data", "updated_tasks") + if len(updatedTasks) != 1 { + t.Fatalf("expected one updated task after failed reconcile, got %#v", updatedTasks) + } + task, ok := updatedTasks[0].(map[string]any) + if !ok { + t.Fatalf("expected updated task object, got %#v", updatedTasks[0]) + } + if got, _ := task["task_id"].(string); got != "T1" { + t.Fatalf("expected updated task T1, got %#v", task["task_id"]) + } + if got, _ := task["status"].(string); got != "failed" { + t.Fatalf("expected reconciled task status failed, got %#v", task["status"]) + } + taskCounts, ok := nestedValue(t, reconcileResp, "data", "task_counts").(map[string]any) + if !ok { + t.Fatalf("expected task_counts object, got %#v", nestedValue(t, reconcileResp, "data", "task_counts")) + } + if got, _ := taskCounts["failed"].(float64); got != 1 { + t.Fatalf("expected failed task count 1 after reconcile, got %#v", taskCounts["failed"]) + } + + statusOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "status", + "--run", "run_blog_reconcile_001", + ) + + var statusResp map[string]any + mustDecodeJSON(t, statusOut, &statusResp) + if got := nestedString(t, statusResp, "data", "run", "status"); got != "failed" { + t.Fatalf("expected run status failed after failed reconcile, got %q", got) + } +} + +func TestOrchWorkflowStrictWorktreeDispatchToCleanup(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + repoPath := initGitRepo(t) + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "run", "init", + "--run", "run_blog_workflow_worktree_001", + "--goal", "Validate strict worktree dispatch", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "task", "add", + "--run", "run_blog_workflow_worktree_001", + "--task", "T1", + "--title", "Implement backend", + "--default-to", "worker-a", + ) + + dispatchOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "dispatch", + "--run", "run_blog_workflow_worktree_001", + "--task", "T1", + "--repo-path", repoPath, + "--workspace-root", ".orch/worktrees", + "--strict-worktree", + "--body", "Implement inside isolated worktree.", + ) + + var dispatchResp map[string]any + mustDecodeJSON(t, dispatchOut, &dispatchResp) + threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id") + worktreePath := nestedString(t, dispatchResp, "data", "attempt", "worktree_path") + if worktreePath == "" { + t.Fatal("expected non-empty worktree_path for strict worktree workflow") + } + if got := nestedString(t, dispatchResp, "data", "attempt", "workspace_status"); got != "created" { + t.Fatalf("expected workspace_status created, got %q", got) + } + + runInboxCommand( + t, + "--db", dbPath, + "--json", + "claim", + "--agent", "worker-a", + "--thread", threadID, + ) + runInboxCommand( + t, + "--db", dbPath, + "--json", + "done", + "--agent", "worker-a", + "--thread", threadID, + "--summary", "Backend complete", + ) + runOrchCommand( + t, + "--db", dbPath, + "--json", + "reconcile", + "--run", "run_blog_workflow_worktree_001", + ) + + cleanupOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "cleanup", + "--run", "run_blog_workflow_worktree_001", + "--task", "T1", + "--attempt", "1", + ) + + var cleanupResp map[string]any + mustDecodeJSON(t, cleanupOut, &cleanupResp) + cleaned := nestedArray(t, cleanupResp, "data", "cleaned") + if len(cleaned) != 1 { + t.Fatalf("expected one cleaned attempt, got %#v", cleaned) + } + cleanedAttempt, ok := cleaned[0].(map[string]any) + if !ok { + t.Fatalf("expected cleaned attempt object, got %#v", cleaned[0]) + } + if got, _ := cleanedAttempt["workspace_status"].(string); got != "cleaned" { + t.Fatalf("expected cleaned workspace_status, got %#v", cleanedAttempt["workspace_status"]) + } + if _, err := os.Stat(worktreePath); !os.IsNotExist(err) { + t.Fatalf("expected cleaned worktree path to be removed, err=%v", err) + } +} + +func TestOrchWorkflowCouncilReviewEndToEnd(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + runID := "council_blog_workflow_001" + + startOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "council", "start", + "--run", runID, + "--target", "Review the current blog architecture.", + ) + + var startResp map[string]any + mustDecodeJSON(t, startOut, &startResp) + reviewers := nestedArray(t, startResp, "data", "reviewers") + if len(reviewers) != 3 { + t.Fatalf("expected three reviewers from council start, got %#v", reviewers) + } + + completeCouncilWorkflowReviewersForRemainingTests(t, dbPath, runID) + + waitOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "council", "wait", + "--run", runID, + "--timeout-seconds", "2", + ) + + var waitResp map[string]any + mustDecodeJSON(t, waitOut, &waitResp) + if woke, _ := nestedValue(t, waitResp, "data", "woke").(bool); !woke { + t.Fatalf("expected council wait to wake, got %#v", waitResp) + } + if allComplete, _ := nestedValue(t, waitResp, "data", "all_complete").(bool); !allComplete { + t.Fatalf("expected all reviewers complete, got %#v", waitResp) + } + + tallyOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "council", "tally", + "--run", runID, + "--similarity", "normal", + ) + + var tallyResp map[string]any + mustDecodeJSON(t, tallyOut, &tallyResp) + if got := nestedString(t, tallyResp, "data", "similarity"); got != "normal" { + t.Fatalf("expected normal tally similarity, got %q", got) + } + tallyCounts, ok := nestedValue(t, tallyResp, "data", "counts").(map[string]any) + if !ok { + t.Fatalf("expected tally counts object, got %#v", nestedValue(t, tallyResp, "data", "counts")) + } + if got, _ := tallyCounts["consensus"].(float64); got != 1 { + t.Fatalf("expected one consensus group, got %#v", tallyCounts["consensus"]) + } + if got, _ := tallyCounts["majority"].(float64); got != 1 { + t.Fatalf("expected one majority group, got %#v", tallyCounts["majority"]) + } + if got, _ := tallyCounts["minority"].(float64); got != 1 { + t.Fatalf("expected one minority group, got %#v", tallyCounts["minority"]) + } + + reportOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "council", "report", + "--run", runID, + ) + + var reportResp map[string]any + mustDecodeJSON(t, reportOut, &reportResp) + show := nestedArray(t, reportResp, "data", "show") + if len(show) != 2 || show[0] != "consensus" || show[1] != "majority" { + t.Fatalf("expected default report show [consensus majority], got %#v", show) + } + grouped := nestedArray(t, reportResp, "data", "grouped_recommendations") + if len(grouped) != 2 { + t.Fatalf("expected default report to include consensus and majority groups, got %#v", grouped) + } + artifacts := nestedArray(t, reportResp, "data", "report_artifacts") + if len(artifacts) != 1 { + t.Fatalf("expected one report artifact, got %#v", artifacts) + } + artifact, ok := artifacts[0].(map[string]any) + if !ok { + t.Fatalf("expected report artifact object, got %#v", artifacts[0]) + } + reportPath, _ := artifact["path"].(string) + if reportPath == "" { + t.Fatalf("expected report artifact path, got %#v", artifact["path"]) + } + if _, err := os.Stat(reportPath); err != nil { + t.Fatalf("expected report artifact to exist at %q: %v", reportPath, err) + } +} + +func assertNonEmptyNestedString(t *testing.T, value map[string]any, keys ...string) { + t.Helper() + + if got := nestedString(t, value, keys...); got == "" { + t.Fatalf("expected non-empty string at %v", keys) + } +} + +func completeCouncilWorkflowReviewersForRemainingTests(t *testing.T, dbPath, runID string) { + t.Helper() + + completeCouncilReviewer( + t, + dbPath, + runID, + "architecture-reviewer", + `{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}},{"title":"Share helpers","summary":"Council report rendering paths are repeated.","proposal":"Introduce shared council coordinator helpers for report rendering.","rationale":"This keeps report assembly consistent.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`, + ) + completeCouncilReviewer( + t, + dbPath, + runID, + "implementation-reviewer", + `{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"high","tags":["maintainability"],"target_refs":{"repo_path":"."}},{"title":"Reuse report helpers","summary":"Formatting logic should stay shared.","proposal":"Introduce shared council coordinator helpers for report rendering","rationale":"This avoids formatter drift.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`, + ) + completeCouncilReviewer( + t, + dbPath, + runID, + "risk-reviewer", + `{"reviewer_role":"risk-reviewer","findings":[{"title":"Lock contracts","summary":"Contract drift becomes risky over time.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This reduces integration regressions.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}},{"title":"Cover JSON output","summary":"The council report response should stay stable.","proposal":"Add regression tests for council report JSON output.","rationale":"This catches contract regressions earlier.","confidence":"high","tags":["testing"],"target_refs":{"repo_path":"."}}]}`, + ) +} diff --git a/internal/cli/orch/council_report_contracts_test.go b/internal/cli/orch/council_report_contracts_test.go new file mode 100644 index 0000000..5114b5c --- /dev/null +++ b/internal/cli/orch/council_report_contracts_test.go @@ -0,0 +1,199 @@ +package orch + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestOrchCouncilReportRejectsBeforeTally(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + runID := "council_blog_report_010" + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "council", "start", + "--run", runID, + "--target", "Review the council reporting flow.", + ) + + stdout, _, exitCode := executeOrchCommand( + "--db", dbPath, + "--json", + "council", "report", + "--run", runID, + ) + if exitCode != 30 { + t.Fatalf("expected invalid_state exit code 30, got %d\nstdout:\n%s", exitCode, stdout) + } + assertErrorJSON(t, stdout, "invalid_state") + + if msg := orchErrorMessage(t, stdout); !strings.Contains(msg, "run council tally first") { + t.Fatalf("expected error message to require council tally first, got %q", msg) + } +} + +func TestOrchCouncilReportRejectsInvalidShow(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + runID := "council_blog_report_012" + seedCouncilReportRun(t, dbPath, runID) + + stdout, _, exitCode := executeOrchCommand( + "--db", dbPath, + "--json", + "council", "report", + "--run", runID, + "--show", "consensus,invalid", + ) + if exitCode != 30 { + t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout) + } + assertErrorJSON(t, stdout, "invalid_input") + + msg := orchErrorMessage(t, stdout) + for _, expected := range []string{"consensus", "majority", "minority", "all"} { + if !strings.Contains(msg, expected) { + t.Fatalf("expected invalid --show message to mention %q, got %q", expected, msg) + } + } +} + +func TestOrchCouncilReportDefaultsToConsensusForOnlyUnanimousRun(t *testing.T) { + t.Parallel() + + dbPath := filepath.Join(t.TempDir(), "coord.db") + runID := "council_blog_report_011" + seedOnlyUnanimousCouncilReportRun(t, dbPath, runID) + + reportOut := runOrchCommand( + t, + "--db", dbPath, + "--json", + "council", "report", + "--run", runID, + ) + + var reportResp map[string]any + mustDecodeJSON(t, reportOut, &reportResp) + if ok, _ := reportResp["ok"].(bool); !ok { + t.Fatalf("expected ok=true, got %#v", reportResp) + } + if got := nestedString(t, reportResp, "data", "run_id"); got != runID { + t.Fatalf("expected run id %q, got %q", runID, got) + } + + show := nestedArray(t, reportResp, "data", "show") + if len(show) != 1 || show[0] != "consensus" { + t.Fatalf("expected unanimous-only default show bucket [consensus], got %#v", show) + } + + summary, ok := nestedValue(t, reportResp, "data", "summary").(map[string]any) + if !ok { + t.Fatalf("expected summary object, got %#v", nestedValue(t, reportResp, "data", "summary")) + } + if got, _ := summary["consensus"].(float64); got != 1 { + t.Fatalf("expected one consensus group, got %#v", summary["consensus"]) + } + if got, _ := summary["majority"].(float64); got != 1 { + t.Fatalf("expected one majority group, got %#v", summary["majority"]) + } + if got, _ := summary["minority"].(float64); got != 1 { + t.Fatalf("expected one minority group, got %#v", summary["minority"]) + } + + groups := nestedArray(t, reportResp, "data", "grouped_recommendations") + if len(groups) != 1 { + t.Fatalf("expected one reported recommendation group, got %#v", groups) + } + group, ok := groups[0].(map[string]any) + if !ok { + t.Fatalf("expected grouped recommendation object, got %#v", groups[0]) + } + if got, _ := group["bucket"].(string); got != "consensus" { + t.Fatalf("expected only reported bucket to be consensus, got %#v", group["bucket"]) + } + + artifacts := nestedArray(t, reportResp, "data", "report_artifacts") + if len(artifacts) != 1 { + t.Fatalf("expected one report artifact, got %#v", artifacts) + } + artifact, ok := artifacts[0].(map[string]any) + if !ok { + t.Fatalf("expected report artifact object, got %#v", artifacts[0]) + } + reportPath, _ := artifact["path"].(string) + if reportPath == "" { + t.Fatalf("expected markdown artifact path, got %#v", artifact["path"]) + } + if _, err := os.Stat(reportPath); err != nil { + t.Fatalf("expected markdown artifact to exist at %q: %v", reportPath, err) + } +} + +func orchErrorMessage(t *testing.T, raw string) string { + t.Helper() + + var payload map[string]any + mustDecodeJSON(t, raw, &payload) + errorValue, ok := payload["error"].(map[string]any) + if !ok { + t.Fatalf("expected error object, got %#v", payload["error"]) + } + msg, ok := errorValue["message"].(string) + if !ok { + t.Fatalf("expected error message string, got %#v", errorValue["message"]) + } + return msg +} + +func seedOnlyUnanimousCouncilReportRun(t *testing.T, dbPath, runID string) { + t.Helper() + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "council", "start", + "--run", runID, + "--target", "Review the council reporting flow.", + "--only-unanimous", + ) + + completeCouncilReviewer( + t, + dbPath, + runID, + "architecture-reviewer", + `{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}},{"title":"Share helpers","summary":"Council report rendering paths are repeated.","proposal":"Introduce shared council coordinator helpers for report rendering.","rationale":"This keeps report assembly consistent.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`, + ) + completeCouncilReviewer( + t, + dbPath, + runID, + "implementation-reviewer", + `{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"high","tags":["maintainability"],"target_refs":{"repo_path":"."}},{"title":"Reuse report helpers","summary":"Formatting logic should stay shared.","proposal":"Introduce shared council coordinator helpers for report rendering","rationale":"This avoids formatter drift.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`, + ) + completeCouncilReviewer( + t, + dbPath, + runID, + "risk-reviewer", + `{"reviewer_role":"risk-reviewer","findings":[{"title":"Lock contracts","summary":"Contract drift becomes risky over time.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This reduces integration regressions.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}},{"title":"Cover JSON output","summary":"The council report response should stay stable.","proposal":"Add regression tests for council report JSON output.","rationale":"This catches contract regressions earlier.","confidence":"high","tags":["testing"],"target_refs":{"repo_path":"."}}]}`, + ) + + runOrchCommand( + t, + "--db", dbPath, + "--json", + "council", "tally", + "--run", runID, + "--similarity", "normal", + ) +}