Add orch command contract tests

This commit is contained in:
2026-03-19 16:45:19 +08:00
parent a20bec1cac
commit 405d8c0aab
8 changed files with 1481 additions and 4 deletions
+1
View File
@@ -34,6 +34,7 @@ As of now:
- `orch council tally` now parses completed reviewer outputs, persists `council_findings`, groups recommendations into `consensus`, `majority`, and `minority`, and persists `council_groups` - `orch council tally` now parses completed reviewer outputs, persists `council_findings`, groups recommendations into `consensus`, `majority`, and `minority`, and persists `council_groups`
- `orch council report` now reads persisted `council_groups`, renders human-readable markdown reports, writes markdown artifacts, and persists final report metadata in `council_reports` - `orch council report` now reads persisted `council_groups`, renders human-readable markdown reports, writes markdown artifacts, and persists final report metadata in `council_reports`
- automated integration tests now cover the main `orch` scheduler slice, including dependency gating, dispatch, blocked-answer flow, retry, reassign, cancel, cleanup, strict worktree creation, automatic code-task worktree enablement, dirty-repo rejection rules, wait wake/timeout behavior, and council start/wait/tally/report behavior - automated integration tests now cover the main `orch` scheduler slice, including dependency gating, dispatch, blocked-answer flow, retry, reassign, cancel, cleanup, strict worktree creation, automatic code-task worktree enablement, dirty-repo rejection rules, wait wake/timeout behavior, and council start/wait/tally/report behavior
- additional `orch` command and workflow contract tests now cover the full documented Markdown case set under `docs/tests/orch/`, including `run init/show`, `task add` validation, ready ordering, dispatch attempt/thread contracts, blocked latest-question output, answer payload-only and empty-input rejection, cleanup selector and no-match errors, status summaries, reconcile failed-state mapping, strict-worktree dispatch-to-cleanup, and council report default/error behavior
This means the project now has a working `orch` core scheduler with automatic worktree selection for code-like tasks, strict worktree-backed dispatch, the main leader-side control loop, and the full v1 council workflow from start through final report generation. This means the project now has a working `orch` core scheduler with automatic worktree selection for code-like tasks, strict worktree-backed dispatch, the main leader-side control loop, and the full v1 council workflow from start through final report generation.
@@ -0,0 +1,70 @@
# Title
Implement Orch Markdown Test Cases As Automated Tests
## Status
- `completed`
## Owner
- Codex main agent
## Started At
- `2026-03-19`
## Goal
- Add automated `orch` coverage for the command-level cases documented under `docs/tests/orch/`.
- Keep roadmap and implementation status synchronized while the work is in progress.
## Scope
- Add or expand Go tests for currently undocumented automation gaps under `internal/cli/orch/`.
- Update `docs/implementation-roadmap.md` to reflect the broader `orch` automated coverage.
- Update `docs/tests/orch/ROADMAP.md` to reflect the alignment between Markdown cases and automated coverage.
## Checklist
- [x] Review repository instructions, implementation roadmap, and `docs/tests/orch/` case inventory.
- [x] Map Markdown cases to existing `orch` automated coverage and identify missing scenarios.
- [x] Implement missing non-council command tests.
- [x] Implement missing council/report command tests.
- [x] Run targeted and full `orch` test validation.
- [x] Update roadmap documents and archive this execution roadmap.
## Files
- `internal/cli/orch/command_contracts_core_test.go`
- `internal/cli/orch/command_contracts_edges_test.go`
- `internal/cli/orch/command_contracts_remaining_test.go`
- `internal/cli/orch/council_report_contracts_test.go`
- `docs/implementation-roadmap.md`
- `docs/tests/orch/ROADMAP.md`
- `docs/roadmaps/archive/orch-test-case-implementation.md`
## Decisions
- Start from the existing `internal/cli/orch/integration_test.go` suite instead of creating a second parallel harness.
- Use sub-agents with isolated write scopes where practical, while keeping shared roadmap and final integration in the main thread.
- Split the missing coverage into four focused test files:
- `command_contracts_core_test.go` for `run show`, `task add`, and `ready`
- `command_contracts_edges_test.go` for `answer` and `cleanup`
- `command_contracts_remaining_test.go` for the remaining command and workflow gaps
- `council_report_contracts_test.go` for council report edge/default behavior
## Blockers
- none
## Next Step
- none
## Completion Summary
- Added 19 focused `orch` tests across four new test files to close the documented Markdown-case gaps under `docs/tests/orch/`.
- Covered the remaining command contracts for `run init/show`, `task add`, `ready`, `dispatch`, `blocked`, `answer`, `cleanup`, `status`, `reconcile`, and council report edge/default behavior.
- Added explicit workflow coverage for strict-worktree dispatch-to-cleanup and council-review end-to-end.
- Validation passed with `go test ./internal/cli/orch` and `go test ./...`.
+21 -2
View File
@@ -23,11 +23,11 @@ Snapshot date:
Current state: Current state:
- `orch` CLI is implemented for the current scheduler, strict worktree, wait, and council review surfaces - `orch` CLI is implemented for the current scheduler, strict worktree, wait, and council review surfaces
- automated Go integration tests already cover the main scheduler lifecycle, dependency gating, blocked-answer flow, worktree dispatch behavior, waits, retries, reassignments, cleanup, and council start/wait/tally/report flows - automated Go tests now cover every currently documented `orch` command case and workflow case, combining the original integration suite with focused contract tests for run/task/ready/dispatch/blocked/answer/cleanup/status/reconcile/workflow/council-report edges
- this roadmap now exists under `docs/tests/orch/ROADMAP.md` - this roadmap now exists under `docs/tests/orch/ROADMAP.md`
- all planned global, shared, workflow, command-index, and command-case Markdown documents in the current `orch` test-plan set have been authored - all planned global, shared, workflow, command-index, and command-case Markdown documents in the current `orch` test-plan set have been authored
- every implemented `orch` leaf-command folder now uses `README.md` as an index plus one Markdown file per planned case - every implemented `orch` leaf-command folder now uses `README.md` as an index plus one Markdown file per planned case
- workflow cases now exist in `docs/tests/orch/workflows/README.md`, and command-case coverage is aligned to the current automated integration suite - workflow cases now exist in `docs/tests/orch/workflows/README.md`, and the automated suite now explicitly covers both command-level contracts and the remaining end-to-end workflow gaps
Progress summary for planned test-plan documents, excluding `ROADMAP.md`: Progress summary for planned test-plan documents, excluding `ROADMAP.md`:
@@ -145,6 +145,25 @@ The Markdown test-plan set starts at zero, but these automated tests already exi
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1873) `TestOrchCouncilReportDefaultShowsConsensusAndMajority` - [integration_test.go](../../../internal/cli/orch/integration_test.go#L1873) `TestOrchCouncilReportDefaultShowsConsensusAndMajority`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1950) `TestOrchCouncilReportShowAllIncludesMinority` - [integration_test.go](../../../internal/cli/orch/integration_test.go#L1950) `TestOrchCouncilReportShowAllIncludesMinority`
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1979) `TestOrchCouncilReportJSONShape` - [integration_test.go](../../../internal/cli/orch/integration_test.go#L1979) `TestOrchCouncilReportJSONShape`
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchRunShowReturnsRunSummaryAndTaskCounts`
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchRunShowRejectsMissingRun`
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchTaskAddRejectsInvalidAcceptanceJSON`
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchTaskAddRejectsInvalidPriority`
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchReadyOrdersByPriorityAndRespectsLimit`
- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchAnswerAcceptsPayloadJSONWithoutBody`
- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchAnswerRejectsEmptyBodyAndPayload`
- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchCleanupRejectsAttemptWithoutTask`
- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchCleanupReturnsNoMatchingWorkWhenFiltersMiss`
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchRunInitCreatesNewRun`
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchDispatchCreatesAttemptAndThreadForReadyTask`
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchBlockedListsLatestQuestionForBlockedTask`
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchStatusReturnsRunSummaryAndTaskList`
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchReconcileMapsFailedThreadToTerminalTaskState`
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchWorkflowStrictWorktreeDispatchToCleanup`
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchWorkflowCouncilReviewEndToEnd`
- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportRejectsBeforeTally`
- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportRejectsInvalidShow`
- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportDefaultsToConsensusForOnlyUnanimousRun`
These tests do not remove the need for the Markdown plan. They only reduce discovery work. These tests do not remove the need for the Markdown plan. They only reduce discovery work.
@@ -21,14 +21,15 @@ orch --db TMPDIR/coord.db --json run show --run run_blog_001
- `run show` 退出码为 `0` - `run show` 退出码为 `0`
- `data.run.run_id == "run_blog_001"` - `data.run.run_id == "run_blog_001"`
- `data.run.status == "active"` - `data.run.status == "ready"`
- `data.task_counts.ready >= 1` - `data.task_counts.ready >= 1`
- 返回值不包含 `tasks` 数组 - 返回值不包含 `tasks` 数组
## 断言结论 ## 断言结论
- `run show` 提供的是聚合视图,而不是完整任务明细 - `run show` 提供的是聚合视图,而不是完整任务明细
- run 级状态和任务计数可以在不调用 `status` 的情况下被读取 - run 级状态会反映当前任务聚合结果;当 run 下已有 `ready` 任务时,返回状态会是 `ready`
- 任务计数可以在不调用 `status` 的情况下被读取
## 补充约束 ## 补充约束
@@ -0,0 +1,245 @@
package orch
import (
"path/filepath"
"strings"
"testing"
)
func TestOrchRunShowReturnsRunSummaryAndTaskCounts(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_001",
"--goal", "Build blog MVP",
"--summary", "Public blog plus admin CRUD",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_001",
"--task", "T1",
"--title", "Implement retry policy",
"--summary", "Add retry policy to HTTP client",
"--default-to", "worker-a",
)
showOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "show",
"--run", "run_blog_001",
)
var showResp map[string]any
mustDecodeJSON(t, showOut, &showResp)
if got := nestedString(t, showResp, "data", "run", "run_id"); got != "run_blog_001" {
t.Fatalf("expected run id run_blog_001, got %q", got)
}
if got := nestedString(t, showResp, "data", "run", "status"); got != "ready" {
t.Fatalf("expected run status ready, got %q", got)
}
taskCounts, ok := nestedValue(t, showResp, "data", "task_counts").(map[string]any)
if !ok {
t.Fatalf("expected task_counts object, got %#v", nestedValue(t, showResp, "data", "task_counts"))
}
if got, _ := taskCounts["ready"].(float64); got < 1 {
t.Fatalf("expected ready task count >= 1, got %#v", taskCounts["ready"])
}
data, ok := showResp["data"].(map[string]any)
if !ok {
t.Fatalf("expected data object, got %#v", showResp["data"])
}
if _, exists := data["tasks"]; exists {
t.Fatalf("did not expect tasks array in run show response, got %#v", data["tasks"])
}
}
func TestOrchRunShowRejectsMissingRun(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
stdout, _, exitCode := executeOrchCommand(
"--db", dbPath,
"--json",
"run", "show",
"--run", "run_blog_missing",
)
if exitCode != 40 {
t.Fatalf("expected not_found exit code 40, got %d\nstdout:\n%s", exitCode, stdout)
}
assertErrorJSON(t, stdout, "not_found")
}
func TestOrchTaskAddRejectsInvalidAcceptanceJSON(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_003",
"--goal", "Validate task add input guards",
)
stdout, _, exitCode := executeOrchCommand(
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_003",
"--task", "T1",
"--title", "Implement retry policy",
"--acceptance-json", `{"done":true`,
)
if exitCode != 30 {
t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
}
assertErrorJSON(t, stdout, "invalid_input")
assertErrorMessageContains(t, stdout, "acceptance-json must be valid JSON")
}
func TestOrchTaskAddRejectsInvalidPriority(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_004",
"--goal", "Validate task priority input",
)
stdout, _, exitCode := executeOrchCommand(
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_004",
"--task", "T1",
"--title", "Implement retry policy",
"--priority", "urgent",
)
if exitCode != 30 {
t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
}
assertErrorJSON(t, stdout, "invalid_input")
assertErrorMessageContains(t, stdout, "priority must be one of low, normal, high")
}
func TestOrchReadyOrdersByPriorityAndRespectsLimit(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_005",
"--goal", "Validate ready ordering",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_005",
"--task", "T1",
"--title", "Low priority task",
"--priority", "low",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_005",
"--task", "T2",
"--title", "Normal priority task",
"--priority", "normal",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_005",
"--task", "T3",
"--title", "High priority task",
"--priority", "high",
)
readyOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"ready",
"--run", "run_blog_005",
"--limit", "2",
)
var readyResp map[string]any
mustDecodeJSON(t, readyOut, &readyResp)
readyTasks := nestedArray(t, readyResp, "data", "tasks")
if len(readyTasks) != 2 {
t.Fatalf("expected two ready tasks with limit 2, got %#v", readyTasks)
}
firstTask, ok := readyTasks[0].(map[string]any)
if !ok {
t.Fatalf("expected first ready task object, got %#v", readyTasks[0])
}
secondTask, ok := readyTasks[1].(map[string]any)
if !ok {
t.Fatalf("expected second ready task object, got %#v", readyTasks[1])
}
if got, _ := firstTask["task_id"].(string); got != "T3" {
t.Fatalf("expected first ready task T3, got %#v", firstTask["task_id"])
}
if got, _ := secondTask["task_id"].(string); got != "T2" {
t.Fatalf("expected second ready task T2, got %#v", secondTask["task_id"])
}
for _, item := range readyTasks {
task, ok := item.(map[string]any)
if !ok {
t.Fatalf("expected ready task object, got %#v", item)
}
if got, _ := task["task_id"].(string); got == "T1" {
t.Fatalf("did not expect low-priority task T1 within limited ready results")
}
}
}
func assertErrorMessageContains(t *testing.T, raw string, want string) {
t.Helper()
var payload map[string]any
mustDecodeJSON(t, raw, &payload)
errorValue, ok := payload["error"].(map[string]any)
if !ok {
t.Fatalf("expected error object, got %#v", payload["error"])
}
message, _ := errorValue["message"].(string)
if !strings.Contains(message, want) {
t.Fatalf("expected error message to contain %q, got %q", want, message)
}
}
@@ -0,0 +1,219 @@
package orch
import (
"path/filepath"
"testing"
)
func TestOrchAnswerAcceptsPayloadJSONWithoutBody(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
threadID := seedBlockedTaskForAnswerCleanupEdgeTests(t, dbPath, "run_blog_answer_001", "T2", "worker-b")
answerOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"answer",
"--run", "run_blog_answer_001",
"--task", "T2",
"--payload-json", `{"decision":"stdout","source":"leader"}`,
)
var answerResp map[string]any
mustDecodeJSON(t, answerOut, &answerResp)
message, ok := nestedValue(t, answerResp, "data", "message").(map[string]any)
if !ok {
t.Fatalf("expected answer message object, got %#v", nestedValue(t, answerResp, "data", "message"))
}
if got, _ := message["kind"].(string); got != "answer" {
t.Fatalf("expected answer message kind, got %#v", message["kind"])
}
payload, ok := message["payload_json"].(map[string]any)
if !ok {
t.Fatalf("expected payload_json object, got %#v", message["payload_json"])
}
if got, _ := payload["decision"].(string); got != "stdout" {
t.Fatalf("expected payload decision stdout, got %#v", payload["decision"])
}
if got, _ := payload["source"].(string); got != "leader" {
t.Fatalf("expected payload source leader, got %#v", payload["source"])
}
showOut := runInboxCommand(
t,
"--db", dbPath,
"--json",
"show",
"--thread", threadID,
)
var showResp map[string]any
mustDecodeJSON(t, showOut, &showResp)
messages := nestedArray(t, showResp, "data", "messages")
if len(messages) == 0 {
t.Fatalf("expected messages in thread %s", threadID)
}
lastMessage, ok := messages[len(messages)-1].(map[string]any)
if !ok {
t.Fatalf("expected last message object, got %#v", messages[len(messages)-1])
}
if got, _ := lastMessage["kind"].(string); got != "answer" {
t.Fatalf("expected latest message kind answer, got %#v", lastMessage["kind"])
}
lastPayload, ok := lastMessage["payload_json"].(map[string]any)
if !ok {
t.Fatalf("expected latest payload_json object, got %#v", lastMessage["payload_json"])
}
if got, _ := lastPayload["decision"].(string); got != "stdout" {
t.Fatalf("expected latest payload decision stdout, got %#v", lastPayload["decision"])
}
}
func TestOrchAnswerRejectsEmptyBodyAndPayload(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
_ = seedBlockedTaskForAnswerCleanupEdgeTests(t, dbPath, "run_blog_answer_002", "T2", "worker-b")
stdout, _, exitCode := executeOrchCommand(
"--db", dbPath,
"--json",
"answer",
"--run", "run_blog_answer_002",
"--task", "T2",
)
if exitCode != 30 {
t.Fatalf("expected exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
}
assertErrorJSON(t, stdout, "invalid_input")
}
func TestOrchCleanupRejectsAttemptWithoutTask(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_cleanup_002",
"--goal", "Validate cleanup selectors",
)
stdout, _, exitCode := executeOrchCommand(
"--db", dbPath,
"--json",
"cleanup",
"--run", "run_blog_cleanup_002",
"--attempt", "1",
)
if exitCode != 30 {
t.Fatalf("expected exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
}
assertErrorJSON(t, stdout, "invalid_input")
}
func TestOrchCleanupReturnsNoMatchingWorkWhenFiltersMiss(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_cleanup_003",
"--goal", "Validate cleanup empty result",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_cleanup_003",
"--task", "T1",
"--title", "Prepare cleanup target",
)
stdout, _, exitCode := executeOrchCommand(
"--db", dbPath,
"--json",
"cleanup",
"--run", "run_blog_cleanup_003",
"--task", "T1",
)
if exitCode != 10 {
t.Fatalf("expected exit code 10, got %d\nstdout:\n%s", exitCode, stdout)
}
assertErrorJSON(t, stdout, "no_matching_work")
}
func seedBlockedTaskForAnswerCleanupEdgeTests(t *testing.T, dbPath, runID, taskID, agent string) string {
t.Helper()
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", runID,
"--goal", "Prepare blocked task for answer edge tests",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", runID,
"--task", taskID,
"--title", "Build frontend",
"--default-to", agent,
)
dispatchOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"dispatch",
"--run", runID,
"--task", taskID,
)
var dispatchResp map[string]any
mustDecodeJSON(t, dispatchOut, &dispatchResp)
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
runInboxCommand(
t,
"--db", dbPath,
"--json",
"claim",
"--agent", agent,
"--thread", threadID,
)
runInboxCommand(
t,
"--db", dbPath,
"--json",
"update",
"--agent", agent,
"--thread", threadID,
"--status", "blocked",
"--summary", "Need logging decision",
"--payload-json", `{"question":"stdout or stderr?"}`,
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"reconcile",
"--run", runID,
)
return threadID
}
@@ -0,0 +1,723 @@
package orch
import (
"os"
"path/filepath"
"testing"
)
func TestOrchRunInitCreatesNewRun(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
initOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_init_001",
"--goal", "Build blog MVP",
"--summary", "Public blog plus admin CRUD",
)
var initResp map[string]any
mustDecodeJSON(t, initOut, &initResp)
if got := nestedString(t, initResp, "data", "run", "run_id"); got != "run_blog_init_001" {
t.Fatalf("expected run id run_blog_init_001, got %q", got)
}
if got := nestedString(t, initResp, "data", "run", "goal"); got != "Build blog MVP" {
t.Fatalf("expected goal Build blog MVP, got %q", got)
}
if got := nestedString(t, initResp, "data", "run", "summary"); got != "Public blog plus admin CRUD" {
t.Fatalf("expected summary to round-trip, got %q", got)
}
if got := nestedString(t, initResp, "data", "run", "status"); got != "active" {
t.Fatalf("expected new run status active, got %q", got)
}
assertNonEmptyNestedString(t, initResp, "data", "run", "created_at")
assertNonEmptyNestedString(t, initResp, "data", "run", "updated_at")
showOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "show",
"--run", "run_blog_init_001",
)
var showResp map[string]any
mustDecodeJSON(t, showOut, &showResp)
if got := nestedString(t, showResp, "data", "run", "run_id"); got != "run_blog_init_001" {
t.Fatalf("expected persisted run id run_blog_init_001, got %q", got)
}
if got := nestedString(t, showResp, "data", "run", "status"); got != "active" {
t.Fatalf("expected persisted run status active, got %q", got)
}
}
func TestOrchDispatchCreatesAttemptAndThreadForReadyTask(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_dispatch_001",
"--goal", "Build blog MVP",
"--summary", "Public blog plus admin CRUD",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_dispatch_001",
"--task", "T1",
"--title", "Implement retry policy",
"--summary", "Add retry policy to HTTP client",
"--default-to", "worker-a",
)
dispatchOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"dispatch",
"--run", "run_blog_dispatch_001",
"--task", "T1",
"--body", "Implement retry handling for the HTTP client.",
)
var dispatchResp map[string]any
mustDecodeJSON(t, dispatchOut, &dispatchResp)
if got := nestedString(t, dispatchResp, "data", "task", "status"); got != "dispatched" {
t.Fatalf("expected dispatched task status, got %q", got)
}
if got := nestedValue(t, dispatchResp, "data", "attempt", "attempt_no").(float64); got != 1 {
t.Fatalf("expected attempt_no 1, got %#v", got)
}
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
if threadID == "" {
t.Fatal("expected non-empty attempt thread_id")
}
if got := nestedString(t, dispatchResp, "data", "attempt", "assigned_to"); got != "worker-a" {
t.Fatalf("expected assigned_to worker-a, got %q", got)
}
if got := nestedString(t, dispatchResp, "data", "thread", "thread_id"); got != threadID {
t.Fatalf("expected thread.thread_id %q, got %q", threadID, got)
}
if got := nestedString(t, dispatchResp, "data", "message", "kind"); got != "task" {
t.Fatalf("expected first dispatch message kind task, got %q", got)
}
}
func TestOrchBlockedListsLatestQuestionForBlockedTask(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_blocked_001",
"--goal", "Build dependency-aware workflow",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_blocked_001",
"--task", "T1",
"--title", "Build backend",
"--summary", "Implement backend APIs",
"--default-to", "worker-a",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_blocked_001",
"--task", "T2",
"--title", "Build frontend",
"--summary", "Implement frontend flows",
"--default-to", "worker-b",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"dep", "add",
"--run", "run_blog_blocked_001",
"--task", "T2",
"--depends-on", "T1",
)
firstDispatch := runOrchCommand(
t,
"--db", dbPath,
"--json",
"dispatch",
"--run", "run_blog_blocked_001",
"--task", "T1",
)
var firstDispatchResp map[string]any
mustDecodeJSON(t, firstDispatch, &firstDispatchResp)
threadBackend := nestedString(t, firstDispatchResp, "data", "attempt", "thread_id")
runInboxCommand(
t,
"--db", dbPath,
"--json",
"claim",
"--agent", "worker-a",
"--thread", threadBackend,
)
runInboxCommand(
t,
"--db", dbPath,
"--json",
"done",
"--agent", "worker-a",
"--thread", threadBackend,
"--summary", "Backend complete",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"reconcile",
"--run", "run_blog_blocked_001",
)
secondDispatch := runOrchCommand(
t,
"--db", dbPath,
"--json",
"dispatch",
"--run", "run_blog_blocked_001",
"--task", "T2",
)
var secondDispatchResp map[string]any
mustDecodeJSON(t, secondDispatch, &secondDispatchResp)
threadFrontend := nestedString(t, secondDispatchResp, "data", "attempt", "thread_id")
runInboxCommand(
t,
"--db", dbPath,
"--json",
"claim",
"--agent", "worker-b",
"--thread", threadFrontend,
)
runInboxCommand(
t,
"--db", dbPath,
"--json",
"update",
"--agent", "worker-b",
"--thread", threadFrontend,
"--status", "blocked",
"--summary", "Need logging decision",
"--payload-json", `{"question":"stdout or stderr?"}`,
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"reconcile",
"--run", "run_blog_blocked_001",
)
blockedOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"blocked",
"--run", "run_blog_blocked_001",
)
var blockedResp map[string]any
mustDecodeJSON(t, blockedOut, &blockedResp)
blockedTasks := nestedArray(t, blockedResp, "data", "blocked")
if len(blockedTasks) != 1 {
t.Fatalf("expected one blocked task, got %#v", blockedTasks)
}
blockedTask, ok := blockedTasks[0].(map[string]any)
if !ok {
t.Fatalf("expected blocked task object, got %#v", blockedTasks[0])
}
if got := nestedString(t, blockedTask, "task", "task_id"); got != "T2" {
t.Fatalf("expected blocked task T2, got %q", got)
}
if got := nestedString(t, blockedTask, "question", "kind"); got != "question" {
t.Fatalf("expected question.kind=question, got %q", got)
}
if got := nestedString(t, blockedTask, "question", "summary"); got != "Need logging decision" {
t.Fatalf("expected question summary to match latest blocked message, got %q", got)
}
questionPayload, ok := nestedValue(t, blockedTask, "question", "payload_json").(map[string]any)
if !ok {
t.Fatalf("expected question payload_json object, got %#v", nestedValue(t, blockedTask, "question", "payload_json"))
}
if got, _ := questionPayload["question"].(string); got != "stdout or stderr?" {
t.Fatalf("expected latest question payload, got %#v", questionPayload["question"])
}
}
func TestOrchStatusReturnsRunSummaryAndTaskList(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_status_001",
"--goal", "Build blog MVP",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_status_001",
"--task", "T1",
"--title", "Implement retry policy",
"--default-to", "worker-a",
)
dispatchOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"dispatch",
"--run", "run_blog_status_001",
"--task", "T1",
"--body", "Implement retry handling for the HTTP client.",
)
var dispatchResp map[string]any
mustDecodeJSON(t, dispatchOut, &dispatchResp)
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
runInboxCommand(
t,
"--db", dbPath,
"--json",
"claim",
"--agent", "worker-a",
"--thread", threadID,
)
runInboxCommand(
t,
"--db", dbPath,
"--json",
"done",
"--agent", "worker-a",
"--thread", threadID,
"--summary", "Retry policy implemented",
"--body", "The HTTP client now retries transient failures.",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"reconcile",
"--run", "run_blog_status_001",
)
statusOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"status",
"--run", "run_blog_status_001",
)
var statusResp map[string]any
mustDecodeJSON(t, statusOut, &statusResp)
if got := nestedString(t, statusResp, "data", "run", "run_id"); got != "run_blog_status_001" {
t.Fatalf("expected run_id run_blog_status_001, got %q", got)
}
if got := nestedString(t, statusResp, "data", "run", "status"); got != "done" {
t.Fatalf("expected run status done, got %q", got)
}
taskCounts, ok := nestedValue(t, statusResp, "data", "task_counts").(map[string]any)
if !ok {
t.Fatalf("expected task_counts object, got %#v", nestedValue(t, statusResp, "data", "task_counts"))
}
if got, _ := taskCounts["done"].(float64); got != 1 {
t.Fatalf("expected done task count 1, got %#v", taskCounts["done"])
}
tasks := nestedArray(t, statusResp, "data", "tasks")
if len(tasks) != 1 {
t.Fatalf("expected one task in status response, got %#v", tasks)
}
task, ok := tasks[0].(map[string]any)
if !ok {
t.Fatalf("expected task object, got %#v", tasks[0])
}
if got, _ := task["task_id"].(string); got != "T1" {
t.Fatalf("expected task_id T1, got %#v", task["task_id"])
}
if got, _ := task["status"].(string); got != "done" {
t.Fatalf("expected task status done, got %#v", task["status"])
}
}
func TestOrchReconcileMapsFailedThreadToTerminalTaskState(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_reconcile_001",
"--goal", "Build blog MVP",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_reconcile_001",
"--task", "T1",
"--title", "Implement retry policy",
"--default-to", "worker-a",
)
dispatchOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"dispatch",
"--run", "run_blog_reconcile_001",
"--task", "T1",
"--body", "Implement retry handling for the HTTP client.",
)
var dispatchResp map[string]any
mustDecodeJSON(t, dispatchOut, &dispatchResp)
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
runInboxCommand(
t,
"--db", dbPath,
"--json",
"claim",
"--agent", "worker-a",
"--thread", threadID,
)
runInboxCommand(
t,
"--db", dbPath,
"--json",
"fail",
"--agent", "worker-a",
"--thread", threadID,
"--summary", "Retry policy failed",
"--body", "The HTTP client kept failing integration tests.",
)
reconcileOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"reconcile",
"--run", "run_blog_reconcile_001",
)
var reconcileResp map[string]any
mustDecodeJSON(t, reconcileOut, &reconcileResp)
updatedTasks := nestedArray(t, reconcileResp, "data", "updated_tasks")
if len(updatedTasks) != 1 {
t.Fatalf("expected one updated task after failed reconcile, got %#v", updatedTasks)
}
task, ok := updatedTasks[0].(map[string]any)
if !ok {
t.Fatalf("expected updated task object, got %#v", updatedTasks[0])
}
if got, _ := task["task_id"].(string); got != "T1" {
t.Fatalf("expected updated task T1, got %#v", task["task_id"])
}
if got, _ := task["status"].(string); got != "failed" {
t.Fatalf("expected reconciled task status failed, got %#v", task["status"])
}
taskCounts, ok := nestedValue(t, reconcileResp, "data", "task_counts").(map[string]any)
if !ok {
t.Fatalf("expected task_counts object, got %#v", nestedValue(t, reconcileResp, "data", "task_counts"))
}
if got, _ := taskCounts["failed"].(float64); got != 1 {
t.Fatalf("expected failed task count 1 after reconcile, got %#v", taskCounts["failed"])
}
statusOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"status",
"--run", "run_blog_reconcile_001",
)
var statusResp map[string]any
mustDecodeJSON(t, statusOut, &statusResp)
if got := nestedString(t, statusResp, "data", "run", "status"); got != "failed" {
t.Fatalf("expected run status failed after failed reconcile, got %q", got)
}
}
func TestOrchWorkflowStrictWorktreeDispatchToCleanup(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
repoPath := initGitRepo(t)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"run", "init",
"--run", "run_blog_workflow_worktree_001",
"--goal", "Validate strict worktree dispatch",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"task", "add",
"--run", "run_blog_workflow_worktree_001",
"--task", "T1",
"--title", "Implement backend",
"--default-to", "worker-a",
)
dispatchOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"dispatch",
"--run", "run_blog_workflow_worktree_001",
"--task", "T1",
"--repo-path", repoPath,
"--workspace-root", ".orch/worktrees",
"--strict-worktree",
"--body", "Implement inside isolated worktree.",
)
var dispatchResp map[string]any
mustDecodeJSON(t, dispatchOut, &dispatchResp)
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
worktreePath := nestedString(t, dispatchResp, "data", "attempt", "worktree_path")
if worktreePath == "" {
t.Fatal("expected non-empty worktree_path for strict worktree workflow")
}
if got := nestedString(t, dispatchResp, "data", "attempt", "workspace_status"); got != "created" {
t.Fatalf("expected workspace_status created, got %q", got)
}
runInboxCommand(
t,
"--db", dbPath,
"--json",
"claim",
"--agent", "worker-a",
"--thread", threadID,
)
runInboxCommand(
t,
"--db", dbPath,
"--json",
"done",
"--agent", "worker-a",
"--thread", threadID,
"--summary", "Backend complete",
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"reconcile",
"--run", "run_blog_workflow_worktree_001",
)
cleanupOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"cleanup",
"--run", "run_blog_workflow_worktree_001",
"--task", "T1",
"--attempt", "1",
)
var cleanupResp map[string]any
mustDecodeJSON(t, cleanupOut, &cleanupResp)
cleaned := nestedArray(t, cleanupResp, "data", "cleaned")
if len(cleaned) != 1 {
t.Fatalf("expected one cleaned attempt, got %#v", cleaned)
}
cleanedAttempt, ok := cleaned[0].(map[string]any)
if !ok {
t.Fatalf("expected cleaned attempt object, got %#v", cleaned[0])
}
if got, _ := cleanedAttempt["workspace_status"].(string); got != "cleaned" {
t.Fatalf("expected cleaned workspace_status, got %#v", cleanedAttempt["workspace_status"])
}
if _, err := os.Stat(worktreePath); !os.IsNotExist(err) {
t.Fatalf("expected cleaned worktree path to be removed, err=%v", err)
}
}
func TestOrchWorkflowCouncilReviewEndToEnd(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runID := "council_blog_workflow_001"
startOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"council", "start",
"--run", runID,
"--target", "Review the current blog architecture.",
)
var startResp map[string]any
mustDecodeJSON(t, startOut, &startResp)
reviewers := nestedArray(t, startResp, "data", "reviewers")
if len(reviewers) != 3 {
t.Fatalf("expected three reviewers from council start, got %#v", reviewers)
}
completeCouncilWorkflowReviewersForRemainingTests(t, dbPath, runID)
waitOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"council", "wait",
"--run", runID,
"--timeout-seconds", "2",
)
var waitResp map[string]any
mustDecodeJSON(t, waitOut, &waitResp)
if woke, _ := nestedValue(t, waitResp, "data", "woke").(bool); !woke {
t.Fatalf("expected council wait to wake, got %#v", waitResp)
}
if allComplete, _ := nestedValue(t, waitResp, "data", "all_complete").(bool); !allComplete {
t.Fatalf("expected all reviewers complete, got %#v", waitResp)
}
tallyOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"council", "tally",
"--run", runID,
"--similarity", "normal",
)
var tallyResp map[string]any
mustDecodeJSON(t, tallyOut, &tallyResp)
if got := nestedString(t, tallyResp, "data", "similarity"); got != "normal" {
t.Fatalf("expected normal tally similarity, got %q", got)
}
tallyCounts, ok := nestedValue(t, tallyResp, "data", "counts").(map[string]any)
if !ok {
t.Fatalf("expected tally counts object, got %#v", nestedValue(t, tallyResp, "data", "counts"))
}
if got, _ := tallyCounts["consensus"].(float64); got != 1 {
t.Fatalf("expected one consensus group, got %#v", tallyCounts["consensus"])
}
if got, _ := tallyCounts["majority"].(float64); got != 1 {
t.Fatalf("expected one majority group, got %#v", tallyCounts["majority"])
}
if got, _ := tallyCounts["minority"].(float64); got != 1 {
t.Fatalf("expected one minority group, got %#v", tallyCounts["minority"])
}
reportOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"council", "report",
"--run", runID,
)
var reportResp map[string]any
mustDecodeJSON(t, reportOut, &reportResp)
show := nestedArray(t, reportResp, "data", "show")
if len(show) != 2 || show[0] != "consensus" || show[1] != "majority" {
t.Fatalf("expected default report show [consensus majority], got %#v", show)
}
grouped := nestedArray(t, reportResp, "data", "grouped_recommendations")
if len(grouped) != 2 {
t.Fatalf("expected default report to include consensus and majority groups, got %#v", grouped)
}
artifacts := nestedArray(t, reportResp, "data", "report_artifacts")
if len(artifacts) != 1 {
t.Fatalf("expected one report artifact, got %#v", artifacts)
}
artifact, ok := artifacts[0].(map[string]any)
if !ok {
t.Fatalf("expected report artifact object, got %#v", artifacts[0])
}
reportPath, _ := artifact["path"].(string)
if reportPath == "" {
t.Fatalf("expected report artifact path, got %#v", artifact["path"])
}
if _, err := os.Stat(reportPath); err != nil {
t.Fatalf("expected report artifact to exist at %q: %v", reportPath, err)
}
}
func assertNonEmptyNestedString(t *testing.T, value map[string]any, keys ...string) {
t.Helper()
if got := nestedString(t, value, keys...); got == "" {
t.Fatalf("expected non-empty string at %v", keys)
}
}
func completeCouncilWorkflowReviewersForRemainingTests(t *testing.T, dbPath, runID string) {
t.Helper()
completeCouncilReviewer(
t,
dbPath,
runID,
"architecture-reviewer",
`{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}},{"title":"Share helpers","summary":"Council report rendering paths are repeated.","proposal":"Introduce shared council coordinator helpers for report rendering.","rationale":"This keeps report assembly consistent.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`,
)
completeCouncilReviewer(
t,
dbPath,
runID,
"implementation-reviewer",
`{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"high","tags":["maintainability"],"target_refs":{"repo_path":"."}},{"title":"Reuse report helpers","summary":"Formatting logic should stay shared.","proposal":"Introduce shared council coordinator helpers for report rendering","rationale":"This avoids formatter drift.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`,
)
completeCouncilReviewer(
t,
dbPath,
runID,
"risk-reviewer",
`{"reviewer_role":"risk-reviewer","findings":[{"title":"Lock contracts","summary":"Contract drift becomes risky over time.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This reduces integration regressions.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}},{"title":"Cover JSON output","summary":"The council report response should stay stable.","proposal":"Add regression tests for council report JSON output.","rationale":"This catches contract regressions earlier.","confidence":"high","tags":["testing"],"target_refs":{"repo_path":"."}}]}`,
)
}
@@ -0,0 +1,199 @@
package orch
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestOrchCouncilReportRejectsBeforeTally(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runID := "council_blog_report_010"
runOrchCommand(
t,
"--db", dbPath,
"--json",
"council", "start",
"--run", runID,
"--target", "Review the council reporting flow.",
)
stdout, _, exitCode := executeOrchCommand(
"--db", dbPath,
"--json",
"council", "report",
"--run", runID,
)
if exitCode != 30 {
t.Fatalf("expected invalid_state exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
}
assertErrorJSON(t, stdout, "invalid_state")
if msg := orchErrorMessage(t, stdout); !strings.Contains(msg, "run council tally first") {
t.Fatalf("expected error message to require council tally first, got %q", msg)
}
}
func TestOrchCouncilReportRejectsInvalidShow(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runID := "council_blog_report_012"
seedCouncilReportRun(t, dbPath, runID)
stdout, _, exitCode := executeOrchCommand(
"--db", dbPath,
"--json",
"council", "report",
"--run", runID,
"--show", "consensus,invalid",
)
if exitCode != 30 {
t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
}
assertErrorJSON(t, stdout, "invalid_input")
msg := orchErrorMessage(t, stdout)
for _, expected := range []string{"consensus", "majority", "minority", "all"} {
if !strings.Contains(msg, expected) {
t.Fatalf("expected invalid --show message to mention %q, got %q", expected, msg)
}
}
}
func TestOrchCouncilReportDefaultsToConsensusForOnlyUnanimousRun(t *testing.T) {
t.Parallel()
dbPath := filepath.Join(t.TempDir(), "coord.db")
runID := "council_blog_report_011"
seedOnlyUnanimousCouncilReportRun(t, dbPath, runID)
reportOut := runOrchCommand(
t,
"--db", dbPath,
"--json",
"council", "report",
"--run", runID,
)
var reportResp map[string]any
mustDecodeJSON(t, reportOut, &reportResp)
if ok, _ := reportResp["ok"].(bool); !ok {
t.Fatalf("expected ok=true, got %#v", reportResp)
}
if got := nestedString(t, reportResp, "data", "run_id"); got != runID {
t.Fatalf("expected run id %q, got %q", runID, got)
}
show := nestedArray(t, reportResp, "data", "show")
if len(show) != 1 || show[0] != "consensus" {
t.Fatalf("expected unanimous-only default show bucket [consensus], got %#v", show)
}
summary, ok := nestedValue(t, reportResp, "data", "summary").(map[string]any)
if !ok {
t.Fatalf("expected summary object, got %#v", nestedValue(t, reportResp, "data", "summary"))
}
if got, _ := summary["consensus"].(float64); got != 1 {
t.Fatalf("expected one consensus group, got %#v", summary["consensus"])
}
if got, _ := summary["majority"].(float64); got != 1 {
t.Fatalf("expected one majority group, got %#v", summary["majority"])
}
if got, _ := summary["minority"].(float64); got != 1 {
t.Fatalf("expected one minority group, got %#v", summary["minority"])
}
groups := nestedArray(t, reportResp, "data", "grouped_recommendations")
if len(groups) != 1 {
t.Fatalf("expected one reported recommendation group, got %#v", groups)
}
group, ok := groups[0].(map[string]any)
if !ok {
t.Fatalf("expected grouped recommendation object, got %#v", groups[0])
}
if got, _ := group["bucket"].(string); got != "consensus" {
t.Fatalf("expected only reported bucket to be consensus, got %#v", group["bucket"])
}
artifacts := nestedArray(t, reportResp, "data", "report_artifacts")
if len(artifacts) != 1 {
t.Fatalf("expected one report artifact, got %#v", artifacts)
}
artifact, ok := artifacts[0].(map[string]any)
if !ok {
t.Fatalf("expected report artifact object, got %#v", artifacts[0])
}
reportPath, _ := artifact["path"].(string)
if reportPath == "" {
t.Fatalf("expected markdown artifact path, got %#v", artifact["path"])
}
if _, err := os.Stat(reportPath); err != nil {
t.Fatalf("expected markdown artifact to exist at %q: %v", reportPath, err)
}
}
func orchErrorMessage(t *testing.T, raw string) string {
t.Helper()
var payload map[string]any
mustDecodeJSON(t, raw, &payload)
errorValue, ok := payload["error"].(map[string]any)
if !ok {
t.Fatalf("expected error object, got %#v", payload["error"])
}
msg, ok := errorValue["message"].(string)
if !ok {
t.Fatalf("expected error message string, got %#v", errorValue["message"])
}
return msg
}
func seedOnlyUnanimousCouncilReportRun(t *testing.T, dbPath, runID string) {
t.Helper()
runOrchCommand(
t,
"--db", dbPath,
"--json",
"council", "start",
"--run", runID,
"--target", "Review the council reporting flow.",
"--only-unanimous",
)
completeCouncilReviewer(
t,
dbPath,
runID,
"architecture-reviewer",
`{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}},{"title":"Share helpers","summary":"Council report rendering paths are repeated.","proposal":"Introduce shared council coordinator helpers for report rendering.","rationale":"This keeps report assembly consistent.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`,
)
completeCouncilReviewer(
t,
dbPath,
runID,
"implementation-reviewer",
`{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"high","tags":["maintainability"],"target_refs":{"repo_path":"."}},{"title":"Reuse report helpers","summary":"Formatting logic should stay shared.","proposal":"Introduce shared council coordinator helpers for report rendering","rationale":"This avoids formatter drift.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`,
)
completeCouncilReviewer(
t,
dbPath,
runID,
"risk-reviewer",
`{"reviewer_role":"risk-reviewer","findings":[{"title":"Lock contracts","summary":"Contract drift becomes risky over time.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This reduces integration regressions.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}},{"title":"Cover JSON output","summary":"The council report response should stay stable.","proposal":"Add regression tests for council report JSON output.","rationale":"This catches contract regressions earlier.","confidence":"high","tags":["testing"],"target_refs":{"repo_path":"."}}]}`,
)
runOrchCommand(
t,
"--db", dbPath,
"--json",
"council", "tally",
"--run", runID,
"--similarity", "normal",
)
}