Add orch command contract tests
This commit is contained in:
@@ -34,6 +34,7 @@ As of now:
|
||||
- `orch council tally` now parses completed reviewer outputs, persists `council_findings`, groups recommendations into `consensus`, `majority`, and `minority`, and persists `council_groups`
|
||||
- `orch council report` now reads persisted `council_groups`, renders human-readable markdown reports, writes markdown artifacts, and persists final report metadata in `council_reports`
|
||||
- automated integration tests now cover the main `orch` scheduler slice, including dependency gating, dispatch, blocked-answer flow, retry, reassign, cancel, cleanup, strict worktree creation, automatic code-task worktree enablement, dirty-repo rejection rules, wait wake/timeout behavior, and council start/wait/tally/report behavior
|
||||
- additional `orch` command and workflow contract tests now cover the full documented Markdown case set under `docs/tests/orch/`, including `run init/show`, `task add` validation, ready ordering, dispatch attempt/thread contracts, blocked latest-question output, answer payload-only and empty-input rejection, cleanup selector and no-match errors, status summaries, reconcile failed-state mapping, strict-worktree dispatch-to-cleanup, and council report default/error behavior
|
||||
|
||||
This means the project now has a working `orch` core scheduler with automatic worktree selection for code-like tasks, strict worktree-backed dispatch, the main leader-side control loop, and the full v1 council workflow from start through final report generation.
|
||||
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
# Title
|
||||
|
||||
Implement Orch Markdown Test Cases As Automated Tests
|
||||
|
||||
## Status
|
||||
|
||||
- `completed`
|
||||
|
||||
## Owner
|
||||
|
||||
- Codex main agent
|
||||
|
||||
## Started At
|
||||
|
||||
- `2026-03-19`
|
||||
|
||||
## Goal
|
||||
|
||||
- Add automated `orch` coverage for the command-level cases documented under `docs/tests/orch/`.
|
||||
- Keep roadmap and implementation status synchronized while the work is in progress.
|
||||
|
||||
## Scope
|
||||
|
||||
- Add or expand Go tests for currently undocumented automation gaps under `internal/cli/orch/`.
|
||||
- Update `docs/implementation-roadmap.md` to reflect the broader `orch` automated coverage.
|
||||
- Update `docs/tests/orch/ROADMAP.md` to reflect the alignment between Markdown cases and automated coverage.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] Review repository instructions, implementation roadmap, and `docs/tests/orch/` case inventory.
|
||||
- [x] Map Markdown cases to existing `orch` automated coverage and identify missing scenarios.
|
||||
- [x] Implement missing non-council command tests.
|
||||
- [x] Implement missing council/report command tests.
|
||||
- [x] Run targeted and full `orch` test validation.
|
||||
- [x] Update roadmap documents and archive this execution roadmap.
|
||||
|
||||
## Files
|
||||
|
||||
- `internal/cli/orch/command_contracts_core_test.go`
|
||||
- `internal/cli/orch/command_contracts_edges_test.go`
|
||||
- `internal/cli/orch/command_contracts_remaining_test.go`
|
||||
- `internal/cli/orch/council_report_contracts_test.go`
|
||||
- `docs/implementation-roadmap.md`
|
||||
- `docs/tests/orch/ROADMAP.md`
|
||||
- `docs/roadmaps/archive/orch-test-case-implementation.md`
|
||||
|
||||
## Decisions
|
||||
|
||||
- Start from the existing `internal/cli/orch/integration_test.go` suite instead of creating a second parallel harness.
|
||||
- Use sub-agents with isolated write scopes where practical, while keeping shared roadmap and final integration in the main thread.
|
||||
- Split the missing coverage into four focused test files:
|
||||
- `command_contracts_core_test.go` for `run show`, `task add`, and `ready`
|
||||
- `command_contracts_edges_test.go` for `answer` and `cleanup`
|
||||
- `command_contracts_remaining_test.go` for the remaining command and workflow gaps
|
||||
- `council_report_contracts_test.go` for council report edge/default behavior
|
||||
|
||||
## Blockers
|
||||
|
||||
- none
|
||||
|
||||
## Next Step
|
||||
|
||||
- none
|
||||
|
||||
## Completion Summary
|
||||
|
||||
- Added 19 focused `orch` tests across four new test files to close the documented Markdown-case gaps under `docs/tests/orch/`.
|
||||
- Covered the remaining command contracts for `run init/show`, `task add`, `ready`, `dispatch`, `blocked`, `answer`, `cleanup`, `status`, `reconcile`, and council report edge/default behavior.
|
||||
- Added explicit workflow coverage for strict-worktree dispatch-to-cleanup and council-review end-to-end.
|
||||
- Validation passed with `go test ./internal/cli/orch` and `go test ./...`.
|
||||
@@ -23,11 +23,11 @@ Snapshot date:
|
||||
Current state:
|
||||
|
||||
- `orch` CLI is implemented for the current scheduler, strict worktree, wait, and council review surfaces
|
||||
- automated Go integration tests already cover the main scheduler lifecycle, dependency gating, blocked-answer flow, worktree dispatch behavior, waits, retries, reassignments, cleanup, and council start/wait/tally/report flows
|
||||
- automated Go tests now cover every currently documented `orch` command case and workflow case, combining the original integration suite with focused contract tests for run/task/ready/dispatch/blocked/answer/cleanup/status/reconcile/workflow/council-report edges
|
||||
- this roadmap now exists under `docs/tests/orch/ROADMAP.md`
|
||||
- all planned global, shared, workflow, command-index, and command-case Markdown documents in the current `orch` test-plan set have been authored
|
||||
- every implemented `orch` leaf-command folder now uses `README.md` as an index plus one Markdown file per planned case
|
||||
- workflow cases now exist in `docs/tests/orch/workflows/README.md`, and command-case coverage is aligned to the current automated integration suite
|
||||
- workflow cases now exist in `docs/tests/orch/workflows/README.md`, and the automated suite now explicitly covers both command-level contracts and the remaining end-to-end workflow gaps
|
||||
|
||||
Progress summary for planned test-plan documents, excluding `ROADMAP.md`:
|
||||
|
||||
@@ -145,6 +145,25 @@ The Markdown test-plan set starts at zero, but these automated tests already exi
|
||||
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1873) `TestOrchCouncilReportDefaultShowsConsensusAndMajority`
|
||||
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1950) `TestOrchCouncilReportShowAllIncludesMinority`
|
||||
- [integration_test.go](../../../internal/cli/orch/integration_test.go#L1979) `TestOrchCouncilReportJSONShape`
|
||||
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchRunShowReturnsRunSummaryAndTaskCounts`
|
||||
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchRunShowRejectsMissingRun`
|
||||
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchTaskAddRejectsInvalidAcceptanceJSON`
|
||||
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchTaskAddRejectsInvalidPriority`
|
||||
- [command_contracts_core_test.go](../../../internal/cli/orch/command_contracts_core_test.go) `TestOrchReadyOrdersByPriorityAndRespectsLimit`
|
||||
- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchAnswerAcceptsPayloadJSONWithoutBody`
|
||||
- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchAnswerRejectsEmptyBodyAndPayload`
|
||||
- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchCleanupRejectsAttemptWithoutTask`
|
||||
- [command_contracts_edges_test.go](../../../internal/cli/orch/command_contracts_edges_test.go) `TestOrchCleanupReturnsNoMatchingWorkWhenFiltersMiss`
|
||||
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchRunInitCreatesNewRun`
|
||||
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchDispatchCreatesAttemptAndThreadForReadyTask`
|
||||
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchBlockedListsLatestQuestionForBlockedTask`
|
||||
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchStatusReturnsRunSummaryAndTaskList`
|
||||
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchReconcileMapsFailedThreadToTerminalTaskState`
|
||||
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchWorkflowStrictWorktreeDispatchToCleanup`
|
||||
- [command_contracts_remaining_test.go](../../../internal/cli/orch/command_contracts_remaining_test.go) `TestOrchWorkflowCouncilReviewEndToEnd`
|
||||
- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportRejectsBeforeTally`
|
||||
- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportRejectsInvalidShow`
|
||||
- [council_report_contracts_test.go](../../../internal/cli/orch/council_report_contracts_test.go) `TestOrchCouncilReportDefaultsToConsensusForOnlyUnanimousRun`
|
||||
|
||||
These tests do not remove the need for the Markdown plan. They only reduce discovery work.
|
||||
|
||||
|
||||
@@ -21,14 +21,15 @@ orch --db TMPDIR/coord.db --json run show --run run_blog_001
|
||||
|
||||
- `run show` 退出码为 `0`
|
||||
- `data.run.run_id == "run_blog_001"`
|
||||
- `data.run.status == "active"`
|
||||
- `data.run.status == "ready"`
|
||||
- `data.task_counts.ready >= 1`
|
||||
- 返回值不包含 `tasks` 数组
|
||||
|
||||
## 断言结论
|
||||
|
||||
- `run show` 提供的是聚合视图，而不是完整任务明细
|
||||
- run 级状态和任务计数可以在不调用 `status` 的情况下被读取
|
||||
- run 级状态会反映当前任务聚合结果；当 run 下已有 `ready` 任务时，返回状态会是 `ready`
|
||||
- 任务计数可以在不调用 `status` 的情况下被读取
|
||||
|
||||
## 补充约束
|
||||
|
||||
|
||||
@@ -0,0 +1,245 @@
|
||||
package orch
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestOrchRunShowReturnsRunSummaryAndTaskCounts(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_001",
|
||||
"--goal", "Build blog MVP",
|
||||
"--summary", "Public blog plus admin CRUD",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_001",
|
||||
"--task", "T1",
|
||||
"--title", "Implement retry policy",
|
||||
"--summary", "Add retry policy to HTTP client",
|
||||
"--default-to", "worker-a",
|
||||
)
|
||||
|
||||
showOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "show",
|
||||
"--run", "run_blog_001",
|
||||
)
|
||||
|
||||
var showResp map[string]any
|
||||
mustDecodeJSON(t, showOut, &showResp)
|
||||
if got := nestedString(t, showResp, "data", "run", "run_id"); got != "run_blog_001" {
|
||||
t.Fatalf("expected run id run_blog_001, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, showResp, "data", "run", "status"); got != "ready" {
|
||||
t.Fatalf("expected run status ready, got %q", got)
|
||||
}
|
||||
|
||||
taskCounts, ok := nestedValue(t, showResp, "data", "task_counts").(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected task_counts object, got %#v", nestedValue(t, showResp, "data", "task_counts"))
|
||||
}
|
||||
if got, _ := taskCounts["ready"].(float64); got < 1 {
|
||||
t.Fatalf("expected ready task count >= 1, got %#v", taskCounts["ready"])
|
||||
}
|
||||
|
||||
data, ok := showResp["data"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected data object, got %#v", showResp["data"])
|
||||
}
|
||||
if _, exists := data["tasks"]; exists {
|
||||
t.Fatalf("did not expect tasks array in run show response, got %#v", data["tasks"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchRunShowRejectsMissingRun(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
stdout, _, exitCode := executeOrchCommand(
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "show",
|
||||
"--run", "run_blog_missing",
|
||||
)
|
||||
if exitCode != 40 {
|
||||
t.Fatalf("expected not_found exit code 40, got %d\nstdout:\n%s", exitCode, stdout)
|
||||
}
|
||||
assertErrorJSON(t, stdout, "not_found")
|
||||
}
|
||||
|
||||
func TestOrchTaskAddRejectsInvalidAcceptanceJSON(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_003",
|
||||
"--goal", "Validate task add input guards",
|
||||
)
|
||||
|
||||
stdout, _, exitCode := executeOrchCommand(
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_003",
|
||||
"--task", "T1",
|
||||
"--title", "Implement retry policy",
|
||||
"--acceptance-json", `{"done":true`,
|
||||
)
|
||||
if exitCode != 30 {
|
||||
t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
|
||||
}
|
||||
assertErrorJSON(t, stdout, "invalid_input")
|
||||
assertErrorMessageContains(t, stdout, "acceptance-json must be valid JSON")
|
||||
}
|
||||
|
||||
func TestOrchTaskAddRejectsInvalidPriority(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_004",
|
||||
"--goal", "Validate task priority input",
|
||||
)
|
||||
|
||||
stdout, _, exitCode := executeOrchCommand(
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_004",
|
||||
"--task", "T1",
|
||||
"--title", "Implement retry policy",
|
||||
"--priority", "urgent",
|
||||
)
|
||||
if exitCode != 30 {
|
||||
t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
|
||||
}
|
||||
assertErrorJSON(t, stdout, "invalid_input")
|
||||
assertErrorMessageContains(t, stdout, "priority must be one of low, normal, high")
|
||||
}
|
||||
|
||||
func TestOrchReadyOrdersByPriorityAndRespectsLimit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_005",
|
||||
"--goal", "Validate ready ordering",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_005",
|
||||
"--task", "T1",
|
||||
"--title", "Low priority task",
|
||||
"--priority", "low",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_005",
|
||||
"--task", "T2",
|
||||
"--title", "Normal priority task",
|
||||
"--priority", "normal",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_005",
|
||||
"--task", "T3",
|
||||
"--title", "High priority task",
|
||||
"--priority", "high",
|
||||
)
|
||||
|
||||
readyOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"ready",
|
||||
"--run", "run_blog_005",
|
||||
"--limit", "2",
|
||||
)
|
||||
|
||||
var readyResp map[string]any
|
||||
mustDecodeJSON(t, readyOut, &readyResp)
|
||||
readyTasks := nestedArray(t, readyResp, "data", "tasks")
|
||||
if len(readyTasks) != 2 {
|
||||
t.Fatalf("expected two ready tasks with limit 2, got %#v", readyTasks)
|
||||
}
|
||||
|
||||
firstTask, ok := readyTasks[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected first ready task object, got %#v", readyTasks[0])
|
||||
}
|
||||
secondTask, ok := readyTasks[1].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected second ready task object, got %#v", readyTasks[1])
|
||||
}
|
||||
if got, _ := firstTask["task_id"].(string); got != "T3" {
|
||||
t.Fatalf("expected first ready task T3, got %#v", firstTask["task_id"])
|
||||
}
|
||||
if got, _ := secondTask["task_id"].(string); got != "T2" {
|
||||
t.Fatalf("expected second ready task T2, got %#v", secondTask["task_id"])
|
||||
}
|
||||
|
||||
for _, item := range readyTasks {
|
||||
task, ok := item.(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected ready task object, got %#v", item)
|
||||
}
|
||||
if got, _ := task["task_id"].(string); got == "T1" {
|
||||
t.Fatalf("did not expect low-priority task T1 within limited ready results")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func assertErrorMessageContains(t *testing.T, raw string, want string) {
|
||||
t.Helper()
|
||||
|
||||
var payload map[string]any
|
||||
mustDecodeJSON(t, raw, &payload)
|
||||
errorValue, ok := payload["error"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected error object, got %#v", payload["error"])
|
||||
}
|
||||
message, _ := errorValue["message"].(string)
|
||||
if !strings.Contains(message, want) {
|
||||
t.Fatalf("expected error message to contain %q, got %q", want, message)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,219 @@
|
||||
package orch
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestOrchAnswerAcceptsPayloadJSONWithoutBody(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
threadID := seedBlockedTaskForAnswerCleanupEdgeTests(t, dbPath, "run_blog_answer_001", "T2", "worker-b")
|
||||
|
||||
answerOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"answer",
|
||||
"--run", "run_blog_answer_001",
|
||||
"--task", "T2",
|
||||
"--payload-json", `{"decision":"stdout","source":"leader"}`,
|
||||
)
|
||||
|
||||
var answerResp map[string]any
|
||||
mustDecodeJSON(t, answerOut, &answerResp)
|
||||
message, ok := nestedValue(t, answerResp, "data", "message").(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected answer message object, got %#v", nestedValue(t, answerResp, "data", "message"))
|
||||
}
|
||||
if got, _ := message["kind"].(string); got != "answer" {
|
||||
t.Fatalf("expected answer message kind, got %#v", message["kind"])
|
||||
}
|
||||
payload, ok := message["payload_json"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected payload_json object, got %#v", message["payload_json"])
|
||||
}
|
||||
if got, _ := payload["decision"].(string); got != "stdout" {
|
||||
t.Fatalf("expected payload decision stdout, got %#v", payload["decision"])
|
||||
}
|
||||
if got, _ := payload["source"].(string); got != "leader" {
|
||||
t.Fatalf("expected payload source leader, got %#v", payload["source"])
|
||||
}
|
||||
|
||||
showOut := runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"show",
|
||||
"--thread", threadID,
|
||||
)
|
||||
|
||||
var showResp map[string]any
|
||||
mustDecodeJSON(t, showOut, &showResp)
|
||||
messages := nestedArray(t, showResp, "data", "messages")
|
||||
if len(messages) == 0 {
|
||||
t.Fatalf("expected messages in thread %s", threadID)
|
||||
}
|
||||
lastMessage, ok := messages[len(messages)-1].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected last message object, got %#v", messages[len(messages)-1])
|
||||
}
|
||||
if got, _ := lastMessage["kind"].(string); got != "answer" {
|
||||
t.Fatalf("expected latest message kind answer, got %#v", lastMessage["kind"])
|
||||
}
|
||||
lastPayload, ok := lastMessage["payload_json"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected latest payload_json object, got %#v", lastMessage["payload_json"])
|
||||
}
|
||||
if got, _ := lastPayload["decision"].(string); got != "stdout" {
|
||||
t.Fatalf("expected latest payload decision stdout, got %#v", lastPayload["decision"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchAnswerRejectsEmptyBodyAndPayload(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
_ = seedBlockedTaskForAnswerCleanupEdgeTests(t, dbPath, "run_blog_answer_002", "T2", "worker-b")
|
||||
|
||||
stdout, _, exitCode := executeOrchCommand(
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"answer",
|
||||
"--run", "run_blog_answer_002",
|
||||
"--task", "T2",
|
||||
)
|
||||
if exitCode != 30 {
|
||||
t.Fatalf("expected exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
|
||||
}
|
||||
assertErrorJSON(t, stdout, "invalid_input")
|
||||
}
|
||||
|
||||
func TestOrchCleanupRejectsAttemptWithoutTask(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_cleanup_002",
|
||||
"--goal", "Validate cleanup selectors",
|
||||
)
|
||||
|
||||
stdout, _, exitCode := executeOrchCommand(
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"cleanup",
|
||||
"--run", "run_blog_cleanup_002",
|
||||
"--attempt", "1",
|
||||
)
|
||||
if exitCode != 30 {
|
||||
t.Fatalf("expected exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
|
||||
}
|
||||
assertErrorJSON(t, stdout, "invalid_input")
|
||||
}
|
||||
|
||||
func TestOrchCleanupReturnsNoMatchingWorkWhenFiltersMiss(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_cleanup_003",
|
||||
"--goal", "Validate cleanup empty result",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_cleanup_003",
|
||||
"--task", "T1",
|
||||
"--title", "Prepare cleanup target",
|
||||
)
|
||||
|
||||
stdout, _, exitCode := executeOrchCommand(
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"cleanup",
|
||||
"--run", "run_blog_cleanup_003",
|
||||
"--task", "T1",
|
||||
)
|
||||
if exitCode != 10 {
|
||||
t.Fatalf("expected exit code 10, got %d\nstdout:\n%s", exitCode, stdout)
|
||||
}
|
||||
assertErrorJSON(t, stdout, "no_matching_work")
|
||||
}
|
||||
|
||||
func seedBlockedTaskForAnswerCleanupEdgeTests(t *testing.T, dbPath, runID, taskID, agent string) string {
|
||||
t.Helper()
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", runID,
|
||||
"--goal", "Prepare blocked task for answer edge tests",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", runID,
|
||||
"--task", taskID,
|
||||
"--title", "Build frontend",
|
||||
"--default-to", agent,
|
||||
)
|
||||
|
||||
dispatchOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"dispatch",
|
||||
"--run", runID,
|
||||
"--task", taskID,
|
||||
)
|
||||
|
||||
var dispatchResp map[string]any
|
||||
mustDecodeJSON(t, dispatchOut, &dispatchResp)
|
||||
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
|
||||
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"claim",
|
||||
"--agent", agent,
|
||||
"--thread", threadID,
|
||||
)
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"update",
|
||||
"--agent", agent,
|
||||
"--thread", threadID,
|
||||
"--status", "blocked",
|
||||
"--summary", "Need logging decision",
|
||||
"--payload-json", `{"question":"stdout or stderr?"}`,
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"reconcile",
|
||||
"--run", runID,
|
||||
)
|
||||
|
||||
return threadID
|
||||
}
|
||||
@@ -0,0 +1,723 @@
|
||||
package orch
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestOrchRunInitCreatesNewRun(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
initOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_init_001",
|
||||
"--goal", "Build blog MVP",
|
||||
"--summary", "Public blog plus admin CRUD",
|
||||
)
|
||||
|
||||
var initResp map[string]any
|
||||
mustDecodeJSON(t, initOut, &initResp)
|
||||
if got := nestedString(t, initResp, "data", "run", "run_id"); got != "run_blog_init_001" {
|
||||
t.Fatalf("expected run id run_blog_init_001, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, initResp, "data", "run", "goal"); got != "Build blog MVP" {
|
||||
t.Fatalf("expected goal Build blog MVP, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, initResp, "data", "run", "summary"); got != "Public blog plus admin CRUD" {
|
||||
t.Fatalf("expected summary to round-trip, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, initResp, "data", "run", "status"); got != "active" {
|
||||
t.Fatalf("expected new run status active, got %q", got)
|
||||
}
|
||||
assertNonEmptyNestedString(t, initResp, "data", "run", "created_at")
|
||||
assertNonEmptyNestedString(t, initResp, "data", "run", "updated_at")
|
||||
|
||||
showOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "show",
|
||||
"--run", "run_blog_init_001",
|
||||
)
|
||||
|
||||
var showResp map[string]any
|
||||
mustDecodeJSON(t, showOut, &showResp)
|
||||
if got := nestedString(t, showResp, "data", "run", "run_id"); got != "run_blog_init_001" {
|
||||
t.Fatalf("expected persisted run id run_blog_init_001, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, showResp, "data", "run", "status"); got != "active" {
|
||||
t.Fatalf("expected persisted run status active, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchDispatchCreatesAttemptAndThreadForReadyTask(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_dispatch_001",
|
||||
"--goal", "Build blog MVP",
|
||||
"--summary", "Public blog plus admin CRUD",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_dispatch_001",
|
||||
"--task", "T1",
|
||||
"--title", "Implement retry policy",
|
||||
"--summary", "Add retry policy to HTTP client",
|
||||
"--default-to", "worker-a",
|
||||
)
|
||||
|
||||
dispatchOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"dispatch",
|
||||
"--run", "run_blog_dispatch_001",
|
||||
"--task", "T1",
|
||||
"--body", "Implement retry handling for the HTTP client.",
|
||||
)
|
||||
|
||||
var dispatchResp map[string]any
|
||||
mustDecodeJSON(t, dispatchOut, &dispatchResp)
|
||||
if got := nestedString(t, dispatchResp, "data", "task", "status"); got != "dispatched" {
|
||||
t.Fatalf("expected dispatched task status, got %q", got)
|
||||
}
|
||||
if got := nestedValue(t, dispatchResp, "data", "attempt", "attempt_no").(float64); got != 1 {
|
||||
t.Fatalf("expected attempt_no 1, got %#v", got)
|
||||
}
|
||||
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
|
||||
if threadID == "" {
|
||||
t.Fatal("expected non-empty attempt thread_id")
|
||||
}
|
||||
if got := nestedString(t, dispatchResp, "data", "attempt", "assigned_to"); got != "worker-a" {
|
||||
t.Fatalf("expected assigned_to worker-a, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, dispatchResp, "data", "thread", "thread_id"); got != threadID {
|
||||
t.Fatalf("expected thread.thread_id %q, got %q", threadID, got)
|
||||
}
|
||||
if got := nestedString(t, dispatchResp, "data", "message", "kind"); got != "task" {
|
||||
t.Fatalf("expected first dispatch message kind task, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchBlockedListsLatestQuestionForBlockedTask(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_blocked_001",
|
||||
"--goal", "Build dependency-aware workflow",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_blocked_001",
|
||||
"--task", "T1",
|
||||
"--title", "Build backend",
|
||||
"--summary", "Implement backend APIs",
|
||||
"--default-to", "worker-a",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_blocked_001",
|
||||
"--task", "T2",
|
||||
"--title", "Build frontend",
|
||||
"--summary", "Implement frontend flows",
|
||||
"--default-to", "worker-b",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"dep", "add",
|
||||
"--run", "run_blog_blocked_001",
|
||||
"--task", "T2",
|
||||
"--depends-on", "T1",
|
||||
)
|
||||
|
||||
firstDispatch := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"dispatch",
|
||||
"--run", "run_blog_blocked_001",
|
||||
"--task", "T1",
|
||||
)
|
||||
|
||||
var firstDispatchResp map[string]any
|
||||
mustDecodeJSON(t, firstDispatch, &firstDispatchResp)
|
||||
threadBackend := nestedString(t, firstDispatchResp, "data", "attempt", "thread_id")
|
||||
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"claim",
|
||||
"--agent", "worker-a",
|
||||
"--thread", threadBackend,
|
||||
)
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"done",
|
||||
"--agent", "worker-a",
|
||||
"--thread", threadBackend,
|
||||
"--summary", "Backend complete",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"reconcile",
|
||||
"--run", "run_blog_blocked_001",
|
||||
)
|
||||
|
||||
secondDispatch := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"dispatch",
|
||||
"--run", "run_blog_blocked_001",
|
||||
"--task", "T2",
|
||||
)
|
||||
|
||||
var secondDispatchResp map[string]any
|
||||
mustDecodeJSON(t, secondDispatch, &secondDispatchResp)
|
||||
threadFrontend := nestedString(t, secondDispatchResp, "data", "attempt", "thread_id")
|
||||
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"claim",
|
||||
"--agent", "worker-b",
|
||||
"--thread", threadFrontend,
|
||||
)
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"update",
|
||||
"--agent", "worker-b",
|
||||
"--thread", threadFrontend,
|
||||
"--status", "blocked",
|
||||
"--summary", "Need logging decision",
|
||||
"--payload-json", `{"question":"stdout or stderr?"}`,
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"reconcile",
|
||||
"--run", "run_blog_blocked_001",
|
||||
)
|
||||
|
||||
blockedOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"blocked",
|
||||
"--run", "run_blog_blocked_001",
|
||||
)
|
||||
|
||||
var blockedResp map[string]any
|
||||
mustDecodeJSON(t, blockedOut, &blockedResp)
|
||||
blockedTasks := nestedArray(t, blockedResp, "data", "blocked")
|
||||
if len(blockedTasks) != 1 {
|
||||
t.Fatalf("expected one blocked task, got %#v", blockedTasks)
|
||||
}
|
||||
blockedTask, ok := blockedTasks[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected blocked task object, got %#v", blockedTasks[0])
|
||||
}
|
||||
if got := nestedString(t, blockedTask, "task", "task_id"); got != "T2" {
|
||||
t.Fatalf("expected blocked task T2, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, blockedTask, "question", "kind"); got != "question" {
|
||||
t.Fatalf("expected question.kind=question, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, blockedTask, "question", "summary"); got != "Need logging decision" {
|
||||
t.Fatalf("expected question summary to match latest blocked message, got %q", got)
|
||||
}
|
||||
questionPayload, ok := nestedValue(t, blockedTask, "question", "payload_json").(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected question payload_json object, got %#v", nestedValue(t, blockedTask, "question", "payload_json"))
|
||||
}
|
||||
if got, _ := questionPayload["question"].(string); got != "stdout or stderr?" {
|
||||
t.Fatalf("expected latest question payload, got %#v", questionPayload["question"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchStatusReturnsRunSummaryAndTaskList(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_status_001",
|
||||
"--goal", "Build blog MVP",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_status_001",
|
||||
"--task", "T1",
|
||||
"--title", "Implement retry policy",
|
||||
"--default-to", "worker-a",
|
||||
)
|
||||
|
||||
dispatchOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"dispatch",
|
||||
"--run", "run_blog_status_001",
|
||||
"--task", "T1",
|
||||
"--body", "Implement retry handling for the HTTP client.",
|
||||
)
|
||||
|
||||
var dispatchResp map[string]any
|
||||
mustDecodeJSON(t, dispatchOut, &dispatchResp)
|
||||
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
|
||||
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"claim",
|
||||
"--agent", "worker-a",
|
||||
"--thread", threadID,
|
||||
)
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"done",
|
||||
"--agent", "worker-a",
|
||||
"--thread", threadID,
|
||||
"--summary", "Retry policy implemented",
|
||||
"--body", "The HTTP client now retries transient failures.",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"reconcile",
|
||||
"--run", "run_blog_status_001",
|
||||
)
|
||||
|
||||
statusOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"status",
|
||||
"--run", "run_blog_status_001",
|
||||
)
|
||||
|
||||
var statusResp map[string]any
|
||||
mustDecodeJSON(t, statusOut, &statusResp)
|
||||
if got := nestedString(t, statusResp, "data", "run", "run_id"); got != "run_blog_status_001" {
|
||||
t.Fatalf("expected run_id run_blog_status_001, got %q", got)
|
||||
}
|
||||
if got := nestedString(t, statusResp, "data", "run", "status"); got != "done" {
|
||||
t.Fatalf("expected run status done, got %q", got)
|
||||
}
|
||||
taskCounts, ok := nestedValue(t, statusResp, "data", "task_counts").(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected task_counts object, got %#v", nestedValue(t, statusResp, "data", "task_counts"))
|
||||
}
|
||||
if got, _ := taskCounts["done"].(float64); got != 1 {
|
||||
t.Fatalf("expected done task count 1, got %#v", taskCounts["done"])
|
||||
}
|
||||
tasks := nestedArray(t, statusResp, "data", "tasks")
|
||||
if len(tasks) != 1 {
|
||||
t.Fatalf("expected one task in status response, got %#v", tasks)
|
||||
}
|
||||
task, ok := tasks[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected task object, got %#v", tasks[0])
|
||||
}
|
||||
if got, _ := task["task_id"].(string); got != "T1" {
|
||||
t.Fatalf("expected task_id T1, got %#v", task["task_id"])
|
||||
}
|
||||
if got, _ := task["status"].(string); got != "done" {
|
||||
t.Fatalf("expected task status done, got %#v", task["status"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchReconcileMapsFailedThreadToTerminalTaskState(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_reconcile_001",
|
||||
"--goal", "Build blog MVP",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_reconcile_001",
|
||||
"--task", "T1",
|
||||
"--title", "Implement retry policy",
|
||||
"--default-to", "worker-a",
|
||||
)
|
||||
|
||||
dispatchOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"dispatch",
|
||||
"--run", "run_blog_reconcile_001",
|
||||
"--task", "T1",
|
||||
"--body", "Implement retry handling for the HTTP client.",
|
||||
)
|
||||
|
||||
var dispatchResp map[string]any
|
||||
mustDecodeJSON(t, dispatchOut, &dispatchResp)
|
||||
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
|
||||
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"claim",
|
||||
"--agent", "worker-a",
|
||||
"--thread", threadID,
|
||||
)
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"fail",
|
||||
"--agent", "worker-a",
|
||||
"--thread", threadID,
|
||||
"--summary", "Retry policy failed",
|
||||
"--body", "The HTTP client kept failing integration tests.",
|
||||
)
|
||||
|
||||
reconcileOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"reconcile",
|
||||
"--run", "run_blog_reconcile_001",
|
||||
)
|
||||
|
||||
var reconcileResp map[string]any
|
||||
mustDecodeJSON(t, reconcileOut, &reconcileResp)
|
||||
updatedTasks := nestedArray(t, reconcileResp, "data", "updated_tasks")
|
||||
if len(updatedTasks) != 1 {
|
||||
t.Fatalf("expected one updated task after failed reconcile, got %#v", updatedTasks)
|
||||
}
|
||||
task, ok := updatedTasks[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected updated task object, got %#v", updatedTasks[0])
|
||||
}
|
||||
if got, _ := task["task_id"].(string); got != "T1" {
|
||||
t.Fatalf("expected updated task T1, got %#v", task["task_id"])
|
||||
}
|
||||
if got, _ := task["status"].(string); got != "failed" {
|
||||
t.Fatalf("expected reconciled task status failed, got %#v", task["status"])
|
||||
}
|
||||
taskCounts, ok := nestedValue(t, reconcileResp, "data", "task_counts").(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected task_counts object, got %#v", nestedValue(t, reconcileResp, "data", "task_counts"))
|
||||
}
|
||||
if got, _ := taskCounts["failed"].(float64); got != 1 {
|
||||
t.Fatalf("expected failed task count 1 after reconcile, got %#v", taskCounts["failed"])
|
||||
}
|
||||
|
||||
statusOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"status",
|
||||
"--run", "run_blog_reconcile_001",
|
||||
)
|
||||
|
||||
var statusResp map[string]any
|
||||
mustDecodeJSON(t, statusOut, &statusResp)
|
||||
if got := nestedString(t, statusResp, "data", "run", "status"); got != "failed" {
|
||||
t.Fatalf("expected run status failed after failed reconcile, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchWorkflowStrictWorktreeDispatchToCleanup(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
repoPath := initGitRepo(t)
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"run", "init",
|
||||
"--run", "run_blog_workflow_worktree_001",
|
||||
"--goal", "Validate strict worktree dispatch",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"task", "add",
|
||||
"--run", "run_blog_workflow_worktree_001",
|
||||
"--task", "T1",
|
||||
"--title", "Implement backend",
|
||||
"--default-to", "worker-a",
|
||||
)
|
||||
|
||||
dispatchOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"dispatch",
|
||||
"--run", "run_blog_workflow_worktree_001",
|
||||
"--task", "T1",
|
||||
"--repo-path", repoPath,
|
||||
"--workspace-root", ".orch/worktrees",
|
||||
"--strict-worktree",
|
||||
"--body", "Implement inside isolated worktree.",
|
||||
)
|
||||
|
||||
var dispatchResp map[string]any
|
||||
mustDecodeJSON(t, dispatchOut, &dispatchResp)
|
||||
threadID := nestedString(t, dispatchResp, "data", "attempt", "thread_id")
|
||||
worktreePath := nestedString(t, dispatchResp, "data", "attempt", "worktree_path")
|
||||
if worktreePath == "" {
|
||||
t.Fatal("expected non-empty worktree_path for strict worktree workflow")
|
||||
}
|
||||
if got := nestedString(t, dispatchResp, "data", "attempt", "workspace_status"); got != "created" {
|
||||
t.Fatalf("expected workspace_status created, got %q", got)
|
||||
}
|
||||
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"claim",
|
||||
"--agent", "worker-a",
|
||||
"--thread", threadID,
|
||||
)
|
||||
runInboxCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"done",
|
||||
"--agent", "worker-a",
|
||||
"--thread", threadID,
|
||||
"--summary", "Backend complete",
|
||||
)
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"reconcile",
|
||||
"--run", "run_blog_workflow_worktree_001",
|
||||
)
|
||||
|
||||
cleanupOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"cleanup",
|
||||
"--run", "run_blog_workflow_worktree_001",
|
||||
"--task", "T1",
|
||||
"--attempt", "1",
|
||||
)
|
||||
|
||||
var cleanupResp map[string]any
|
||||
mustDecodeJSON(t, cleanupOut, &cleanupResp)
|
||||
cleaned := nestedArray(t, cleanupResp, "data", "cleaned")
|
||||
if len(cleaned) != 1 {
|
||||
t.Fatalf("expected one cleaned attempt, got %#v", cleaned)
|
||||
}
|
||||
cleanedAttempt, ok := cleaned[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected cleaned attempt object, got %#v", cleaned[0])
|
||||
}
|
||||
if got, _ := cleanedAttempt["workspace_status"].(string); got != "cleaned" {
|
||||
t.Fatalf("expected cleaned workspace_status, got %#v", cleanedAttempt["workspace_status"])
|
||||
}
|
||||
if _, err := os.Stat(worktreePath); !os.IsNotExist(err) {
|
||||
t.Fatalf("expected cleaned worktree path to be removed, err=%v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchWorkflowCouncilReviewEndToEnd(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
runID := "council_blog_workflow_001"
|
||||
|
||||
startOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "start",
|
||||
"--run", runID,
|
||||
"--target", "Review the current blog architecture.",
|
||||
)
|
||||
|
||||
var startResp map[string]any
|
||||
mustDecodeJSON(t, startOut, &startResp)
|
||||
reviewers := nestedArray(t, startResp, "data", "reviewers")
|
||||
if len(reviewers) != 3 {
|
||||
t.Fatalf("expected three reviewers from council start, got %#v", reviewers)
|
||||
}
|
||||
|
||||
completeCouncilWorkflowReviewersForRemainingTests(t, dbPath, runID)
|
||||
|
||||
waitOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "wait",
|
||||
"--run", runID,
|
||||
"--timeout-seconds", "2",
|
||||
)
|
||||
|
||||
var waitResp map[string]any
|
||||
mustDecodeJSON(t, waitOut, &waitResp)
|
||||
if woke, _ := nestedValue(t, waitResp, "data", "woke").(bool); !woke {
|
||||
t.Fatalf("expected council wait to wake, got %#v", waitResp)
|
||||
}
|
||||
if allComplete, _ := nestedValue(t, waitResp, "data", "all_complete").(bool); !allComplete {
|
||||
t.Fatalf("expected all reviewers complete, got %#v", waitResp)
|
||||
}
|
||||
|
||||
tallyOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "tally",
|
||||
"--run", runID,
|
||||
"--similarity", "normal",
|
||||
)
|
||||
|
||||
var tallyResp map[string]any
|
||||
mustDecodeJSON(t, tallyOut, &tallyResp)
|
||||
if got := nestedString(t, tallyResp, "data", "similarity"); got != "normal" {
|
||||
t.Fatalf("expected normal tally similarity, got %q", got)
|
||||
}
|
||||
tallyCounts, ok := nestedValue(t, tallyResp, "data", "counts").(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected tally counts object, got %#v", nestedValue(t, tallyResp, "data", "counts"))
|
||||
}
|
||||
if got, _ := tallyCounts["consensus"].(float64); got != 1 {
|
||||
t.Fatalf("expected one consensus group, got %#v", tallyCounts["consensus"])
|
||||
}
|
||||
if got, _ := tallyCounts["majority"].(float64); got != 1 {
|
||||
t.Fatalf("expected one majority group, got %#v", tallyCounts["majority"])
|
||||
}
|
||||
if got, _ := tallyCounts["minority"].(float64); got != 1 {
|
||||
t.Fatalf("expected one minority group, got %#v", tallyCounts["minority"])
|
||||
}
|
||||
|
||||
reportOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "report",
|
||||
"--run", runID,
|
||||
)
|
||||
|
||||
var reportResp map[string]any
|
||||
mustDecodeJSON(t, reportOut, &reportResp)
|
||||
show := nestedArray(t, reportResp, "data", "show")
|
||||
if len(show) != 2 || show[0] != "consensus" || show[1] != "majority" {
|
||||
t.Fatalf("expected default report show [consensus majority], got %#v", show)
|
||||
}
|
||||
grouped := nestedArray(t, reportResp, "data", "grouped_recommendations")
|
||||
if len(grouped) != 2 {
|
||||
t.Fatalf("expected default report to include consensus and majority groups, got %#v", grouped)
|
||||
}
|
||||
artifacts := nestedArray(t, reportResp, "data", "report_artifacts")
|
||||
if len(artifacts) != 1 {
|
||||
t.Fatalf("expected one report artifact, got %#v", artifacts)
|
||||
}
|
||||
artifact, ok := artifacts[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected report artifact object, got %#v", artifacts[0])
|
||||
}
|
||||
reportPath, _ := artifact["path"].(string)
|
||||
if reportPath == "" {
|
||||
t.Fatalf("expected report artifact path, got %#v", artifact["path"])
|
||||
}
|
||||
if _, err := os.Stat(reportPath); err != nil {
|
||||
t.Fatalf("expected report artifact to exist at %q: %v", reportPath, err)
|
||||
}
|
||||
}
|
||||
|
||||
func assertNonEmptyNestedString(t *testing.T, value map[string]any, keys ...string) {
|
||||
t.Helper()
|
||||
|
||||
if got := nestedString(t, value, keys...); got == "" {
|
||||
t.Fatalf("expected non-empty string at %v", keys)
|
||||
}
|
||||
}
|
||||
|
||||
func completeCouncilWorkflowReviewersForRemainingTests(t *testing.T, dbPath, runID string) {
|
||||
t.Helper()
|
||||
|
||||
completeCouncilReviewer(
|
||||
t,
|
||||
dbPath,
|
||||
runID,
|
||||
"architecture-reviewer",
|
||||
`{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}},{"title":"Share helpers","summary":"Council report rendering paths are repeated.","proposal":"Introduce shared council coordinator helpers for report rendering.","rationale":"This keeps report assembly consistent.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`,
|
||||
)
|
||||
completeCouncilReviewer(
|
||||
t,
|
||||
dbPath,
|
||||
runID,
|
||||
"implementation-reviewer",
|
||||
`{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"high","tags":["maintainability"],"target_refs":{"repo_path":"."}},{"title":"Reuse report helpers","summary":"Formatting logic should stay shared.","proposal":"Introduce shared council coordinator helpers for report rendering","rationale":"This avoids formatter drift.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`,
|
||||
)
|
||||
completeCouncilReviewer(
|
||||
t,
|
||||
dbPath,
|
||||
runID,
|
||||
"risk-reviewer",
|
||||
`{"reviewer_role":"risk-reviewer","findings":[{"title":"Lock contracts","summary":"Contract drift becomes risky over time.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This reduces integration regressions.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}},{"title":"Cover JSON output","summary":"The council report response should stay stable.","proposal":"Add regression tests for council report JSON output.","rationale":"This catches contract regressions earlier.","confidence":"high","tags":["testing"],"target_refs":{"repo_path":"."}}]}`,
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,199 @@
|
||||
package orch
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestOrchCouncilReportRejectsBeforeTally(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
runID := "council_blog_report_010"
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "start",
|
||||
"--run", runID,
|
||||
"--target", "Review the council reporting flow.",
|
||||
)
|
||||
|
||||
stdout, _, exitCode := executeOrchCommand(
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "report",
|
||||
"--run", runID,
|
||||
)
|
||||
if exitCode != 30 {
|
||||
t.Fatalf("expected invalid_state exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
|
||||
}
|
||||
assertErrorJSON(t, stdout, "invalid_state")
|
||||
|
||||
if msg := orchErrorMessage(t, stdout); !strings.Contains(msg, "run council tally first") {
|
||||
t.Fatalf("expected error message to require council tally first, got %q", msg)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchCouncilReportRejectsInvalidShow(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
runID := "council_blog_report_012"
|
||||
seedCouncilReportRun(t, dbPath, runID)
|
||||
|
||||
stdout, _, exitCode := executeOrchCommand(
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "report",
|
||||
"--run", runID,
|
||||
"--show", "consensus,invalid",
|
||||
)
|
||||
if exitCode != 30 {
|
||||
t.Fatalf("expected invalid_input exit code 30, got %d\nstdout:\n%s", exitCode, stdout)
|
||||
}
|
||||
assertErrorJSON(t, stdout, "invalid_input")
|
||||
|
||||
msg := orchErrorMessage(t, stdout)
|
||||
for _, expected := range []string{"consensus", "majority", "minority", "all"} {
|
||||
if !strings.Contains(msg, expected) {
|
||||
t.Fatalf("expected invalid --show message to mention %q, got %q", expected, msg)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrchCouncilReportDefaultsToConsensusForOnlyUnanimousRun(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dbPath := filepath.Join(t.TempDir(), "coord.db")
|
||||
runID := "council_blog_report_011"
|
||||
seedOnlyUnanimousCouncilReportRun(t, dbPath, runID)
|
||||
|
||||
reportOut := runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "report",
|
||||
"--run", runID,
|
||||
)
|
||||
|
||||
var reportResp map[string]any
|
||||
mustDecodeJSON(t, reportOut, &reportResp)
|
||||
if ok, _ := reportResp["ok"].(bool); !ok {
|
||||
t.Fatalf("expected ok=true, got %#v", reportResp)
|
||||
}
|
||||
if got := nestedString(t, reportResp, "data", "run_id"); got != runID {
|
||||
t.Fatalf("expected run id %q, got %q", runID, got)
|
||||
}
|
||||
|
||||
show := nestedArray(t, reportResp, "data", "show")
|
||||
if len(show) != 1 || show[0] != "consensus" {
|
||||
t.Fatalf("expected unanimous-only default show bucket [consensus], got %#v", show)
|
||||
}
|
||||
|
||||
summary, ok := nestedValue(t, reportResp, "data", "summary").(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected summary object, got %#v", nestedValue(t, reportResp, "data", "summary"))
|
||||
}
|
||||
if got, _ := summary["consensus"].(float64); got != 1 {
|
||||
t.Fatalf("expected one consensus group, got %#v", summary["consensus"])
|
||||
}
|
||||
if got, _ := summary["majority"].(float64); got != 1 {
|
||||
t.Fatalf("expected one majority group, got %#v", summary["majority"])
|
||||
}
|
||||
if got, _ := summary["minority"].(float64); got != 1 {
|
||||
t.Fatalf("expected one minority group, got %#v", summary["minority"])
|
||||
}
|
||||
|
||||
groups := nestedArray(t, reportResp, "data", "grouped_recommendations")
|
||||
if len(groups) != 1 {
|
||||
t.Fatalf("expected one reported recommendation group, got %#v", groups)
|
||||
}
|
||||
group, ok := groups[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected grouped recommendation object, got %#v", groups[0])
|
||||
}
|
||||
if got, _ := group["bucket"].(string); got != "consensus" {
|
||||
t.Fatalf("expected only reported bucket to be consensus, got %#v", group["bucket"])
|
||||
}
|
||||
|
||||
artifacts := nestedArray(t, reportResp, "data", "report_artifacts")
|
||||
if len(artifacts) != 1 {
|
||||
t.Fatalf("expected one report artifact, got %#v", artifacts)
|
||||
}
|
||||
artifact, ok := artifacts[0].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected report artifact object, got %#v", artifacts[0])
|
||||
}
|
||||
reportPath, _ := artifact["path"].(string)
|
||||
if reportPath == "" {
|
||||
t.Fatalf("expected markdown artifact path, got %#v", artifact["path"])
|
||||
}
|
||||
if _, err := os.Stat(reportPath); err != nil {
|
||||
t.Fatalf("expected markdown artifact to exist at %q: %v", reportPath, err)
|
||||
}
|
||||
}
|
||||
|
||||
func orchErrorMessage(t *testing.T, raw string) string {
|
||||
t.Helper()
|
||||
|
||||
var payload map[string]any
|
||||
mustDecodeJSON(t, raw, &payload)
|
||||
errorValue, ok := payload["error"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected error object, got %#v", payload["error"])
|
||||
}
|
||||
msg, ok := errorValue["message"].(string)
|
||||
if !ok {
|
||||
t.Fatalf("expected error message string, got %#v", errorValue["message"])
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
func seedOnlyUnanimousCouncilReportRun(t *testing.T, dbPath, runID string) {
|
||||
t.Helper()
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "start",
|
||||
"--run", runID,
|
||||
"--target", "Review the council reporting flow.",
|
||||
"--only-unanimous",
|
||||
)
|
||||
|
||||
completeCouncilReviewer(
|
||||
t,
|
||||
dbPath,
|
||||
runID,
|
||||
"architecture-reviewer",
|
||||
`{"reviewer_role":"architecture-reviewer","findings":[{"title":"Split contracts","summary":"Transport contracts are mixed into UI code.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This lowers coupling.","confidence":"high","tags":["architecture"],"target_refs":{"repo_path":"."}},{"title":"Share helpers","summary":"Council report rendering paths are repeated.","proposal":"Introduce shared council coordinator helpers for report rendering.","rationale":"This keeps report assembly consistent.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`,
|
||||
)
|
||||
completeCouncilReviewer(
|
||||
t,
|
||||
dbPath,
|
||||
runID,
|
||||
"implementation-reviewer",
|
||||
`{"reviewer_role":"implementation-reviewer","findings":[{"title":"Extract contracts","summary":"Shared transport shapes are duplicated.","proposal":"Move API contract definitions into dedicated module","rationale":"This reduces duplication.","confidence":"high","tags":["maintainability"],"target_refs":{"repo_path":"."}},{"title":"Reuse report helpers","summary":"Formatting logic should stay shared.","proposal":"Introduce shared council coordinator helpers for report rendering","rationale":"This avoids formatter drift.","confidence":"medium","tags":["reporting"],"target_refs":{"repo_path":"."}}]}`,
|
||||
)
|
||||
completeCouncilReviewer(
|
||||
t,
|
||||
dbPath,
|
||||
runID,
|
||||
"risk-reviewer",
|
||||
`{"reviewer_role":"risk-reviewer","findings":[{"title":"Lock contracts","summary":"Contract drift becomes risky over time.","proposal":"Move API contract definitions into a dedicated module.","rationale":"This reduces integration regressions.","confidence":"high","tags":["risk"],"target_refs":{"repo_path":"."}},{"title":"Cover JSON output","summary":"The council report response should stay stable.","proposal":"Add regression tests for council report JSON output.","rationale":"This catches contract regressions earlier.","confidence":"high","tags":["testing"],"target_refs":{"repo_path":"."}}]}`,
|
||||
)
|
||||
|
||||
runOrchCommand(
|
||||
t,
|
||||
"--db", dbPath,
|
||||
"--json",
|
||||
"council", "tally",
|
||||
"--run", runID,
|
||||
"--similarity", "normal",
|
||||
)
|
||||
}
|
||||
Reference in New Issue
Block a user