Add council-review skill test plan docs

2026-03-19 17:25:40 +08:00
parent 8b26815d53
commit 0b533a70f9
7 changed files with 584 additions and 0 deletions
@@ -0,0 +1,77 @@
+# Case: `council-wait-timeout-through-bundled-cli`
+
+## Test Type
+
+This is a `forward-test` and a timeout-path council workflow validation.
+
+The goal is to verify that a leader using the packaged `council-review` skill sees the expected timeout contract when reviewer tasks do not complete.
+
+## Purpose
+
+Validate that all of the following can be true at the same time:
+
+- the leader can start a council run through the bundled skill CLI
+- the leader can call `council wait` with a short timeout
+- the command reports `woke == false` and `all_complete == false`
+- reviewer task metadata remains visible for later follow-up
+
+## Preconditions
+
+- council-review skill path exists: `COUNCIL_SKILL_PATH=skills/council-review`
+- bundled CLI executable exists at `COUNCIL_SKILL_PATH/assets/orch`
+- inbox skill path exists: `INBOX_SKILL_PATH=skills/inbox`
+- use an empty temporary directory `TMPDIR`
+- initialize `TMPDIR/coord.db` before launching role agents through `INBOX_SKILL_PATH/assets/inbox --db TMPDIR/coord.db --json init`
+
+## Agent Topology
+
+- `leader`
+
+## Inputs
+
+### Leader Prompt
+
+```text
+Use $council-review at COUNCIL_SKILL_PATH to act as leader on the already initialized SQLite DB TMPDIR/coord.db. Only coordinate through the bundled orch CLI from the skill. Workflow: 1) start council run council_skill_003 with a short review target, 2) immediately call council wait with a short timeout such as 1 second, 3) stop after reporting RUN_ID and the wait result you observed. Do not use ordinary chat to simulate reviewer output.
+```
+
+## Execution Parameters
+
+- use the shared execution contract from [README.md](./README.md)
+- override the council wait timeout to a short interval such as `1s`
+- do not override the default cleanup policy
+
+## Execution Steps
+
+1. Initialize `TMPDIR/coord.db` once through the bundled inbox CLI before launching agents
+2. Inject `skills/council-review/` into `leader`
+3. Point the leader at the database path `TMPDIR/coord.db`
+4. Launch the leader
+5. Wait for the leader to finish
+6. Independently run the validation commands from the main thread
+
+## Validation Commands
+
+```bash
+COUNCIL_SKILL_PATH/assets/orch --db TMPDIR/coord.db --json council wait --run council_skill_003 --timeout-seconds 1
+COUNCIL_SKILL_PATH/assets/orch --db TMPDIR/coord.db --json status --run council_skill_003
+```
+
+## Expected Outcomes
+
+- the leader successfully starts `council_skill_003`
+- `council wait` times out cleanly
+- the wait response still includes three reviewer statuses
+- the run remains non-terminal because reviewers have not completed
+
+## Assertions
+
+- `wait.data.woke == false`
+- `wait.data.all_complete == false`
+- `wait.data.reviewers` length is `3`
+- `status.data.run.status` is not `done`
+
+## Cleanup
+
+- use the default cleanup policy from [README.md](./README.md)
+- if the run fails, retain `TMPDIR` and `coord.db` for replay and manual inspection