whether a leader agent can actually use the packaged council-review skill
whether the bundled ./assets/orch CLI works inside real skill-guided council workflows
whether a council run driven by the skill reaches the expected reviewer, grouping, tally, and report state

Test Model

Shared Execution Contract

Use these defaults unless a case file explicitly overrides them:

run the scenario with real subagents, not simulated transcripts
inject skills/council-review/ into the leader agent
inject skills/inbox/ into reviewer agents whenever reviewer task completion is required
initialize the shared SQLite DB before launching role agents with INBOX_SKILL_PATH/assets/inbox --db TMPDIR/coord.db --json init
require the leader to coordinate through the bundled ./assets/orch CLI from the council-review skill instead of ordinary chat
require reviewer agents to coordinate through the bundled ./assets/inbox CLI from their skill instead of ordinary chat
validate final council run, reviewer task state, and report state independently from the main thread after the agents stop
create any required repo fixture before launching agents for mixed or repo-target cases

Use one test-runner agent to execute each case.

The test-runner agent is responsible for:

reading this README.md first, then one specific case file
creating an isolated temporary directory and DB path for that run
initializing the DB once through the bundled inbox CLI before launching role agents
creating any required temporary Git repo fixture before launching role agents
launching the role agents described in Agent Topology
injecting skills/council-review/ into the leader and skills/inbox/ into reviewers
passing each role agent the prompt text from the case file with concrete values substituted for COUNCIL_SKILL_PATH, INBOX_SKILL_PATH, TMPDIR, RUN_ID, THREAD_ID, and REPORT_PATH when needed
coordinating launch order or parallel start according to the case file
collecting agent final summaries as evidence
resolving final run ids, thread ids, and report artifact paths from agent outputs
running the Validation Commands from the main thread after the role agents stop
comparing the observed results against Expected Outcomes and Assertions
returning a final pass/fail judgment with concrete evidence

The role agents are responsible for:

acting only within the role assigned in the case file
using the injected skill bundle rather than ad hoc repository discovery
coordinating through the bundled CLI and shared DB
reporting concrete run ids, thread ids, report artifact paths, and key command outcomes back to the test-runner agent

The test-runner agent should treat a case as passed only when:

all role agents reach a final state without violating the case contract
the independent validation commands succeed
the final council, orch, and inbox state matches the assertions in the case file

The test-runner agent should treat a case as failed when:

any required agent times out or stalls
a required council, orch, or inbox action is skipped
the leader falls back to ordinary chat for workflow control that should go through the bundled council-review skill
reviewer agents fall back to ordinary chat instead of returning results through inbox
the final council grouping, summary, or report state conflicts with the documented assertions

The test-runner agent should report results in this shape:

Use these defaults unless a case file explicitly overrides them:

Treat the test as failed if any of the following happens:

any required agent does not reach a final state before timeout
any required council, orch, or inbox command returns a non-success result unless the case expects that failure
the final council report --json output does not match the expected grouped recommendations
the final orch status output does not match the expected reviewer task state
a required markdown report artifact is missing when the case expects one
the agents fall back to ordinary chat for critical coordination instead of the bundled CLIs

Collect at least the following artifacts for every run:

agent final summaries
final council report --json output when the case reaches report stage
final orch status --run RUN_ID --json output
final inbox show --thread THREAD_ID --json output for every relevant reviewer thread when reviewers participated
any council wait or council tally output relevant to the case
the temporary DB path, resolved run id, resolved thread ids, and any report artifact paths

Use these defaults unless a case file explicitly overrides them:

keep the temporary DB, repo fixture, and working directory on failure for debugging
cleanup the temporary working directory on success only if the caller does not need replay artifacts

Each case file should use this structure:

Case Slug	File	Coverage Note
`council-brainstorm-end-to-end-through-bundled-cli`	council-brainstorm-end-to-end-through-bundled-cli.md	validates that the council-review skill can drive `start -> wait -> tally -> report` with three real reviewer agents
`council-unanimous-only-default-report-through-bundled-cli`	council-unanimous-only-default-report-through-bundled-cli.md	validates that unanimous-only runs default to `consensus` output while preserving the underlying summary counts
`council-wait-timeout-through-bundled-cli`	council-wait-timeout-through-bundled-cli.md	validates that the leader sees the expected timeout contract when reviewer tasks do not complete
`council-report-rejects-before-tally-through-bundled-cli`	council-report-rejects-before-tally-through-bundled-cli.md	validates that the skill surfaces the stable invalid-state error when report is attempted before tally

In scope:

explicit $council-review skill invocation
bundled ./assets/orch CLI usage for orch council ...
end-to-end council start, wait, tally, and report flows
interaction between a leader using skills/council-review/ and reviewers using skills/inbox/
default report policy, unanimous-only behavior, and timeout/error-path validation

Out of scope:

../orch/ covers CLI command behavior
../orch-skill/ covers generic leader-side orchestration behavior on top of orch
../inbox-skill/ covers worker-side skill-guided behavior on top of inbox
this directory covers the separate user-facing council-review skill on top of orch council