ai-workflow-skill/docs/council-review.md

Council Review Workflow

Purpose

This document defines a user-facing brainstorming and review workflow where three reviewer agents analyze the same target from different perspectives, return structured suggestions, and then have their suggestions grouped and tallied.

This is intended to be exposed as a separate skill because a user may explicitly ask for this workflow, for example:

• "Use the council review skill"
• "Start a three-reviewer brainstorm"
• "Have three reviewers propose optimizations"

Under the hood, this workflow should reuse orch and inbox. It does not need a separate infrastructure binary in v1.

Why This Is A Separate Skill

This workflow is not just generic scheduling. It has:

• fixed reviewer roles
• a fixed collection and tally process
• a distinct output format
• user-visible semantics around agreement and disagreement

That makes it a good skill boundary even if the execution still rides on orch.

Default Model

The default council has three reviewer agents with intentionally different lenses:

• architecture-reviewer: architecture, boundaries, interfaces, composition
• implementation-reviewer: code simplicity, maintainability, duplication, practicality
• risk-reviewer: regressions, correctness, security, operability

These are roles, not necessarily permanently running services. orch council start may dispatch them as three worker tasks for the same target.

Default Choices

If the user does not specify otherwise, use these defaults:

• analysis only, no code changes
• show consensus and majority in the main report
• keep minority out of the main report unless it is unusually valuable
• fixed reviewer roles: architecture-reviewer, implementation-reviewer, risk-reviewer
• support both text targets and codebase targets
• emit both a human-readable markdown report and --json
• persist council inputs, reviewer outputs, grouped recommendations, and final report metadata in orch
• use normal proposal grouping instead of strict
• use low|medium|high confidence values
• reuse the normal orch run_id namespace rather than creating a separate id space

These defaults are intended to make the workflow useful out of the box without making it too expensive or too conservative.

What The Council Produces

Each reviewer should return:

• key findings
• proposed modifications
• rationale
• confidence
• optional risks or tradeoffs

The system then groups similar proposals across reviewers and assigns a vote count.

Voting Model

Do not collapse everything into a single "all or nothing" rule by default.

Recommended output buckets:

• consensus: 3 of 3 reviewers support the proposal
• majority: 2 of 3 reviewers support the proposal
• minority: 1 of 3 reviewers supports the proposal

Recommended behavior:

• if the user asks for "only unanimous ideas", show only consensus
• otherwise show consensus first, then majority, and only include minority if it is especially insightful

This keeps user feedback useful without making the system overly conservative.

Execution Model

The council workflow should be implemented as an orch council command group.

Recommended phases:

1. create a council review run for a target
2. dispatch three reviewer tasks
3. wait for all three reviewer responses
4. group similar suggestions
5. tally votes
6. produce a report for the leader or directly for the user

The recommended default report policy is:

• main report: consensus and majority
• optional appendix: minority

CLI Shape

Recommended command group:

• orch council start
• orch council wait
• orch council tally
• orch council report

orch council start

Create and dispatch a three-reviewer council run.

Suggested flags:

• --run RUN_ID
• --target TEXT
• --target-file PATH
• --target-type text|repo|mixed
• --mode brainstorm|review
• --only-unanimous
• --output markdown|json|both

Behavior:

• creates three review tasks with fixed reviewer roles
• dispatches them through orch
• records the council metadata for later tally

Default behavior:

• --mode brainstorm
• --target-type mixed
• --output both
• unanimous-only disabled unless explicitly requested
• reviewer count fixed at 3 in v1

orch council wait

Block until all reviewer outputs arrive or the timeout is reached.

Suggested flags:

• --run RUN_ID
• --timeout-seconds N

Behavior:

• waits until all three reviewer tasks reach terminal task states
• returns current reviewer task statuses on both wake and timeout

orch council tally

Group similar suggestions and count supporting reviewers.

Suggested flags:

• --run RUN_ID
• --similarity strict|normal
• --json

Behavior:

• reads the three reviewer outputs
• groups proposals by normalized intent
• records supporter count and dissent
• persists grouped recommendations in orch storage

orch council report

Produce the final council report.

Suggested flags:

• --run RUN_ID
• --show consensus|majority|minority|all
• --json

Default behavior:

• report consensus,majority
• also allow a markdown artifact for user-facing output

Run Identity

Council runs should reuse the existing orch run_id namespace.

This keeps:

• storage simpler
• waiting and event handling consistent
• reporting compatible with the rest of orch

There is no need for a second run identifier system in v1.

Supported Inputs

The workflow should support three common target shapes:

• text: a prompt, design note, requirement, or copied problem statement
• repo: a repository path or code snapshot to inspect
• mixed: a text brief plus repository context

Recommended default is mixed because many useful brainstorms need both goal context and code context.

Target Reference Schema

The council input should allow both free text and explicit references.

Suggested shape:

{
  "run_id": "council_blog_001",
  "target": {
    "target_type": "mixed",
    "prompt": "Review the current blog architecture and propose optimizations.",
    "target_file": "brief.md",
    "repo_path": ".",
    "task_id": "T4"
  }
}

Recommended field meanings:

• prompt: optional human brief
• target_file: optional design note or copied context file
• repo_path: optional repository root for code inspection
• task_id: optional existing orch task reference when the brainstorm is about a specific task

This keeps the input flexible without making the protocol ambiguous.

Confidence Enum

Use a simple confidence enum in v1:

• low
• medium
• high

Avoid more granular values unless later evidence shows they are useful.

Reviewer Output Schema

Each reviewer should return a structured document like:

{
  "reviewer_role": "architecture-reviewer",
  "findings": [
    {
      "title": "Split API contract from UI implementation details",
      "summary": "The current structure mixes transport contracts with view concerns.",
      "proposal": "Move API contract definitions into a dedicated module and keep UI-specific mapping local to the UI layer.",
      "rationale": "This lowers coupling and reduces integration churn.",
      "confidence": "high",
      "tags": ["architecture", "coupling"],
      "target_refs": {
        "repo_path": ".",
        "files": ["src/api/contracts.ts", "src/ui/admin/posts.tsx"]
      }
    }
  ]
}

The tally step should map semantically similar proposal values into one grouped recommendation.

Required reviewer output expectations:

• reviewer_role must match one of the assigned council roles
• confidence must use low|medium|high
• proposal should describe one actionable recommendation
• target_refs may point to files, repo path, or task id when relevant

Grouped Recommendation Schema

After tally, the system should persist grouped proposals with support counts.

Suggested shape:

{
  "run_id": "council_blog_001",
  "grouped_recommendations": [
    {
      "group_id": "grp_01",
      "proposal": "Move API contract definitions into a dedicated module.",
      "bucket": "consensus",
      "support_count": 3,
      "supporters": [
        "architecture-reviewer",
        "implementation-reviewer",
        "risk-reviewer"
      ],
      "dissenters": [],
      "rationale_summary": "All three reviewers agree the current structure is too coupled.",
      "tags": ["architecture", "coupling"],
      "source_finding_ids": [
        "architecture-reviewer:f01",
        "implementation-reviewer:f03",
        "risk-reviewer:f02"
      ]
    }
  ]
}

Proposal Grouping Rules

The default grouping mode should be normal.

Recommended interpretation:

• strict: only group nearly identical proposals
• normal: group proposals that clearly recommend the same underlying change even if phrased differently

In v1, normal should be the default because it is more useful for brainstorming workflows.

Persistence Model

Council review should persist its state in orch instead of remaining purely ephemeral.

Recommended persisted objects:

• council run metadata
• council reviewer assignments
• reviewer outputs
• grouped recommendations
• final report metadata and artifact paths

This makes it possible to:

• resume after interruption
• audit how a recommendation was formed
• regenerate user-facing reports
• compare multiple council runs over time

JSON Output Shapes

orch council start should return enough information for downstream automation.

Suggested shape:

{
  "ok": true,
  "command": "council-start",
  "run_id": "council_blog_001",
  "mode": "brainstorm",
  "target_type": "mixed",
  "output": "both",
  "reviewers": [
    {
      "reviewer_role": "architecture-reviewer",
      "task_id": "CR1",
      "status": "dispatched"
    },
    {
      "reviewer_role": "implementation-reviewer",
      "task_id": "CR2",
      "status": "dispatched"
    },
    {
      "reviewer_role": "risk-reviewer",
      "task_id": "CR3",
      "status": "dispatched"
    }
  ]
}

orch council tally should return grouped recommendations.

Suggested shape:

{
  "ok": true,
  "command": "council-tally",
  "run_id": "council_blog_001",
  "similarity": "normal",
  "counts": {
    "consensus": 2,
    "majority": 3,
    "minority": 1
  },
  "grouped_recommendations": [
    {
      "group_id": "grp_01",
      "bucket": "consensus",
      "support_count": 3,
      "proposal": "Move API contract definitions into a dedicated module."
    }
  ]
}

orch council report should support both machine and human output.

Suggested JSON shape:

{
  "ok": true,
  "command": "council-report",
  "run_id": "council_blog_001",
  "show": ["consensus", "majority"],
  "report_artifacts": [
    {
      "kind": "markdown",
      "path": ".orch/reports/council_blog_001.md"
    }
  ],
  "summary": {
    "consensus": 2,
    "majority": 3,
    "minority": 1
  }
}

Read-Only By Default

This workflow should be analysis-first by default.

• reviewers should inspect the target and return suggestions
• reviewers should not change code unless the user explicitly asks for proposal patches

If patch proposals are later enabled, they can still run through orch with worktrees, but that should be a separate mode.

When To Use It

Use this workflow when:

• the user explicitly asks for structured brainstorming
• the user wants multiple perspectives on a design or codebase
• the user wants proposals ranked by agreement
• the user wants a more rigorous alternative to a single-agent suggestion list

Do not use it when:

• the task is a simple direct implementation request
• a single code review is enough
• time or cost should be minimized

Relationship To Existing Docs

• architecture.md: overall system split
• orch-cli.md: leader-facing scheduling surface used to execute the council
• inbox-cli.md: transport used by reviewer tasks

Recommended Storage Shape

One simple approach is to add council-specific tables inside the shared orch database.

Suggested tables:

CREATE TABLE IF NOT EXISTS council_runs (
  run_id TEXT PRIMARY KEY,
  mode TEXT NOT NULL,
  target_type TEXT NOT NULL,
  output_mode TEXT NOT NULL,
  only_unanimous INTEGER NOT NULL DEFAULT 0,
  created_at TEXT NOT NULL,
  updated_at TEXT NOT NULL
);

CREATE TABLE IF NOT EXISTS council_inputs (
  run_id TEXT PRIMARY KEY,
  prompt TEXT NOT NULL DEFAULT '',
  target_file TEXT NOT NULL DEFAULT '',
  repo_path TEXT NOT NULL DEFAULT '',
  target_task_id TEXT NOT NULL DEFAULT '',
  created_at TEXT NOT NULL,
  updated_at TEXT NOT NULL
);

CREATE TABLE IF NOT EXISTS council_reviewers (
  run_id TEXT NOT NULL,
  reviewer_role TEXT NOT NULL,
  task_id TEXT NOT NULL,
  status TEXT NOT NULL,
  PRIMARY KEY (run_id, reviewer_role)
);

CREATE TABLE IF NOT EXISTS council_findings (
  run_id TEXT NOT NULL,
  reviewer_role TEXT NOT NULL,
  finding_id TEXT NOT NULL,
  title TEXT NOT NULL,
  summary TEXT NOT NULL,
  proposal TEXT NOT NULL,
  rationale TEXT NOT NULL,
  confidence TEXT NOT NULL,
  tags_json TEXT NOT NULL DEFAULT '[]',
  target_refs_json TEXT NOT NULL DEFAULT '{}',
  PRIMARY KEY (run_id, reviewer_role, finding_id)
);

CREATE TABLE IF NOT EXISTS council_groups (
  run_id TEXT NOT NULL,
  group_id TEXT NOT NULL,
  proposal TEXT NOT NULL,
  bucket TEXT NOT NULL,
  support_count INTEGER NOT NULL,
  supporters_json TEXT NOT NULL DEFAULT '[]',
  dissenters_json TEXT NOT NULL DEFAULT '[]',
  rationale_summary TEXT NOT NULL DEFAULT '',
  tags_json TEXT NOT NULL DEFAULT '[]',
  source_finding_ids_json TEXT NOT NULL DEFAULT '[]',
  PRIMARY KEY (run_id, group_id)
);

Embedded Skill Draft

The following block is a draft SKILL.md for the user-facing council review skill.

```markdown
---
name: council-review
description: Use this skill when the user explicitly asks for a multi-reviewer brainstorming or review workflow. It launches three reviewer roles with different perspectives, collects their suggestions, groups similar proposals, tallies agreement, and reports consensus, majority, and minority recommendations. Use it for structured idea generation or design critique, not for ordinary task execution.
---

# Council Review

Use this skill when the user wants a structured three-reviewer brainstorm or review.

## Reviewer Roles

- architecture-reviewer
- implementation-reviewer
- risk-reviewer

## Rules

- Treat this as an analysis workflow, not a code-writing workflow, unless the user explicitly asks for patch proposals.
- Use `orch council` as the execution surface.
- Default to the fixed reviewer roles architecture, implementation, and risk.
- Collect all three reviewer responses before tallying.
- Group similar suggestions before counting votes.
- Use `normal` grouping and `low|medium|high` confidence unless the user asks otherwise.
- If the user asks for unanimous-only output, show only `consensus`.
- Otherwise present `consensus` first, then `majority`, then optional `minority`.
- Support both text and repository context when available.
- Persist the council run and grouped outputs in `orch`.

## Typical Commands

```bash
orch council start --run council_blog_001 --target-file brief.md --target-type mixed --mode brainstorm --output both --json
orch council wait --run council_blog_001 --timeout-seconds 900 --json
orch council tally --run council_blog_001 --similarity normal --json
orch council report --run council_blog_001 --show consensus,majority --json
```
```