Add orch Markdown test-plan skeleton

2026-03-19 16:00:11 +08:00
parent 7f3eb1c24b
commit dd8da4aa58
25 changed files with 724 additions and 4 deletions
@@ -0,0 +1,121 @@
+# Orch Shared Test Conventions
+
+## Purpose
+
+This document captures shared assumptions used by multiple `orch` test-plan documents so command and workflow files can stay focused on behavior instead of repeating setup boilerplate.
+
+## Recommended Fixture Shape
+
+Use an isolated temp workspace per case:
+
+- database path: `TMPDIR/coord.db`
+- optional repository path: `TMPDIR/repo`
+- optional workspace root: `TMPDIR/worktrees` or `REPO/.orch/worktrees`
+- optional body file: `TMPDIR/body.md`
+
+Recommended minimal bootstrap command:
+
+```bash
+orch --db TMPDIR/coord.db --json run init --run run_demo_001 --goal "Demo run"
+```
+
+Some cases need additional bootstrap:
+
+- `task add` for task-oriented flows
+- `dep add` for dependency-gating flows
+- `inbox` commands when simulating worker claim, progress, blocked, done, or fail transitions
+
+## Global Flags
+
+Root-level flags apply to every `orch` subcommand:
+
+- `--db`: SQLite database path, default `.agents/coord.db`
+- `--json`: emit machine-readable JSON
+
+The current root command does not define a global `--agent`; worker-side activity should be modeled through `inbox` using the same database.
+
+## Success JSON Contract
+
+Successful JSON output uses this shape:
+
+```json
+{
+  "ok": true,
+  "command": "dispatch",
+  "data": {}
+}
+```
+
+Shared assertion points:
+
+- `ok` is `true`
+- `command` matches the invoked leaf command
+- `data` contains the command-specific payload
+
+## Error JSON Contract
+
+Failure JSON output uses this shape:
+
+```json
+{
+  "ok": false,
+  "error": {
+    "code": "invalid_input",
+    "message": "..."
+  }
+}
+```
+
+Shared assertion points:
+
+- `ok` is `false`
+- `error.code` matches the stable contract
+- `error.message` is present and human-readable
+
+## Exit Code Contract
+
+The current `orch` CLI contract primarily uses these exit codes:
+
+| Exit Code | Meaning | Typical Error Code |
+| --- | --- | --- |
+| `0` | success | none |
+| `30` | invalid input, invalid state, or usage-style error | `invalid_input` or `invalid_state` |
+| `40` | referenced run, task, or thread missing | `not_found` |
+| `50` | unexpected internal failure | `internal_error` |
+
+When a case expects failure, assert both the exit code and the JSON error code.
+
+## Body Input Rules
+
+Commands that support `--body` and `--body-file` should be documented with these shared rules:
+
+- `--body` and `--body-file` are mutually exclusive
+- `--body-file` content is read verbatim into the dispatched or answer body
+- unreadable `--body-file` should be treated as `invalid_input`
+
+Relevant commands:
+
+- `dispatch`
+- `answer`
+- `retry`
+
+## Worktree And Repo Rules
+
+Cases covering worktree behavior should state:
+
+- whether the source repository is clean or dirty
+- whether `--strict-worktree` is enabled explicitly or inferred automatically
+- whether `--base-ref` is omitted or explicitly provided
+- where the expected worktree path should be created
+
+When worktree behavior is under test, assert at least:
+
+- attempt `base_ref`
+- attempt `base_commit`
+- attempt `branch_name`
+- attempt `worktree_path`
+- attempt `workspace_status`
+
+## Workflow Authoring Rule
+
+If a case spans multiple `orch` commands, place the end-to-end narrative in `workflows/README.md` first, then add narrower command-level cases only when they are easier to reason about in isolation.