docs(harness): document control plane workflow
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
This commit is contained in:
@@ -0,0 +1,83 @@
|
||||
# Harness Control Plane Hardening
|
||||
|
||||
- Status: `in-progress`
|
||||
- Owner: `sisyphus`
|
||||
- Date: `2026-03-24`
|
||||
|
||||
## Goal
|
||||
|
||||
Finish the next harness-engineering hardening slice by making changed-suite selection authoritative in
|
||||
CI, promoting harness artifacts to first-class debugging outputs, centralizing browser and a11y
|
||||
coverage under one explicit contract, and turning orchestration from a thin wrapper into a safer
|
||||
repo capability with its own tests and docs.
|
||||
|
||||
## Scope
|
||||
|
||||
- In scope:
|
||||
- broaden harness selection so high-risk control-plane and release changes escalate to the right
|
||||
validation gate
|
||||
- make CI run the selected harness suites instead of always forcing the PR gate
|
||||
- upload harness, a11y, and browser test artifacts from CI
|
||||
- replace duplicated story coverage lists with one shared harness coverage contract
|
||||
- improve orchestration binary discovery, preflight checks, and plan-linked dispatch support
|
||||
- add tests for harness control-plane helpers and update docs to match the implementation
|
||||
- Out of scope:
|
||||
- redesigning the component library or Storybook content beyond harness metadata needs
|
||||
- replacing the external orchestration engine itself
|
||||
- introducing visual snapshot infrastructure or hosted browser grids
|
||||
|
||||
## Constraints
|
||||
|
||||
- Preserve the existing `pnpm harness:*` commands as the main public interface.
|
||||
- Keep package-first release validation as the canonical release path.
|
||||
- Reuse execution plans as the source of orchestration intent instead of creating a parallel plan
|
||||
format.
|
||||
- Keep new harness checks targeted so CI cost stays proportional to risk.
|
||||
|
||||
## Affected Surfaces
|
||||
|
||||
- `scripts/harness/*`
|
||||
- `tests/e2e/*`
|
||||
- `.github/workflows/harness-validate.yml`
|
||||
- `package.json`
|
||||
- `vitest.config.ts`
|
||||
- `docs/harness-engineering.md`
|
||||
- `docs/orchestration.md`
|
||||
- `README.md`
|
||||
- `CONTRIBUTING.md`
|
||||
|
||||
## Plan
|
||||
|
||||
1. Refactor harness suite selection so release-risk and harness-risk changes escalate beyond
|
||||
static checks and let `changed` execution become the CI entrypoint.
|
||||
2. Promote `.artifacts/*` outputs into uploaded CI artifacts and make changed-suite execution log
|
||||
its selected suites clearly.
|
||||
3. Introduce one shared harness coverage contract for Storybook browser smoke and a11y checks,
|
||||
then move both runners onto it.
|
||||
4. Add orchestration preflight and plan-linked dispatch helpers so execution plans directly support
|
||||
orchestration runs.
|
||||
5. Add control-plane tests plus doc updates so harness behavior, workflow guidance, and public
|
||||
commands stay aligned.
|
||||
|
||||
## Validation
|
||||
|
||||
- `pnpm exec vitest run scripts/harness/**/*.test.ts`
|
||||
- `pnpm harness:suites`
|
||||
- `pnpm harness:select -- --json --changed-file scripts/harness/core.mjs`
|
||||
- `pnpm harness:validate:changed -- --dry-run --changed-file scripts/harness/core.mjs`
|
||||
- `pnpm test:e2e:a11y`
|
||||
- `pnpm test:e2e:smoke`
|
||||
- `pnpm harness:validate:pr`
|
||||
|
||||
## Orchestration Task Sketch
|
||||
|
||||
- `T1`: harden suite selection and CI execution flow
|
||||
- `T2`: centralize browser and a11y coverage contract
|
||||
- `T3`: improve orchestration preflight and plan-linked dispatch
|
||||
- `T4`: add harness tests and reconcile docs
|
||||
|
||||
## Status Log
|
||||
|
||||
- `2026-03-24 18:40` reviewed harness docs, scripts, CI wiring, and recent release hardening work;
|
||||
confirmed the main gaps are authoritative suite selection, CI artifact visibility, explicit
|
||||
browser coverage ownership, orchestration preflight, and missing control-plane tests
|
||||
@@ -40,6 +40,9 @@ Describe the outcome in one short paragraph.
|
||||
## Orchestration Task Sketch
|
||||
|
||||
- Optional task IDs and dependencies when the work should be dispatched through `pnpm harness:orch`
|
||||
- Suggested format:
|
||||
- `T1`: isolated first slice
|
||||
- `T2 -> T1`: follow-up slice that depends on `T1`
|
||||
|
||||
## Status Log
|
||||
|
||||
|
||||
Reference in New Issue
Block a user