kurihada
800db360b3
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
84 lines
3.3 KiB
Markdown
84 lines
3.3 KiB
Markdown
# Harness Control Plane Hardening
|
|
|
|
- Status: `in-progress`
|
|
- Owner: `sisyphus`
|
|
- Date: `2026-03-24`
|
|
|
|
## Goal
|
|
|
|
Finish the next harness-engineering hardening slice by making changed-suite selection authoritative in
|
|
CI, promoting harness artifacts to first-class debugging outputs, centralizing browser and a11y
|
|
coverage under one explicit contract, and turning orchestration from a thin wrapper into a safer
|
|
repo capability with its own tests and docs.
|
|
|
|
## Scope
|
|
|
|
- In scope:
|
|
- broaden harness selection so high-risk control-plane and release changes escalate to the right
|
|
validation gate
|
|
- make CI run the selected harness suites instead of always forcing the PR gate
|
|
- upload harness, a11y, and browser test artifacts from CI
|
|
- replace duplicated story coverage lists with one shared harness coverage contract
|
|
- improve orchestration binary discovery, preflight checks, and plan-linked dispatch support
|
|
- add tests for harness control-plane helpers and update docs to match the implementation
|
|
- Out of scope:
|
|
- redesigning the component library or Storybook content beyond harness metadata needs
|
|
- replacing the external orchestration engine itself
|
|
- introducing visual snapshot infrastructure or hosted browser grids
|
|
|
|
## Constraints
|
|
|
|
- Preserve the existing `pnpm harness:*` commands as the main public interface.
|
|
- Keep package-first release validation as the canonical release path.
|
|
- Reuse execution plans as the source of orchestration intent instead of creating a parallel plan
|
|
format.
|
|
- Keep new harness checks targeted so CI cost stays proportional to risk.
|
|
|
|
## Affected Surfaces
|
|
|
|
- `scripts/harness/*`
|
|
- `tests/e2e/*`
|
|
- `.github/workflows/harness-validate.yml`
|
|
- `package.json`
|
|
- `vitest.config.ts`
|
|
- `docs/harness-engineering.md`
|
|
- `docs/orchestration.md`
|
|
- `README.md`
|
|
- `CONTRIBUTING.md`
|
|
|
|
## Plan
|
|
|
|
1. Refactor harness suite selection so release-risk and harness-risk changes escalate beyond
|
|
static checks and let `changed` execution become the CI entrypoint.
|
|
2. Promote `.artifacts/*` outputs into uploaded CI artifacts and make changed-suite execution log
|
|
its selected suites clearly.
|
|
3. Introduce one shared harness coverage contract for Storybook browser smoke and a11y checks,
|
|
then move both runners onto it.
|
|
4. Add orchestration preflight and plan-linked dispatch helpers so execution plans directly support
|
|
orchestration runs.
|
|
5. Add control-plane tests plus doc updates so harness behavior, workflow guidance, and public
|
|
commands stay aligned.
|
|
|
|
## Validation
|
|
|
|
- `pnpm exec vitest run scripts/harness/**/*.test.ts`
|
|
- `pnpm harness:suites`
|
|
- `pnpm harness:select -- --json --changed-file scripts/harness/core.mjs`
|
|
- `pnpm harness:validate:changed -- --dry-run --changed-file scripts/harness/core.mjs`
|
|
- `pnpm test:e2e:a11y`
|
|
- `pnpm test:e2e:smoke`
|
|
- `pnpm harness:validate:pr`
|
|
|
|
## Orchestration Task Sketch
|
|
|
|
- `T1`: harden suite selection and CI execution flow
|
|
- `T2`: centralize browser and a11y coverage contract
|
|
- `T3`: improve orchestration preflight and plan-linked dispatch
|
|
- `T4`: add harness tests and reconcile docs
|
|
|
|
## Status Log
|
|
|
|
- `2026-03-24 18:40` reviewed harness docs, scripts, CI wiring, and recent release hardening work;
|
|
confirmed the main gaps are authoritative suite selection, CI artifact visibility, explicit
|
|
browser coverage ownership, orchestration preflight, and missing control-plane tests
|