Add harness workflow and Material showcase design system
This commit is contained in:
@@ -0,0 +1,138 @@
|
||||
# Harness Engineering
|
||||
|
||||
Cadence UI already has good validation primitives. Harness engineering makes those primitives
|
||||
agent-usable by turning the repo into a system that can explain itself, accept explicit execution
|
||||
plans, and expose repeatable machine-runnable feedback loops.
|
||||
|
||||
## What it means in this repo
|
||||
|
||||
For Cadence UI, harness engineering means:
|
||||
|
||||
- the repository has clear system-of-record files for architecture, contracts, and release rules
|
||||
- non-trivial work starts with an execution plan checked into git
|
||||
- validation is exposed as stable suites that humans, agents, and CI can run the same way
|
||||
- known gaps are recorded explicitly instead of being rediscovered by every new task
|
||||
|
||||
This follows the direction described in OpenAI's harness engineering guidance: repositories should
|
||||
be more legible to agents, use explicit execution plans, and prefer faster feedback loops over
|
||||
purely ad hoc prompting.
|
||||
|
||||
## System Of Record
|
||||
|
||||
Agents and contributors should treat these files as the repository's baseline knowledge:
|
||||
|
||||
- `DESIGN.md`: active visual language, dynamic color direction, and motion rules
|
||||
- `README.md`: repo purpose, workspace layout, distribution modes, and QA surface
|
||||
- `CONTRIBUTING.md`: component contract, review expectations, and definition of done
|
||||
- `roadmap.md`: current system direction and planned component work
|
||||
- `packages/ui/src/lib/contracts.ts`: public authoring contract for components
|
||||
- `apps/docs/src/component-authoring.stories.tsx`: review surface for authoring rules
|
||||
- `docs/registry.md`: source-copy registry contract
|
||||
- `docs/releasing.md`: package release contract
|
||||
- `docs/rfcs/*`: design decisions that should not be silently bypassed
|
||||
- `AGENTS.md`: agent operating mode for this repository
|
||||
|
||||
## Execution Plans
|
||||
|
||||
Non-trivial changes should start with an execution plan under `docs/exec-plans/`.
|
||||
|
||||
Use an execution plan when the work:
|
||||
|
||||
- touches multiple repo surfaces such as `packages/ui`, `apps/docs`, `tests`, or `registry`
|
||||
- changes public component contracts or release behavior
|
||||
- introduces new dependencies, workflows, or automation
|
||||
- is large enough that another engineer or agent may need to resume it later
|
||||
|
||||
Plans should state:
|
||||
|
||||
- the problem or goal
|
||||
- constraints and non-goals
|
||||
- affected surfaces and likely files
|
||||
- validation suites to run
|
||||
- a status log with concrete checkpoints
|
||||
|
||||
## Validation Suites
|
||||
|
||||
Harness validation is exposed through `scripts/harness/validate.mjs` and root `pnpm` scripts.
|
||||
|
||||
Primary suites:
|
||||
|
||||
- `pnpm harness:validate:static`
|
||||
- lint and workspace typecheck for general repo changes
|
||||
- `pnpm harness:validate:component`
|
||||
- lint, typecheck, and unit coverage for normal component work
|
||||
- `pnpm harness:validate:docs`
|
||||
- Storybook build for docs-surface changes
|
||||
- `pnpm harness:validate:docs-smoke`
|
||||
- Playwright smoke coverage for high-value Storybook flows
|
||||
- `pnpm harness:validate:consumers`
|
||||
- registry metadata plus consumer smoke validation
|
||||
- `pnpm harness:validate:pr`
|
||||
- baseline pull request gate for packages, docs build, and consumer surfaces
|
||||
- `pnpm harness:validate:release`
|
||||
- full release gate, including browser-driven smoke coverage
|
||||
- `pnpm harness:validate:changed`
|
||||
- selects suites from git diff or working tree changes before validating
|
||||
|
||||
Each run writes a JSON report to `.artifacts/harness/<suite>.json`.
|
||||
|
||||
## Working Loop
|
||||
|
||||
Recommended change loop:
|
||||
|
||||
1. Read the relevant system-of-record files.
|
||||
2. Create or update an execution plan when the change is non-trivial.
|
||||
3. Modify the smallest surface that can prove the change.
|
||||
4. Run the narrowest useful harness suite first.
|
||||
5. Escalate to broader suites before merge.
|
||||
6. Record any skipped checks or known failures in the execution plan or PR.
|
||||
|
||||
## Diff-aware Selection
|
||||
|
||||
Harness selection is exposed through `pnpm harness:select` and `pnpm harness:validate:changed`.
|
||||
|
||||
- `pnpm harness:select`
|
||||
- reads the current working tree diff by default
|
||||
- `pnpm harness:select -- --from <ref> --to <ref>`
|
||||
- selects suites from an explicit git range
|
||||
- `pnpm harness:validate:changed`
|
||||
- runs the selected suites with a JSON report
|
||||
|
||||
Selection intentionally maps repo surfaces to validation surfaces:
|
||||
|
||||
- package source changes select `component`, `docs`, `docs-smoke`, and `consumers`
|
||||
- docs/story changes select `static`, `docs`, and `docs-smoke`
|
||||
- registry/consumer changes select `static` and `consumers`
|
||||
- doc-only or metadata-only changes may select no suites
|
||||
|
||||
## Worktree Orchestration
|
||||
|
||||
Cadence UI also exposes an orchestration wrapper:
|
||||
|
||||
- `pnpm harness:orch -- <orch command>`
|
||||
|
||||
The wrapper keeps orchestration state under `.artifacts/orch/` and applies worktree defaults for
|
||||
dispatch. Details live in [docs/orchestration.md](/Users/xd/project/cadence-ui/docs/orchestration.md).
|
||||
|
||||
## Current Rollout Scope
|
||||
|
||||
This repository is adding harness engineering in phases.
|
||||
|
||||
Phase 1 establishes:
|
||||
|
||||
- a documented harness workflow
|
||||
- execution-plan conventions
|
||||
- shared validation suites
|
||||
- a pull request workflow that runs the PR suite
|
||||
|
||||
Phase 2 adds:
|
||||
|
||||
- change-aware suite selection from git diff
|
||||
- a stabilized Storybook smoke harness
|
||||
- worktree-oriented orchestration defaults
|
||||
|
||||
Future phases can layer on:
|
||||
|
||||
- richer browser/app harnesses for interactive review
|
||||
- deeper plan-to-task automation for orchestration runs
|
||||
- stronger safety rails for agent-owned automation
|
||||
Reference in New Issue
Block a user