Harness Engineering

Cadence UI already has good validation primitives. Harness engineering makes those primitives agent-usable by turning the repo into a system that can explain itself, accept explicit execution plans, and expose repeatable machine-runnable feedback loops.

What it means in this repo

For Cadence UI, harness engineering means:

the repository has clear system-of-record files for architecture, contracts, and release rules
non-trivial work starts with an execution plan checked into git
validation is exposed as stable suites that humans, agents, and CI can run the same way
known gaps are recorded explicitly instead of being rediscovered by every new task

This follows the direction described in OpenAI's harness engineering guidance: repositories should be more legible to agents, use explicit execution plans, and prefer faster feedback loops over purely ad hoc prompting.

System Of Record

Agents and contributors should treat these files as the repository's baseline knowledge:

DESIGN.md: active visual language, dynamic color direction, and motion rules
README.md: repo purpose, workspace layout, distribution modes, and QA surface
CONTRIBUTING.md: component contract, review expectations, and definition of done
roadmap.md: current system direction and planned component work
packages/ui/src/lib/contracts.ts: public authoring contract for components
apps/docs/src/component-authoring.stories.tsx: review surface for authoring rules
docs/registry.md: source-copy registry contract
docs/releasing.md: package release contract
docs/rfcs/*: design decisions that should not be silently bypassed
AGENTS.md: agent operating mode for this repository

Execution Plans

Non-trivial changes should start with an execution plan under docs/exec-plans/.

Use an execution plan when the work:

touches multiple repo surfaces such as packages/ui, apps/docs, tests, or registry
changes public component contracts or release behavior
introduces new dependencies, workflows, or automation
is large enough that another engineer or agent may need to resume it later

Plans should state:

the problem or goal
constraints and non-goals
affected surfaces and likely files
validation suites to run
a status log with concrete checkpoints

Validation Suites

Harness validation is exposed through scripts/harness/validate.mjs and root pnpm scripts.

Primary suites:

pnpm harness:validate:static
- lint and workspace typecheck for general repo changes
pnpm harness:validate:component
- lint, typecheck, and unit coverage for normal component work
pnpm harness:validate:docs
- Storybook build for docs-surface changes
pnpm harness:validate:docs-smoke
- Playwright smoke coverage for high-value Storybook flows
pnpm harness:validate:consumers
- registry metadata plus consumer smoke validation
pnpm harness:validate:pr
- baseline pull request gate for packages, docs build, and consumer surfaces
pnpm harness:validate:release
- full release gate, including browser-driven smoke coverage
pnpm harness:validate:changed
- selects suites from git diff or working tree changes before validating

Each run writes a JSON report to .artifacts/harness/<suite>.json.

Working Loop

Recommended change loop:

Read the relevant system-of-record files.
Create or update an execution plan when the change is non-trivial.
Modify the smallest surface that can prove the change.
Run the narrowest useful harness suite first.
Escalate to broader suites before merge.
Record any skipped checks or known failures in the execution plan or PR.

Diff-aware Selection

Harness selection is exposed through pnpm harness:select and pnpm harness:validate:changed.

pnpm harness:select
- reads the current working tree diff by default
pnpm harness:select -- --from <ref> --to <ref>
- selects suites from an explicit git range
pnpm harness:validate:changed
- runs the selected suites with a JSON report

Selection intentionally maps repo surfaces to validation surfaces:

package source changes select component, docs, docs-smoke, and consumers
docs/story changes select static, docs, and docs-smoke
registry/consumer changes select static and consumers
doc-only or metadata-only changes may select no suites

Worktree Orchestration

Cadence UI also exposes an orchestration wrapper:

pnpm harness:orch -- <orch command>

The wrapper keeps orchestration state under .artifacts/orch/ and applies worktree defaults for dispatch. Details live in docs/orchestration.md.

Current Rollout Scope

This repository is adding harness engineering in phases.

Phase 1 establishes:

a documented harness workflow
execution-plan conventions
shared validation suites
a pull request workflow that runs the PR suite

Phase 2 adds:

change-aware suite selection from git diff
a stabilized Storybook smoke harness
worktree-oriented orchestration defaults

Future phases can layer on:

richer browser/app harnesses for interactive review
deeper plan-to-task automation for orchestration runs
stronger safety rails for agent-owned automation

5.2 KiB Raw Blame History