428 lines
13 KiB
Markdown
428 lines
13 KiB
Markdown
# Web Product Monorepo Plan
|
|
|
|
## Purpose
|
|
|
|
This document defines the recommended repository shape and implementation direction for evolving the current local CLI-based coordination stack into a formal web product.
|
|
|
|
The target shape is:
|
|
|
|
- a React frontend for leaders and operators
|
|
- a Go HTTP backend built with `chi`
|
|
- continued reuse of the existing `inbox` and `orch` domain logic
|
|
- one monorepo rather than separate frontend and backend repositories
|
|
|
|
This is an implementation-oriented planning document.
|
|
It does not replace the current protocol and workflow documents for `inbox`, `orch`, worktrees, or council review.
|
|
|
|
## Decision Summary
|
|
|
|
The recommended direction is:
|
|
|
|
- keep a single monorepo
|
|
- add a standalone frontend app under `apps/web`
|
|
- add a standalone HTTP backend entrypoint, now owned under `packages/operator-api/cmd/operator-api`
|
|
- keep the current Go repository as the backend source of truth
|
|
- evolve the backend by introducing shared application services and UI-oriented query layers
|
|
- avoid building the web backend by shelling out to the existing CLIs
|
|
|
|
This gives the product a front/back runtime split without creating a second backend codebase.
|
|
|
|
## Why Monorepo
|
|
|
|
For the current project stage, a monorepo is the most practical choice.
|
|
|
|
Benefits:
|
|
|
|
- the current domain model, scheduler logic, and SQLite schema already live in this repository
|
|
- frontend and backend changes will often move together while the web product is still forming
|
|
- API contracts, event contracts, and backend changes can be versioned together
|
|
- the existing CLI tools can stay in the same repo as operator and debugging surfaces
|
|
- shared scripts, fixtures, and docs remain simple
|
|
|
|
## Why The Backend Should Stay In This Repo
|
|
|
|
The backend should become a first-class HTTP service, but it should not start as a brand new repo.
|
|
|
|
Reasons:
|
|
|
|
- the existing state machine and storage behavior already live here
|
|
- duplicating `store` and schema logic into a second backend codebase would create unnecessary drift
|
|
- wrapping the CLIs from a web server would produce an unstable and hard-to-test integration boundary
|
|
- the fastest safe path is to let both the CLI and HTTP service call the same Go services
|
|
|
|
The backend may be split into another repo later only after the application-service boundary is stable and intentionally extracted.
|
|
|
|
## Product Direction
|
|
|
|
The target product is not just a local dashboard.
|
|
It should be able to evolve into a formal multi-user web application.
|
|
|
|
That means the backend should be designed for:
|
|
|
|
- stable HTTP APIs
|
|
- authenticated users
|
|
- eventual organization and membership concepts
|
|
- server-pushed run and thread updates
|
|
- UI-oriented read models for task boards, blocked queues, and timelines
|
|
- eventual migration away from SQLite if write concurrency or deployment shape requires it
|
|
|
|
## Recommended Repository Layout
|
|
|
|
The recommended monorepo shape is:
|
|
|
|
```text
|
|
.
|
|
├─ apps/
|
|
│ └─ web/ # React frontend application
|
|
├─ api/
|
|
│ ├─ openapi.yaml # HTTP API contract
|
|
│ └─ events.md # SSE or websocket event contract
|
|
├─ cmd/
|
|
│ ├─ inbox/ # existing worker-facing CLI
|
|
│ ├─ orch/ # existing leader-facing CLI
|
|
│ └─ orchd/ # new HTTP backend entrypoint
|
|
├─ internal/
|
|
│ ├─ app/ # shared application services used by CLI and HTTP
|
|
│ ├─ httpapi/ # chi router, handlers, middleware, SSE
|
|
│ ├─ query/ # frontend-oriented read models and aggregate queries
|
|
│ ├─ store/ # existing persistence and orchestration state logic
|
|
│ ├─ db/ # existing SQLite open and migration layer
|
|
│ └─ protocol/ # shared DTO and response helpers where appropriate
|
|
├─ docs/
|
|
├─ scripts/
|
|
├─ package.json
|
|
├─ pnpm-workspace.yaml
|
|
├─ go.mod
|
|
└─ Makefile
|
|
```
|
|
|
|
## Layout Decisions
|
|
|
|
### `apps/web`
|
|
|
|
The frontend should live under `apps/web`.
|
|
|
|
Recommended stack:
|
|
|
|
- React
|
|
- Vite
|
|
- TypeScript
|
|
- TanStack Router
|
|
- TanStack Query
|
|
|
|
The frontend should consume only HTTP APIs and event streams from the backend.
|
|
It should never read SQLite directly and should not depend on CLI output parsing.
|
|
|
|
### `packages/operator-api`
|
|
|
|
The backend HTTP service should live in a package-owned runtime with its Go binary under `packages/operator-api/cmd/operator-api`.
|
|
|
|
Responsibilities:
|
|
|
|
- expose REST or RPC-style HTTP endpoints for the web frontend
|
|
- expose SSE or websocket updates for run and thread activity
|
|
- load the same database as the CLI tools
|
|
- use shared application services rather than reimplementing logic
|
|
|
|
The existing orch and inbox runtimes should remain available as operator and debugging tools through their package-owned binaries under `packages/orch-runtime/cmd/orch` and `packages/inbox-runtime/cmd/inbox`.
|
|
|
|
### `packages/operator-api/internal/app`
|
|
|
|
This package should become the shared application-service layer.
|
|
|
|
It should own business actions that currently sit mostly in CLI command handlers, such as:
|
|
|
|
- dispatch
|
|
- reconcile
|
|
- answer
|
|
- retry
|
|
- reassign
|
|
- cancel
|
|
- council start, wait, tally, and report orchestration entrypoints
|
|
|
|
Both CLI commands and HTTP handlers should call into this layer.
|
|
|
|
### `packages/operator-api/internal/query`
|
|
|
|
This package should hold read models specifically shaped for the web UI.
|
|
|
|
Examples:
|
|
|
|
- run overview with task counts and recent event summaries
|
|
- kanban-style task board grouped by status
|
|
- blocked queue with latest question details
|
|
- thread timeline including artifacts and lease state
|
|
- council run summaries and grouped recommendation views
|
|
|
|
These should be query-oriented models rather than raw database table mirrors.
|
|
|
|
### `packages/operator-api/internal/httpapi`
|
|
|
|
This package should hold the web transport layer.
|
|
|
|
Recommended contents:
|
|
|
|
- `router.go`
|
|
- handler packages by domain
|
|
- auth and request middleware
|
|
- error mapping
|
|
- SSE handlers
|
|
- request validation and response serialization
|
|
|
|
The HTTP layer should stay thin.
|
|
It should not own business rules or embed SQL-heavy logic.
|
|
|
|
## Why `chi`
|
|
|
|
`chi` is the recommended HTTP router for this product direction.
|
|
|
|
Reasons:
|
|
|
|
- it stays close to standard `net/http`
|
|
- it provides cleaner routing and middleware composition than a bare `ServeMux`
|
|
- it avoids the heavier framework lock-in of `gin`
|
|
- it fits a backend that is expected to grow into a real product API
|
|
|
|
The decision here is not about raw performance.
|
|
It is about long-term code organization and maintainable HTTP structure.
|
|
|
|
## Why Not Build The Backend Around The Existing CLIs
|
|
|
|
The new web backend should not execute `orch` or `inbox` binaries as subprocesses.
|
|
|
|
That approach would create avoidable problems:
|
|
|
|
- duplicate input validation paths
|
|
- harder error mapping
|
|
- unstable process-level integration points
|
|
- slower request handling
|
|
- more difficult testing
|
|
- awkward transaction and context handling
|
|
|
|
The CLIs should become alternate entrypoints into shared services, not the implementation boundary for the web product.
|
|
|
|
## API Contract Direction
|
|
|
|
The monorepo should add an explicit API contract under `api/`.
|
|
|
|
Recommended starting files:
|
|
|
|
- `api/openapi.yaml`
|
|
- `api/events.md`
|
|
|
|
`openapi.yaml` should define:
|
|
|
|
- runs list and detail endpoints
|
|
- task list and task action endpoints
|
|
- blocked queue endpoints
|
|
- thread detail endpoints
|
|
- council report endpoints
|
|
|
|
`events.md` should define:
|
|
|
|
- event stream transport choice
|
|
- cursoring model
|
|
- payload shapes for run, task, thread, and council events
|
|
|
|
The web frontend should generate a typed client from the HTTP contract where practical.
|
|
|
|
## Frontend Direction
|
|
|
|
The frontend should be a standalone app in the monorepo, not embedded templates.
|
|
|
|
Recommended responsibilities:
|
|
|
|
- run dashboard
|
|
- task board
|
|
- blocked queue
|
|
- thread detail and timeline
|
|
- council result and report views
|
|
- operator actions such as answer, retry, reassign, and cancel
|
|
|
|
Recommended development model:
|
|
|
|
- local dev server from `apps/web`
|
|
- proxy API requests to `orchd`
|
|
- consume typed API clients instead of hand-written fetch calls where possible
|
|
|
|
## Backend Evolution Plan
|
|
|
|
The current project already has strong backend assets:
|
|
|
|
- durable schema and migrations
|
|
- store-layer logic for `inbox` and `orch`
|
|
- stable JSON conventions
|
|
- working CLI surfaces
|
|
|
|
The backend should therefore evolve in-place:
|
|
|
|
1. keep the current store and schema as the persistence foundation
|
|
2. extract shared application actions into the orchd runtime app layer
|
|
3. add UI-oriented read models in the orchd runtime query layer
|
|
4. add the orchd runtime package and its HTTP transport layer
|
|
5. attach the React frontend after the API contract stabilizes
|
|
|
|
This is safer than first attempting a large repo split or backend rewrite.
|
|
|
|
## Suggested Initial Endpoint Set
|
|
|
|
The first useful web slice should remain narrow.
|
|
|
|
Recommended read endpoints:
|
|
|
|
- `GET /api/runs`
|
|
- `GET /api/runs/{runID}`
|
|
- `GET /api/runs/{runID}/tasks`
|
|
- `GET /api/runs/{runID}/blocked`
|
|
- `GET /api/threads/{threadID}`
|
|
- `GET /api/council/runs/{runID}`
|
|
|
|
Recommended write endpoints:
|
|
|
|
- `POST /api/runs/{runID}/tasks/{taskID}/answer`
|
|
- `POST /api/runs/{runID}/tasks/{taskID}/retry`
|
|
- `POST /api/runs/{runID}/tasks/{taskID}/reassign`
|
|
- `POST /api/runs/{runID}/tasks/{taskID}/cancel`
|
|
|
|
Recommended realtime endpoint:
|
|
|
|
- `GET /api/events/stream`
|
|
|
|
The first UI should prioritize observation over broad mutation.
|
|
|
|
## Realtime Direction
|
|
|
|
The current system already has a shared `events` table for monotonic activity tracking.
|
|
That makes SSE a good first realtime transport.
|
|
|
|
Recommended approach:
|
|
|
|
- begin with SSE rather than websockets
|
|
- stream cursor-based events from the backend
|
|
- let the frontend use events to invalidate or refresh query results
|
|
- keep the server-side event payloads close to run, task, thread, and council activity
|
|
|
|
Websockets can be reconsidered later if bidirectional session behavior becomes necessary.
|
|
|
|
## Database Direction
|
|
|
|
SQLite remains appropriate for the current CLI-focused local workflow and can also support early web product development in controlled environments.
|
|
|
|
However, if the product becomes truly multi-user or centrally hosted, the backend should expect a later database transition.
|
|
|
|
Recommended position:
|
|
|
|
- keep SQLite during the first web API and frontend milestone
|
|
- avoid SQLite-specific assumptions in the new application-service and query layers
|
|
- plan for a future Postgres migration if concurrency, deployment, or tenancy requirements grow
|
|
|
|
The goal is to avoid rewriting product boundaries just because the storage engine changes later.
|
|
|
|
## Implementation Phases
|
|
|
|
### Phase 1: Monorepo Skeleton And Backend Service
|
|
|
|
Add:
|
|
|
|
- `apps/web`
|
|
- `packages/operator-api/cmd/operator-api`
|
|
- `packages/operator-api/internal/httpapi`
|
|
- `packages/operator-api/internal/app`
|
|
- `packages/operator-api/internal/query`
|
|
- `api/openapi.yaml`
|
|
- `api/events.md`
|
|
- root JS workspace files such as `package.json` and `pnpm-workspace.yaml`
|
|
|
|
Deliverable:
|
|
|
|
- the repository builds with the new monorepo structure in place
|
|
- the backend can serve a small read-only API
|
|
|
|
### Phase 2: Read-Only Web UI
|
|
|
|
Build:
|
|
|
|
- runs page
|
|
- run detail view
|
|
- task board
|
|
- blocked queue
|
|
- thread detail
|
|
|
|
Deliverable:
|
|
|
|
- the React frontend can inspect live orchestration state through `orchd`
|
|
|
|
### Phase 3: Realtime Updates
|
|
|
|
Add:
|
|
|
|
- SSE event endpoint
|
|
- frontend subscription and refresh behavior
|
|
|
|
Deliverable:
|
|
|
|
- run and thread views update without manual reload loops
|
|
|
|
### Phase 4: Write Operations
|
|
|
|
Add:
|
|
|
|
- answer
|
|
- retry
|
|
- reassign
|
|
- cancel
|
|
|
|
Deliverable:
|
|
|
|
- the web UI becomes a practical operator surface rather than read-only reporting
|
|
|
|
### Phase 5: Product Hardening
|
|
|
|
Add:
|
|
|
|
- auth
|
|
- user and organization concepts
|
|
- audit trails
|
|
- deployment configuration
|
|
- database migration planning beyond SQLite if needed
|
|
|
|
Deliverable:
|
|
|
|
- the system can evolve beyond a local operator tool into a formal product backend
|
|
|
|
## Non-Goals For The First Web Milestone
|
|
|
|
Do not treat these as part of the initial skeleton task:
|
|
|
|
- a full permissions model
|
|
- background job infrastructure
|
|
- multi-tenant isolation
|
|
- websocket-first architecture
|
|
- immediate database migration
|
|
- full replacement of the CLI workflows
|
|
|
|
The first milestone should create a durable path, not solve every product concern at once.
|
|
|
|
## Migration Notes For Existing Code
|
|
|
|
The current codebase should be adapted incrementally.
|
|
|
|
Recommended rules:
|
|
|
|
- keep the package-owned `orch` and `inbox` binaries working while the HTTP layer is introduced
|
|
- move business logic out of command handlers only when the shared service abstraction is clear
|
|
- do not rewrite `store` just to match HTTP names
|
|
- let the frontend shape drive the orchd runtime query layer, not the table layout alone
|
|
- avoid introducing duplicate orchestration logic between CLI and HTTP
|
|
|
|
## Immediate Recommended Next Step
|
|
|
|
If the project begins this web-product milestone, the first implementation slice should be:
|
|
|
|
1. scaffold the monorepo workspace files and `apps/web`
|
|
2. add the `orchd` runtime package with a `chi` router and a minimal health or runs endpoint
|
|
3. introduce the first shared application or query boundary rather than letting handlers call storage ad hoc
|
|
4. write the initial API contract in `api/openapi.yaml`
|
|
|
|
This gives the project a clean backbone before deeper UI work begins.
|