kurihada/ai-workflow-skill
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7840b2767f16bfad41f975ff63cddd3271ed5045
ai-workflow-skill/docs/tests/orch-skill/leader-reassigns-blocked-task-through-bundled-cli.md
T

6.4 KiB
Raw Blame History

Case: leader-reassigns-blocked-task-through-bundled-cli

Test Type

This is a forward-test and a reassignment-path skill validation.

The goal is to verify that a leader using the packaged orch skill can observe a blocked task, reassign it from one worker to another, and drive the run to completion through the new attempt.

Purpose

Validate that all of the following can be true at the same time:

  • the leader can use blocked, reassign, reconcile, and status through the bundled orch skill
  • worker-a can claim the original attempt and block on a question
  • worker-b can receive the reassigned attempt as a new thread
  • the original thread is cancelled and the new thread reaches done
  • the final run reaches done

Preconditions

  • orch skill path exists: ORCH_SKILL_PATH=skills/orch
  • inbox skill path exists: INBOX_SKILL_PATH=skills/inbox
  • bundled CLI executables exist at ORCH_SKILL_PATH/assets/orch and INBOX_SKILL_PATH/assets/inbox
  • use an empty temporary directory TMPDIR
  • initialize TMPDIR/coord.db before launching role agents through INBOX_SKILL_PATH/assets/inbox --db TMPDIR/coord.db --json init

Agent Topology

  • leader
  • worker-a
  • worker-b

Inputs

Leader Prompt

Use $orch at ORCH_SKILL_PATH to act as leader on the already initialized SQLite DB TMPDIR/coord.db. Only coordinate through the bundled orch CLI from the skill. Workflow: 1) create run run_blog_skill_reassign_001, 2) add and dispatch one task T1 to worker-a, 3) wait until worker-a blocks, 4) inspect blocked tasks, 5) reassign T1 to worker-b with a short reason, 6) wait until worker-b completes the new attempt, 7) reconcile and inspect final status, 8) stop after reporting THREAD_ID_1 and THREAD_ID_2. Do not use ordinary chat to coordinate with the workers.

Worker A Prompt

Use $inbox at INBOX_SKILL_PATH to act as worker-a on SQLite DB TMPDIR/coord.db. Only coordinate through the bundled inbox CLI from the skill. Workflow: 1) fetch and claim the initial assigned thread, 2) send one blocked update with a precise question, 3) stop after reporting THREAD_ID_1 and the blocked summary you sent. Do not use ordinary chat to coordinate with the leader or worker-b.

Worker B Prompt

Use $inbox at INBOX_SKILL_PATH to act as worker-b on SQLite DB TMPDIR/coord.db. Only coordinate through the bundled inbox CLI from the skill. Workflow: 1) wait until reassigned work for worker-b appears, 2) fetch and claim it, 3) complete it with done, 4) stop after reporting THREAD_ID_2. Do not use ordinary chat to coordinate with the leader or worker-a.

Execution Parameters

  • use the shared execution contract from README.md
  • use the shared timeout defaults from README.md
  • do not override the default cleanup policy

Execution Steps

  1. Initialize TMPDIR/coord.db once through the bundled inbox CLI before launching agents
  2. Inject skills/orch/ into leader
  3. Inject skills/inbox/ into worker-a and worker-b
  4. Point all agents at the same database path TMPDIR/coord.db
  5. Launch leader, worker-a, and worker-b in parallel
  6. Wait for all agents to finish
  7. Resolve THREAD_ID_1 and THREAD_ID_2 from the agent outputs
  8. Independently run the validation commands from the main thread

Validation Commands

ORCH_SKILL_PATH/assets/orch --db TMPDIR/coord.db --json status --run run_blog_skill_reassign_001
INBOX_SKILL_PATH/assets/inbox --db TMPDIR/coord.db --json show --thread THREAD_ID_1
INBOX_SKILL_PATH/assets/inbox --db TMPDIR/coord.db --json show --thread THREAD_ID_2

Expected Outcomes

  • worker-a successfully claims the original thread and blocks it
  • the leader successfully reassigns the task to worker-b
  • the original thread reaches cancelled
  • worker-b receives a distinct reassigned thread and completes it
  • the final run reaches done

Assertions

  • THREAD_ID_1 != THREAD_ID_2
  • status.data.run.status == "done"
  • status.data.tasks[0].status == "done"
  • show THREAD_ID_1 reports a terminal cancelled thread state
  • show THREAD_ID_2 reports a terminal done thread state
  • the blocked question remains visible in the original thread history

Cleanup

  • use the default cleanup policy from README.md
  • if the run fails, retain TMPDIR and coord.db for replay and manual inspection

Recorded Example Run

  • recorded on: 2026-03-19
  • execution mode: direct_cli_replay via scripts/run_orch_skill_forward_tests.sh
  • result: pass
  • observed run id: run_blog_skill_reassign_001
  • observed original thread id: thr_0a61240412134de3b3d9ab219b6c8f19
  • observed reassigned thread id: thr_12fbcf6d89d948548306198d013d77a5
  • evidence summary:
  • orch wait --for task_blocked woke after worker-a posted a blocked question with payload Proceed with v1 scope?
  • orch reassign --run run_blog_skill_reassign_001 --task T1 --to worker-b --json returned attempt_no == 2 and assigned the new attempt to worker-b
  • final inbox show on the original thread returned thread.status == "cancelled" and preserved the blocked question message
  • final inbox show on the reassigned thread returned thread.status == "done"
  • final orch status --run run_blog_skill_reassign_001 --json returned run.status == "done" and tasks[0].status == "done"
  • note: this recorded run exercised the packaged binaries directly in a temporary DB and did not spawn separate Codex role agents

Recorded Real Forward Run

  • recorded on: 2026-03-19
  • execution mode: real_subagent_forward_test
  • result: pass
  • evidence root: /tmp/orch-skill-subagents.J1XWgs/leader-reassigns-blocked-task-through-bundled-cli-phased
  • observed run id: run_blog_skill_reassign_001
  • observed original thread id: thr_7d43af5bc1f7467da98a39adb0de5808
  • observed reassigned thread id: thr_eba253db8965423b855d0c784a29702c
  • evidence summary:
  • the same real leader agent using skills/orch/ completed the case in three phases: initial run/task/dispatch, then wait --for task_blocked plus reassign, then final wait --for task_done plus status
  • a real worker-a agent using skills/inbox/ claimed the original thread and posted the blocked question Proceed with v1 scope?
  • a real worker-b agent using skills/inbox/ claimed the reassigned thread and completed it
  • main-thread validation confirmed the original thread finished cancelled, the reassigned thread finished done, and the original blocked question remained visible in thread history
Reference in New Issue View Git Blame Copy Permalink