wololo
Get access

Agent Dispatch & Monitoring

The dispatch system is the engine that drives the pipeline. It runs on a schedule (cron), scans for work at each stage, and spawns agents when conditions are met. Think of it as a scheduler with concurrency control and zombie detection.

How Dispatch Works

Every 15 minutes:
  1. SCAN    — Query GitHub for issues/PRs at each stage
  2. FILTER  — Apply entry conditions + concurrency limits
  3. SPAWN   — Start coding agents for eligible work items
  4. CLEANUP — Kill zombie sessions, prune completed worktrees

Dispatch Cycle

A single dispatch cycle evaluates all stages in order. Earlier stages have priority — research runs before build, build before fix. This prevents a backlog of unresearched issues while fix agents consume all capacity.

OrderStageQueryAction
1ResearchIssues in backlog without classification labelSpawn cloud agent
2BuildIssues with agent label, no PRCreate worktree, spawn local agent
3ReviewPRs without review labelAdd trigger label (no agent)
4FixReviewed PRs without fix label, not stuckSpawn fix agent
5CleanupWorktrees for merged/closed PRsRemove worktree, prune branches

Agent Execution Modes

Different stages use different execution modes based on what the agent needs to do:

StageModeWhy
ResearchCloud (remote)Read-only analysis, offload compute
BuildLocal (tmux session)Needs git, local testing, file system access
ReviewCI (GitHub Action)Triggered by label, no agent session needed
FixLocal (tmux session)Code modifications in existing worktree
E2E TestLocal (tmux session)Browser automation, screenshot capture

Session Management

Every agent runs in a named tmux session with a consistent naming convention. This enables monitoring, takeover, and cleanup.

Naming convention:
  <tool>-<task>-<number>

Examples:
  cc-issue-1055        # Claude Code building issue 1055
  cc-fix-1060          # Claude Code fixing PR 1060
  cc-e2e-1055          # Claude Code running E2E tests
  cc-research-1055     # Claude Code researching issue 1055

Session lifecycle

  1. Create: dispatch spawns a named tmux session in the correct worktree
  2. Monitor: periodic checks capture output, detect prompts, identify completion
  3. Complete: cleanup cron detects completion patterns, updates labels, kills session
  4. Cleanup: worktree removed after PR is merged or closed

Monitoring

Agent prompt detection

A monitoring cron (every 15 minutes) scans all agent sessions for input prompts — cases where the agent is blocked waiting for a response ([Y/n], Choose:, etc.).

  • Detected prompts are routed to notification channels (Slack, WhatsApp, Discord)
  • Auto-proceed rule: if the agent has a recommended option and no human responds within ~10 minutes, the agent proceeds with the recommended choice

Zombie detection

A session is classified as zombie if:

  • Running >2 hours with no output change
  • Shows errors indicating wrong directory
  • Empty output buffer after >1 hour
  • Session exists but the process inside has exited

Zombies are killed and the pipeline stage is retried automatically.

Stale fix detection

A dedicated cron detects when new review comments appear after the comments-fixed label was applied. When this happens, the label is removed so the PR gets re-fixed in the next dispatch cycle.

Cron Schedule

The pipeline runs on a set of coordinated cron jobs:

JobSchedulePurpose
Main dispatchEvery 15 minResearch + build + review + fix
Fix dispatchEvery 30 minFix agents for reviewed PRs
E2E dispatchEvery 30 minE2E test agents
Prompt monitorEvery 15 minDetect agent input prompts
Stale fix checkEvery 2 hoursDetect new comments after fix
Session cleanup4x dailyClean completed sessions
Label reconciliationDailySync labels issue ↔ PR
Pipeline health5x dailyHealth report + anomaly detection

Manual Control

When automation isn't enough, manual controls let humans intervene at any point:

CommandEffect
disablePause all pipeline processing
enableResume processing
advance <issue> <stage>Skip current stage, move to next
retry <issue>Retry the last failed stage
kill <issue>Abort the pipeline, kill agents
reset <issue>Clear all state, start from scratch
inspect <issue>Show detailed pipeline state

Idempotency

Every dispatch cycle is idempotent. Running it twice produces the same result. This is critical for reliability — if a cron fires late or a cycle is interrupted, the next one picks up where it left off.

StageSkip If
ResearchIssue already has classification label
BuildPR already exists for the issue
ReviewReview-complete label present
FixMax attempts reached or agent-stuck
E2ETest result label present