agent-harness/PARALLEL-AGENTS.md

# Parallel Agents — Running Multiple Agents Simultaneously

> The agent harness defaults to sequential work: one task, one iteration, one commit.
> But many projects have independent modules that can be built in parallel.
> This guide teaches you when to parallelize, how to coordinate, and how to merge results.

---

## When to Parallelize

### Good Candidates for Parallel Work

✅ **Independent packages in a monorepo**
```
packages/
├── auth/         ← Agent A
├── payment/      ← Agent B
├── notification/ ← Agent C
└── shared/       ← Build first, then parallelize
```

Each package has its own spec, plan, and tests. No shared mutable state.

✅ **Separate features with no shared code**
```
# Agent A
- [ ] Import QFX files
- [ ] Parse QFX format
- [ ] Test QFX parser

# Agent B  
- [ ] Import CSV files
- [ ] Parse CSV with column mapping
- [ ] Test CSV parser
```

Both implement file import, but use different parsers. No conflicts.

✅ **Documentation and code in parallel**
```
# Agent A: Builds features
- [ ] Implement transaction API

# Agent B: Writes docs
- [ ] Document transaction API endpoints
- [ ] Add usage examples
- [ ] Write integration guide
```

Code and docs touch different files.

✅ **Testing different components**
```
# Agent A: Unit tests
- [ ] Test parser functions

# Agent B: Integration tests  
- [ ] Test API endpoints
```

Different test suites, no overlap.

---

### Bad Candidates for Parallel Work

❌ **Dependent tasks**
```
# These MUST be sequential
- [ ] Create database schema     ← Must finish first
- [ ] Implement data access layer ← Depends on schema
- [ ] Build REST API             ← Depends on data layer
```

Can't parallelize a dependency chain.

❌ **Shared mutable files**
```
# Both agents editing package.json simultaneously
Agent A: Adding dependency "express"
Agent B: Adding dependency "commander"
→ Merge conflict in package.json
```

Single shared configuration file = bottleneck.

❌ **Overlapping code areas**
```
# Both working on the same module
Agent A: Refactoring src/parser.ts
Agent B: Adding feature to src/parser.ts
→ Guaranteed conflict
```

---

## Parallelization Strategies

### Strategy 1: Independent Sub-Projects

**Pattern:** Divide your project into complete, independent sub-projects. Each gets its own agent.

**Example: Microservices**
```
services/
├── auth-service/
│   ├── PROJECT-SPEC.md
│   ├── IMPLEMENTATION_PLAN.md
│   └── src/
├── api-gateway/
│   ├── PROJECT-SPEC.md
│   ├── IMPLEMENTATION_PLAN.md
│   └── src/
└── notification-service/
    ├── PROJECT-SPEC.md
    ├── IMPLEMENTATION_PLAN.md
    └── src/
```

**Orchestration:**
```bash
# Spawn three agents, each with their own directory
sessions_spawn "Read AGENT.md in /path/to/auth-service. Build it." &
sessions_spawn "Read AGENT.md in /path/to/api-gateway. Build it." &
sessions_spawn "Read AGENT.md in /path/to/notification-service. Build it." &
wait
```

**Coordination:** None needed — they don't interact.

**Merge:** Each commits to its own directory. No conflicts.

---

### Strategy 2: Feature-Parallel

**Pattern:** Same codebase, different features. Split the plan into independent feature sets.

**Example: CLI Tool with Multiple Commands**
```
# Agent A: PLAN-A.md
- [ ] Implement `auth` command
- [ ] Test `auth` command

# Agent B: PLAN-B.md
- [ ] Implement `list` command
- [ ] Test `list` command

# Agent C: PLAN-C.md
- [ ] Implement `upload` command
- [ ] Test `upload` command
```

**Orchestration:**
```
# Each agent reads the same AGENT.md but different plans
sessions_spawn "Read AGENT.md, use PLAN-A.md as implementation plan"
sessions_spawn "Read AGENT.md, use PLAN-B.md as implementation plan"
sessions_spawn "Read AGENT.md, use PLAN-C.md as implementation plan"
```

**Coordination:** Each feature is isolated in its own source file:
```
src/
├── commands/
│   ├── auth.ts    ← Agent A
│   ├── list.ts    ← Agent B
│   └── upload.ts  ← Agent C
├── cli.ts         ← Shared, agents don't touch
```

**Merge:** Standard git merge. Conflicts unlikely if directories are separate.

---

### Strategy 3: Test-Parallel

**Pattern:** One agent builds, others write tests.

**Example:**
```
# Agent A (builder): Implements features sequentially
- [ ] Implement parser
- [ ] Implement checker
- [ ] Implement reporter

# Agent B (tester): Writes tests as features complete
- [ ] Write parser tests (wait for Agent A to commit parser)
- [ ] Write checker tests (wait for Agent A to commit checker)
- [ ] Write reporter tests (wait for Agent A to commit reporter)
```

**Orchestration:**
```python
# Pseudo-code for coordinated spawning
agent_a = spawn_agent(plan="PLAN-BUILD.md")
wait_for_commit(agent_a, pattern="feat: implement parser")

agent_b = spawn_agent(plan="PLAN-TEST.md", task="Write parser tests")
# Agent B reads the code Agent A wrote, writes tests
```

**Coordination:** Builder commits first, tester follows immediately.

**Merge:** Sequential commits, no conflicts.

---

### Strategy 4: Layer-Parallel

**Pattern:** Assign agents to different architectural layers.

**Example:**
```
# Agent A: Data layer
- [ ] Database schema
- [ ] Migration scripts
- [ ] Data access functions

# Agent B: Business logic layer
- [ ] (Waits for Agent A to finish schema)
- [ ] Business logic using data layer
- [ ] Validation and rules

# Agent C: API layer
- [ ] (Waits for Agent B to finish business logic)
- [ ] REST endpoints
- [ ] Request/response handling
```

**This is mostly sequential with brief parallel windows.**

**Orchestration:**
```bash
# Phase 1: Only Agent A
agent_a_session=$(sessions_spawn "Build data layer")
wait_until_done $agent_a_session

# Phase 2: Agent B (uses Agent A's output)
agent_b_session=$(sessions_spawn "Build business logic")
wait_until_done $agent_b_session

# Phase 3: Agent C (uses Agent B's output)
agent_c_session=$(sessions_spawn "Build API layer")
```

**Coordination:** Explicit phases with dependencies.

**Merge:** Sequential, each agent builds on the previous layer.

---

## OpenClaw Patterns for Parallel Work

### Pattern A: Manual Spawn with Tracking

```markdown
You: "Spawn three agents to work on auth, payment, and notifications packages."

Cleo: *spawns 3 sub-agents*
  - Agent alpha-1 → auth package
  - Agent alpha-2 → payment package  
  - Agent alpha-3 → notifications package

You: "What's the status of those agents?"

Cleo: *checks sessions_list*
  - alpha-1: Completed 3/5 tasks (auth)
  - alpha-2: Completed 4/5 tasks (payment)
  - alpha-3: Stuck on task 2 (notifications) — reported STUCK

You: "What's alpha-3 stuck on?"

Cleo: *reads session transcript*
  "Can't connect to notification service API. Missing NOTIFICATION_KEY in .env"

You: "Add NOTIFICATION_KEY to .env, then resume alpha-3."

Cleo: *updates .env* → *spawns new agent for notifications* → completes
```

**Pros:** Full visibility and control  
**Cons:** You're the orchestrator (manual coordination)

---

### Pattern B: Cron-Based Parallel Iterations

```json
{
  "jobs": [
    {
      "name": "agent-auth-iteration",
      "schedule": { "kind": "every", "everyMs": 900000 },
      "payload": {
        "kind": "agentTurn",
        "message": "Read AGENT.md in /path/to/auth/. Follow the loop.",
        "model": "sonnet"
      }
    },
    {
      "name": "agent-payment-iteration",
      "schedule": { "kind": "every", "everyMs": 900000 },
      "payload": {
        "kind": "agentTurn",
        "message": "Read AGENT.md in /path/to/payment/. Follow the loop.",
        "model": "sonnet"
      }
    },
    {
      "name": "agent-notification-iteration",
      "schedule": { "kind": "every", "everyMs": 900000 },
      "payload": {
        "kind": "agentTurn",
        "message": "Read AGENT.md in /path/to/notification/. Follow the loop.",
        "model": "sonnet"
      }
    }
  ]
}
```

**Effect:** Three agents iterate every 15 minutes, independently, in parallel.

**Pros:** Fully autonomous  
**Cons:** Merge conflicts require human intervention

---

### Pattern C: Coordinated Phases with Gates

```markdown
You: "Build the project in three phases. Phase 1: data layer. 
      Phase 2: business logic (3 agents in parallel). 
      Phase 3: API layer."

Cleo orchestrates:
  Phase 1: Sequential agent for data layer
    → Waits for DONE signal
  Phase 2: Spawns 3 agents for business logic modules
    → Waits for all 3 to finish
  Phase 3: Sequential agent for API layer
    → Waits for DONE signal

Cleo: "All phases complete. Review?"
```

**Pros:** Maximizes parallelism while respecting dependencies  
**Cons:** Requires sophisticated orchestration logic

---

## Conflict Resolution

### Preventing Conflicts

**1. Namespace isolation**
```
# Bad (shared file)
src/utils.ts  ← Multiple agents editing this = conflict

# Good (separate namespaces)
src/auth/utils.ts         ← Agent A
src/payment/utils.ts      ← Agent B
src/notification/utils.ts ← Agent C
```

**2. Explicit ownership in the plan**
```markdown
# PLAN-A.md (Agent A owns these)
- [ ] src/commands/auth.ts
- [ ] tests/auth.test.ts

# PLAN-B.md (Agent B owns these)
- [ ] src/commands/list.ts
- [ ] tests/list.test.ts
```

**3. Shared files = sequential phases**
```markdown
# Phase 1: One agent sets up shared infrastructure
- [ ] package.json with all dependencies
- [ ] tsconfig.json
- [ ] .eslintrc

# Phase 2: Multiple agents build features (don't touch shared files)
```

---

### Handling Conflicts When They Happen

**Detection:**
```bash
# Agent A commits first
git log --oneline
# a1b2c3d feat: implement auth command

# Agent B tries to commit
git pull
# CONFLICT (content): Merge conflict in src/cli.ts
```

**Resolution strategies:**

**Strategy 1: Last-In Rebases**
```bash
# Agent B's iteration pauses, human resolves conflict
git checkout agent-b-branch
git rebase main
# Resolve conflicts in src/cli.ts
git add src/cli.ts
git rebase --continue
git push --force

# Resume Agent B
```

**Strategy 2: Accept Both Changes**
```bash
# Both agents added a command to src/cli.ts
# Conflict:
<<<<<<< HEAD
program.command('auth')
=======
program.command('list')  
>>>>>>> agent-b-branch

# Resolution: Both are valid, keep both
program.command('auth')
program.command('list')
```

**Strategy 3: Abort and Retry Sequentially**
```bash
# If conflict is complex, give up on parallelism
git merge --abort

# Run Agent B AFTER Agent A finishes
# Sequential is slower but conflict-free
```

---

## Merging Results

### Sequential Integration

**Pattern:** Each agent commits to its branch, you merge one at a time.

```bash
# Agent A finishes (auth feature)
git checkout -b agent-a
# ... Agent A's commits ...
git checkout main
git merge agent-a  # ✅ Clean merge

# Agent B finishes (payment feature)
git checkout -b agent-b
# ... Agent B's commits ...
git checkout main
git merge agent-b  # ✅ Clean merge (no overlap with agent-a)

# Agent C finishes (notification feature)
git checkout -b agent-c
# ... Agent C's commits ...
git checkout main
git merge agent-c  # ✅ Clean merge
```

**When it works:** Features are truly independent.

---

### Simultaneous Integration with Review

**Pattern:** All agents finish, then you review and integrate.

```bash
# All three agents worked overnight
# Morning review:

git log --oneline agent-a
# 5 commits, auth feature complete

git log --oneline agent-b
# 7 commits, payment feature complete

git log --oneline agent-c
# 4 commits, notification feature complete

# Integration strategy:
git checkout main
git merge agent-a  # Merge first
npm test           # ✅ All tests pass

git merge agent-b  # Merge second
npm test           # ✅ All tests pass

git merge agent-c  # Merge third
npm test           # ❌ Test failure! Notification imports auth, but different version?

# Debug:
git show agent-c:src/notification/handler.ts
# Agent C imported from 'auth' v1, but Agent A built v2

# Fix: Update Agent C's branch
git checkout agent-c
# ... update imports ...
git checkout main
git merge agent-c  # ✅ Now works
```

**Lesson:** Integration testing catches issues that unit tests miss.

---

### Squash and Clean

**Pattern:** Parallel work was messy (trial and error), squash before merging.

```bash
# Agent A made 12 commits with several reverts
git log --oneline agent-a
# 12 commits, but really just 1 feature

# Squash into one clean commit
git checkout agent-a
git rebase -i main
# Mark all but first commit as "squash"
# Edit commit message: "feat: implement auth command"

git checkout main
git merge agent-a --ff-only
# Now main has one clean commit instead of 12 messy ones
```

**When to use:** Agent made progress but with messy intermediate states.

---

## Monitoring Parallel Agents

### Status Dashboard (Manual)

```bash
# Check all branches for progress
for branch in agent-a agent-b agent-c; do
  echo "=== $branch ==="
  git checkout $branch
  echo "Commits: $(git log --oneline main..$branch | wc -l)"
  echo "Last: $(git log -1 --oneline)"
  echo "Tests: $(npm test 2>&1 | grep -E 'passing|failing')"
done
```

---

### OpenClaw Session Monitoring

```markdown
You: "List all active agent sessions"
Cleo: *calls sessions_list*
  Active sessions:
  - agent-auth (alpha-1): Running for 5 minutes
  - agent-payment (alpha-2): Running for 3 minutes
  - agent-notification (alpha-3): Completed

You: "What did agent-notification accomplish?"
Cleo: *reads session history for alpha-3*
  Completed tasks:
  - Set up notification package structure
  - Implemented email notification handler
  - Wrote tests for email sender
  
  Summary: 3 commits, all tests passing
```

---

### Git Log Summary

```bash
# See all parallel work at once
git log --all --graph --oneline

# Output:
# *  a1b2c3d (agent-a) feat: add auth command
# | * d4e5f6g (agent-b) feat: add payment processing
# |/
# | * h8i9j0k (agent-c) feat: add notification handler
# |/
# * l0m1n2o (main) chore: project setup
```

Visual representation of parallel branches.

---

## Cost Considerations

### Request-Based Billing (Copilot)

**Parallel agents = More requests**
```
# Sequential: 10 iterations = 10 requests
One agent does 10 tasks sequentially

# Parallel: 10 iterations = 30 requests
Three agents do 10 tasks in parallel (each agent = separate request)
```

**BUT: Wall-clock time is 1/3**
- Sequential: 10 iterations × 2 min = 20 minutes
- Parallel: 10 iterations ÷ 3 agents = 7 minutes

**Tradeoff:** Pay more (3x requests), finish faster (1/3 time).

---

### Token-Based Billing (Anthropic)

**Parallel agents = Better token efficiency**

Each agent starts with fresh context (~2K tokens). If you ran one long session:
- Turn 30 = 90K input tokens (context growth)

Three parallel agents with fresh context:
- Agent A: 10 turns × 5K avg = 50K tokens
- Agent B: 10 turns × 5K avg = 50K tokens
- Agent C: 10 turns × 5K avg = 50K tokens
- Total: 150K tokens

vs. one long session:
- 30 turns with context growth = 300K+ tokens

**Parallel is cheaper on token-based systems.**

---

## Anti-Patterns

### 1. Parallel-by-Default
```
# Bad: Everything in parallel!
spawn("Do task 1") & spawn("Do task 2") & spawn("Do task 3")
# → Tasks 2 and 3 depend on task 1
# → Failures, conflicts, wasted work
```

**Fix:** Default to sequential. Parallelize only when independence is certain.

---

### 2. Ignoring Dependencies
```
# Bad: Parallel agents building layered architecture
Agent A: Building API (needs business logic)
Agent B: Building business logic (needs data layer)
Agent C: Building data layer

# All start at once → A and B fail (missing dependencies)
```

**Fix:** Build dependencies first, then parallelize.

---

### 3. No Integration Testing
```
# Each agent's tests pass
Agent A tests: ✅
Agent B tests: ✅
Agent C tests: ✅

# But together they fail
Integration: ❌ (A calls B with wrong interface)
```

**Fix:** Always run integration tests after merging parallel work.

---

### 4. Over-Parallelization
```
# 10 agents working on 10 tiny tasks
# More time coordinating than building
```

**Fix:** Parallelize only when tasks are substantial (30+ minutes each).

---

## Decision Framework

Use this to decide: sequential or parallel?

```
Is the work truly independent? 
  ├─ NO → Sequential (don't even try)
  └─ YES → Continue
  
Will conflicts be easy to resolve?
  ├─ NO → Sequential (not worth the pain)
  └─ YES → Continue

Is each task substantial (30+ min)?
  ├─ NO → Sequential (overhead not worth it)
  └─ YES → Continue

Do you have time to monitor and merge?
  ├─ NO → Sequential (safer unattended)
  └─ YES → Parallelize!
```

---

_Parallelism is a force multiplier, but only if wielded carefully. Default to sequential. Parallelize when it's obviously safe._