{
  "id": "eval-harness/parallel-runner-with-retries",
  "slug": "parallel-runner-with-retries",
  "title": "Session: parallel-runner-with-retries — 2026-10-02",
  "type": "source",
  "project": "eval-harness",
  "date": "2026-10-02",
  "started": "2026-10-02T09:20:00+00:00",
  "ended": "2026-10-02T11:14:00+00:00",
  "model": "qwen3-coder",
  "cwd": "/home/dev/code/eval-harness",
  "git_branch": "feat/parallel-runner",
  "permission_mode": "default",
  "user_messages": 1,
  "tool_calls": 3,
  "tools_used": [
    "Bash",
    "Edit",
    "Write"
  ],
  "is_subagent": false,
  "wikilinks_out": [
    "Asyncio",
    "Backoff",
    "Checkpointing",
    "eval-harness"
  ],
  "body_text": "Project: eval-harness · Branch: feat/parallel-runner · Mode: default · Model: qwen3-coder\n\nSummary\n\nRewrote the runner to evaluate cases concurrently with asyncio and a bounded semaphore, exponential backoff with jitter on 429/5xx, and resumable runs from a JSONL checkpoint. A 2,000-case suite went from 46 to 7 minutes.\n\nConversation\n\nTurn 1 — User\n\nThe eval suite takes 46 minutes. Make it parallel and resumable.\n\nTurn 1 — Assistant\n\nAsync runner with a semaphore (default 16), retries with full jitter on 429 and 5xx only, and a JSONL checkpoint so a crashed run resumes from the last finished case.\n\nTools used in this turn:\n\n- Edit: harness/runner.py\n- Write: harness/checkpoint.py\n- Bash: python -m harness run suites/support.yaml — 2000 cases in 7m02s\n\nConnections\n\n- eval-harness\n- Asyncio\n- Backoff\n- Checkpointing",
  "sha256": "80ca3ab992e4e5a1",
  "source_url": "sessions/eval-harness/2026-10-02-parallel-runner-with-retries.html"
}