{
  "id": "eval-harness/run-evals-in-ci-on-prompt-changes",
  "slug": "run-evals-in-ci-on-prompt-changes",
  "title": "Session: run-evals-in-ci-on-prompt-changes — 2026-11-04",
  "type": "source",
  "project": "eval-harness",
  "date": "2026-11-04",
  "started": "2026-11-04T10:40:00+00:00",
  "ended": "2026-11-04T11:58:00+00:00",
  "model": "claude-sonnet-4-6",
  "cwd": "/home/dev/code/eval-harness",
  "git_branch": "chore/ci",
  "permission_mode": "default",
  "user_messages": 1,
  "tool_calls": 3,
  "tools_used": [
    "Bash",
    "Edit",
    "Write"
  ],
  "is_subagent": false,
  "wikilinks_out": [
    "GitHubActions",
    "QualityGates",
    "eval-harness"
  ],
  "body_text": "Project: eval-harness · Branch: chore/ci · Mode: default · Model: claude-sonnet-4-6\n\nSummary\n\nWired the harness into CI: a workflow runs the smoke suite on every PR that touches prompts/, posts the score deltas as a PR comment, and fails if faithfulness drops more than two points.\n\nConversation\n\nTurn 1 — User\n\nRun evals automatically when someone changes a prompt.\n\nTurn 1 — Assistant\n\nPath-filtered workflow on prompts/, smoke suite (200 cases), baseline from the latest main run, comment with the delta table, and a threshold gate on faithfulness.\n\nTools used in this turn:\n\n- Write: .github/workflows/evals.yml\n- Edit: harness/cli.py — --fail-on-drop\n- Bash: act pull_request -W .github/workflows/evals.yml — ok\n\nConnections\n\n- eval-harness\n- GitHubActions\n- QualityGates",
  "sha256": "d355685ae17b58d6",
  "source_url": "sessions/eval-harness/2026-11-04-run-evals-in-ci-on-prompt-changes.html"
}