sessionwiki

Session: golden-dataset-curation — 2026-09-08

eval-harness · Gemini · branch data/golden-set · gemini-2.5-pro · 2026-09-08 12:10 · 1 msgs · 2 tools · 1 min read · updated 8 weeks ago

tools: Bash, Write
← eval-harness Download .md .txt .json
Tool calls
Bash Bash: 1 call (50.0%) 1 Write Write: 1 call (50.0%) 1
Token usage78.4K total
Input8.4K
Cache creation16.8K
Cache read50.4K
Output2.8K
Cache hit ratio67%· warming up

Project: eval-harness · Branch: data/golden-set · Mode: default · Model: gemini-2.5-pro

Summary

Curated a 300-case golden dataset from support tickets: de-duplicated near-identical questions with embeddings, stratified by product area, and stored expected answers with source citations.

Conversation

Turn 1 — User

Our eval set is 80% password-reset questions.

Turn 1 — Assistant

Cluster ticket questions by embedding similarity, sample per product area, and keep the citation for each expected answer so faithfulness can be checked.

Tools used in this turn:

  • Write: scripts/curate_golden.py
  • Bash: python scripts/curate_golden.py --target 300 — suites/golden.yaml

Connections

  • [[eval-harness]]
  • [[Datasets]]
  • [[Embeddings]]

Keyboard shortcuts

⌘K / Ctrl+KOpen command palette
/Focus search
g hGo to home
g pGo to projects
g sGo to sessions
j / kNext / prev row (tables)
?Show this help
EscClose dialogs

Structured queries

Mix key:value filters with free text in the palette:

type:sessionOnly session pages
project:llm-wikiFilter by project name (substring)
model:claudeFilter by model name (substring)
date:>2026-03-01Sessions after a date
date:<2026-04-01Sessions before a date
tags:rustPages mentioning a tag/topic
sort:dateSort results by date (newest first)

Example: type:session project:llm-wiki date:>2026-04 sort:date