MVP 0: Baseline metrics for task pipeline

t-680·WorkTask·
·
·
Created1 month ago·Updated1 month ago·pipeline runs →

Description

Edit

Parent epic: t-679

Goal

Establish baseline measurements of the current manual up-spec → coder pipeline before adding automation.

What to measure

1. Task success rate: % of tasks that pass on first coder attempt (no manual revision needed) 2. Revision count: how many times a task needs to be revised/re-filed before the coder succeeds 3. Time-to-done: wall clock from task creation to Done status 4. Failure mode classification: categorize recent failures (underspecified, wrong file refs, missing context, coder bug, etc.)

Implementation

  • Add lightweight tracking fields to task metadata: first_attempt_success: bool, revision_count: int, failure_reason: string
  • Backfill data from recent tasks (last ~50) by reviewing comments and patchsets
  • Create a simple dashboard or script that aggregates these metrics
  • Store in a format that can be compared against future MVP measurements

Acceptance criteria

  • Can run a command/script that outputs current baseline metrics
  • At least 30 tasks have been categorized
  • Failure modes are documented with relative frequency

Timeline (0)

No activity yet.