Pipeline observability: align web/digest with scheduler run state

t-662·WorkTask·
·
·
·Omni/Pipeline/Web.hs
Created2 months ago·Updated2 months ago·pipeline runs →

Dependencies

Description

Edit

Problem

Observability surfaces are still wired to legacy pipeline_runs data, but the scheduler now tracks active runs in scheduler-state.json and agentd logs.

Current mismatch:

  • /pipeline web API/dashboard reads pipeline_runs from state.db
  • pipeline digest email also reads pipeline_runs
  • scheduler run loop does not insert/update pipeline_runs

Observed symptom:

  • dashboard reports stale active run(s) from historical records
  • digest metrics remain tied to old data instead of current scheduler activity

Impact

  • Operators receive incorrect health signals
  • "active runs" can be ghost entries
  • daily digest can report misleading shipped/failure counts

Proposed fix

  • Define a single source of truth for scheduler run state
  • Either:

1) write scheduler lifecycle events into pipeline_runs again, or 2) migrate web/digest to scheduler-native state + agentd run history

  • Add cleanup/migration for stale legacy running rows

Acceptance criteria

  • Dashboard/digest reflect current scheduler activity
  • No stale/ghost active runs after migration

Timeline (1)

🔄[human]Open → Draft2 months ago