Integrate Prompt IR into Ava with observability

Integrate Prompt IR + Adaptive Context into Ava

Overview

The Prompt IR infrastructure is complete (t-477). This task is to wire it into Ava's Telegram handler and add observability to validate it works in practice.

Background

What was built (t-477)

| Module | Purpose | |--------|---------| | Omni/Agent/Prompt/IR.hs | Core types: Section, ToolDef, ContextRequest, CompositionMode, Priority | | Omni/Agent/Prompt/Hydrate.hs | ContextRequest → PromptIR via context sources | | Omni/Agent/Prompt/Compile.hs | PromptIR → CompiledPrompt with budget enforcement | | Omni/Agent/Prompt/MemorySources.hs | Memory-backed context sources (temporal, semantic, knowledge) | | Omni/Agent/Op.hs | Infer takes ContextRequest, InferRaw for legacy | | Omni/Agent/Interpreter/Sequential.hs | Handles hydration + compilation |

The Pipeline

infer(model, ContextRequest)
    → hydrate (using MemorySources)
    → PromptIR (labeled sections + metadata)
    → compile (budget enforcement, priority ordering)
    → LLM call

Current State

Ava currently builds context manually in Telegram.hs:

• Lines ~1131-1157: Memory.getAdaptiveContext for conversation history
• Lines ~1225-1237: Memory.recallMemories for user memories
• Lines ~1325-1340: Manual string concatenation of all context sections

This works but lacks:

• Structured sections with metadata
• Priority-based budget enforcement
• Traceable provenance
• Observability into what context was used

Implementation Plan

Phase 1: Add HydrationConfig to Telegram.hs

1. Import MemorySources module 2. Build HydrationConfig using:

• memoryTemporalSource userId chatId
• memorySemanticSource
• memoryKnowledgeSource userId
• Static sections for time, project, user info

3. Pass to SeqConfig via seqHydrationConfig

Key function to modify: processEngagedMessage in Telegram.hs

Phase 2: Observability

Add logging/tracing to see what context is retrieved:

1. Log hydration results:

• Number of sections by source (temporal, semantic, knowledge, static)
• Token counts per section
• What was dropped due to budget

2. Add to PromptMeta:

• pmSectionsIncluded :: [Text] - section IDs that made it
• pmSectionsDropped :: [Text] - section IDs dropped for budget
• pmRetrievalLatencyMs :: Int - time spent in hydration

3. Emit trace events:

• Use existing Trace.EventCustom for hydration stats
• Include in Ava's audit log

Phase 3: Gradual Rollout

1. Feature flag: Add config option to enable/disable new context system 2. A/B comparison: Log both old and new context (without using new) to compare 3. Full switch: Once validated, remove old code path

Code Pointers

Where to wire up HydrationConfig

-- In Omni/Agent/Telegram.hs, around line 1380
-- Currently:
let seqConfig = (Seq.defaultSeqConfig provider seqToolsAllowed)
      { Seq.seqGetIteration = ...
      , Seq.seqOnEvent = ...
      }

-- Change to:
let hydrationCfg = MS.buildHydrationConfig
      systemPrompt
      tools  
      [MS.mkTimeSection now tz, MS.mkProjectSection proj dir]
      (Memory.unUserId uid)
      chatId
let seqConfig = (Seq.defaultSeqConfig provider seqToolsAllowed)
      { Seq.seqHydrationConfig = Just hydrationCfg
      , Seq.seqGetIteration = ...
      , Seq.seqOnEvent = ...
      }

Where observability goes

-- In Sequential.hs, after hydration (around line 145)
Free (Op.Infer model contextReq k) ->
  case seqHydrationConfig config of
    Just hydrationCfg -> do
      t0 <- Time.getCurrentTime
      promptIR <- Hydrate.hydrate hydrationCfg contextReq
      
      -- NEW: Log hydration stats
      let stats = summarizeHydration promptIR
      seqOnEvent config (Trace.EventCustom "hydration" stats t0)
      
      compiled <- Compile.compile promptIR
      ...

Success Criteria

1. Ava uses the new context system for all conversations 2. Logs show what context was retrieved and used 3. No regression in response quality (subjective, observed over ~1 week) 4. Budget enforcement works (can observe sections being dropped under pressure)

Non-Goals

• Synthetic evals (hard to make meaningful, real usage is the test)
• Changing the Memory.hs implementation (already works well)
• Optimizing retrieval performance (premature)

Related

• t-477: Prompt IR implementation (Done)
• t-345: Hybrid context strategy design (Done)
• Commits: 21f111e, 69ad87d, 32142b6, d472b64, 3dbf484, d844e27, eca7a0d, c81bf09, 6c4d923