Implement empirical prompt compression

t-395·WorkTask·
·
·
·Omni/Agent/Prompt/Compile.hs
Created3 months ago·Updated1 week ago·pipeline runs →

Description

Edit

Add a Compress operation to the free monad that finds shorter prompts preserving output behavior.

Approach

Iterative compression: shorten prompt, verify outputs are 'close enough', repeat. Not rate-distortion optimal, but practically useful for cost reduction.

Prior Art

  • 'Fundamental Limits of Prompt Compression' (NeurIPS 2024) - theoretical framework
  • LLMLingua, AutoCompressor - existing compression methods

Implementation

1. Add Compress constructor to OpF with distortion bound parameter 2. Implement compression strategies:

  • Remove redundant instructions
  • Summarize examples (few-shot -> distilled)
  • Identify sufficient statistics (what info actually matters)

3. Verification: run both prompts, compare outputs

Metrics & Verification

  • Compression ratio (tokens saved)
  • Output fidelity (embedding similarity, exact match on structured outputs)
  • Cost savings in practice

Test Cases

  • Verbose system prompts -> minimal equivalent
  • Many-shot examples -> fewer examples with same behavior
  • Long context -> compressed context

Notes

Need to define 'close enough' carefully. For tool-calling agents, maybe exact same tool sequence? For generation, embedding similarity?

Timeline (23)

🔄[system]Open → InProgress2 months ago
🔄[system]InProgress → Open2 months ago
💬[system]2 months ago

Pipeline: agent died without producing a commit, resetting

🔄[system]Open → InProgress2 months ago
💬[human]2 months ago

Pipeline scheduler: started run=pipeline-omni-agent-prompt-compile-hs-t-395-1771559886 domain=Omni/Agent/Prompt/Compile.hs

🔄[human]InProgress → Review2 months ago
💬[human]2 months ago

Pipeline scheduler: run=pipeline-omni-agent-prompt-compile-hs-t-395-1771559886 domain=Omni/Agent/Prompt/Compile.hs status=failed cost=0c error=OAuth login failed: OAuth login requires an interactive terminal. Please run this command in a terminal session, not in headless mode. (fund-spend=failed)

🔄[human]Review → Open2 months ago
💬[human]2 months ago

Reopened: prior Review transition came from pipeline auth failure, not successful task execution. Re-queued for normal processing.

🔄[system]Open → InProgress2 months ago
💬[human]2 months ago

Pipeline scheduler: started run=pipeline-omni-agent-prompt-compile-hs-t-395-1771560258 domain=Omni/Agent/Prompt/Compile.hs

🔄[human]InProgress → Review2 months ago
💬[human]2 months ago

Pipeline scheduler: run=pipeline-omni-agent-prompt-compile-hs-t-395-1771560258 domain=Omni/Agent/Prompt/Compile.hs status=done cost=49c (fund-spend=failed)

💬[human]1 week ago

Ava triage: pipeline auto-run reached status=done but the agent made NO git commits and reported blockers (missing files, path mismatches, or need clarification). This task is not actually in review — there's nothing to review. Resetting status to Open so it can be re-scoped.

🔄[human]Review → Open1 week ago
🔄[human]Open → Verified1 week ago