Provider Abstraction

t-247·WorkTask·
·
·
Created2 months ago·Updated2 months ago

Description

Edit

Create Omni/Agent/Provider.hs module to abstract LLM backends.

Current State

  • Omni/Agent/Engine.hs contains LLM type with hardcoded OpenRouter logic (lines 267-301)
  • chat and chatWithUsage functions make HTTP calls to OpenRouter API (lines 560-594)
  • Worker.hs reads OPENROUTER_API_KEY from environment (line 243)

Requirements

1. Create Provider.hs with unified interface:

data Provider
  = OpenRouter { orApiKey :: Text, orModel :: Text }
  | Ollama { olBaseUrl :: Text, olModel :: Text }  
  | AmpCLI { ampPromptFile :: FilePath }

chat :: Provider -> [Tool] -> [Message] -> IO (Either Text ChatResult)

2. Extract OpenRouter implementation:

  • Move HTTP logic from Engine.hs chatWithUsage to Provider/OpenRouter.hs
  • Keep existing headers (HTTP-Referer, X-Title)
  • Parse cost from OpenRouter response usage.cost field

3. Implement Ollama provider:

  • Call local Ollama API at http://localhost:11434/api/chat
  • Use same message format (OpenAI-compatible)
  • Handle tool calls if model supports them
  • No API key required

4. Implement Amp subprocess provider:

  • Write prompt to temp file
  • Spawn 'amp --prompt-file <path>' subprocess
  • Capture stdout as response
  • Parse any tool outputs from Amp's format

5. Update Engine.hs:

  • Replace LLM type usage with Provider
  • EngineConfig.engineLLM becomes EngineConfig.engineProvider
  • runAgent uses Provider.chat instead of internal chat

6. Update Jr.hs CLI:

  • Add --engine flag to 'jr work' command: native|ollama|amp (default: native)
  • Pass selected provider to Worker.start

7. Environment variables:

  • OPENROUTER_API_KEY for OpenRouter provider
  • OLLAMA_HOST for custom Ollama URL (optional, default localhost:11434)
  • OLLAMA_MODEL for default model (optional)

Files to modify:

  • Omni/Agent/Engine.hs - refactor to use Provider
  • Omni/Agent/Worker.hs - accept Provider parameter
  • Omni/Jr.hs - add --engine CLI flag

Files to create:

  • Omni/Agent/Provider.hs - main module with Provider type
  • Omni/Agent/Provider/OpenRouter.hs - extracted OpenRouter logic
  • Omni/Agent/Provider/Ollama.hs - new Ollama implementation
  • Omni/Agent/Provider/Amp.hs - new Amp subprocess implementation

Testing:

  • Run: bild --test Omni/Agent/Provider.hs
  • Test OpenRouter with existing API key
  • Test Ollama if local instance available
  • Amp can be stubbed for now

Timeline (2)

🔄[human]Open → InProgress2 months ago
🔄[human]InProgress → Done2 months ago