Before closing the epic, audit the testing situation for the new Engine:
Testing Checklist
Unit Tests (Engine.hs)
- [ ] JSON roundtrip for Message, ToolCall, FunctionCall
- [ ] Tool schema validation
- [ ] LLM config defaults
- [ ] Error handling for malformed responses
Unit Tests (Tools.hs)
- [ ] Each tool has valid JSON schema
- [ ] readFileTool handles missing files
- [ ] writeFileTool creates parent directories
- [ ] editFileTool handles no-match case
- [ ] runBashTool captures exit codes
- [ ] searchCodebaseTool returns structured results
Integration Tests
- [ ] Engine can complete a trivial task (e.g., create a file)
- [ ] Tool execution works end-to-end
- [ ] Cost/token callbacks fire correctly
- [ ] Activity callbacks fire at right times
- [ ] Max iterations limit is respected
Comparison Tests (Optional)
- [ ] Same task via amp vs Engine produces similar quality
- [ ] Cost per task is comparable or better
Manual Testing
- [ ] jr work t-xxx --use-engine completes successfully
- [ ] Activity logging appears in web UI
- [ ] Cost metrics recorded correctly
- [ ] Retry handling works with Engine
Run all tests
bild --test Omni/Agent/Engine.hs
bild --test Omni/Agent/Tools.hs
Success Criteria
- All unit tests pass
- At least one integration test proving end-to-end flow
- Manual verification of a real task completion