Research from Claude Code, OpenAI Codex, and Amp reveals common patterns for effective coding agents that Jr should adopt.
From anthropic.com/engineering/effective-harnesses-for-long-running-agents:
From anthropic.com/engineering/claude-code-best-practices:
1. Add progress tracking:
2. Enforce incremental workflow:
3. Explicit verification:
4. Avoid redundant testing:
I'll start by analyzing the codebase to understand the current Jr Worker implementation and then make the necessary changes to add progress tracking and incremental workflow.
Omni/Agent/Worker.hs1: {-# LANGUAGE OverloadedStrings #-}
2: {-# LANGUAGE NoImplicitPrelude #-}
3:
4: module Omni.Agent.Worker where
5:
6: import Alpha
7: import qualified Data.Aeson as Aeson
8: import qualified Data.Aeson.Key as AesonKey
9: import qualified Data.ByteString.Lazy as BSL
10: import Data.IORef (modifyIORef', newIORef, readIORef)
11: import qualified Data.List as List
12: import qualified Data.Text as Text
13: import qualified Data.Text.Encoding as TE
14: import qualified Data.Time
15: import qualified Omni.Agent.Core as Core
16: import qualified Omni.Agent.Engine as Engine
17: import qualified Omni.Agent.Log as AgentLog
18: import qualified Omni.Agent.Tools as Tools
19: import qualified Omni.Fact as Fact
20: import qualified Omni.Task.Core as TaskCore
21: import qualified System.Directory as Directory
22: import qualified System.Environment as Env
23: import qualified System.Exit as Exit
24: import System.FilePath ((</>))
25: import qualified System.Process as Process
26:
27: start :: Core.Worker -> Maybe Text -> IO ()
28: start worker maybeTaskId = do
29: if Core.workerQuiet worker
30: then putText ("[worker] Starting for " <> Core.workerName worker)
31: else do
32: AgentLog.init (Core.workerName worker)
33: AgentLog.log ("[worker] Starting for " <> Core.workerName worker)
34: case maybeTaskId of
35: Just tid -> logMsg worker ("[worker] Target task: " <> tid)
36: Nothing -> logMsg worker "[worker] No specific task, will pick from ready queue"
37: runOnce worker maybeTaskId
38:
39: -- | Log message respecting quiet mode
40: logMsg :: Core.Worker -> Text -> IO ()
41: logMsg worker msg =
42: if Core.workerQuiet worker
43: then putText msg
44: else AgentLog.log msg
45:
46: -- | Convert key-value pairs to JSON metadata string
47: toMetadata :: [(Text, Text)] -> Text
48: toMetadata pairs =
49: let obj = Aeson.object [(AesonKey.fromText k, Aeson.String v) | (k, v) <- pairs]
50: in TE.decodeUtf8 (BSL.toStrict (Aeson.encode obj))
51:
52: runOnce :: Core.Worker -> Maybe Text -> IO ()
53: runOnce worker maybeTaskId = do
54: -- Find work
55: targetTask <- case maybeTaskId of
56: Just tid -> do
57: TaskCore.findTask tid </ TaskCore.loadTasks
58: Nothing -> do
59: readyTasks <- TaskCore.getReadyTasks
60: case readyTasks of
61: [] -> pure Nothing
62: (task : _) -> pure (Just task)
63:
64: case targetTask of
65: Nothing -> do
66: case maybeTaskId of
67: Just tid -> do
68: unless (Core.workerQuiet worker) <| AgentLog.updateActivity ("Task " <> tid <> " not found.")
69: logMsg worker ("[worker] Task " <> tid <> " not found.")
70: Nothing -> do
71: unless (Core.workerQuiet worker) <| AgentLog.updateActivity "No work found."
72: logMsg worker "[worker] No ready tasks found."
73: Just task -> do
74: processTask worker task
75:
76: processTask :: Core.Worker -> TaskCore.Task -> IO ()
77: processTask worker task = do
78: let repo = Core.workerPath worker
79: let tid = TaskCore.taskId task
80: let quiet = Core.workerQuiet worker
81: let say = logMsg worker
82:
83: unless quiet <| AgentLog.update (\s -> s {AgentLog.statusTask = Just tid})
84: say ("[worker] Claiming task " <> tid)
85:
86: -- Claim task
87: TaskCore.logActivity tid TaskCore.Claiming Nothing
88: TaskCore.updateTaskStatus tid TaskCore.InProgress []
89: say "[worker] Status -> InProgress"
90:
91: -- Run agent with timing
92: startTime <- Data.Time.getCurrentTime
93: activityId <- TaskCore.logActivityWithMetrics tid TaskCore.Running Nothing Nothing (Just startTime) Nothing Nothing Nothing
94:
95: say "[worker] Starting engine..."
96: (exitCode, output, costCents) <- runWithEngine worker repo task
97:
98: endTime <- Data.Time.getCurrentTime
99: say ("[worker] Agent exited with: " <> tshow exitCode)
100:
101: -- Update the activity record with metrics (convert Double to Int by rounding)
102: TaskCore.updateActivityMetrics activityId Nothing (Just endTime) (Just (round costCents)) Nothing
103:
104: case exitCode of
105: Exit.ExitSuccess -> do
106: TaskCore.logActivity tid TaskCore.Reviewing Nothing
107: say "[worker] Running formatters..."
108: _ <- runFormatters repo
109:
110: -- Try to commit (this runs git hooks which may fail)
111: let commitMsg = formatCommitMessage task output
112: say "[worker] Attempting commit..."
113: commitResult <- tryCommit repo commitMsg
114:
115: case commitResult of
116: CommitFailed commitErr -> do
117: say ("[worker] Commit failed: " <> commitErr)
118:
119: -- Save failure context and reopen task for retry
120: maybeCtx <- TaskCore.getRetryContext tid
121: let attempt = maybe 1 (\c -> TaskCore.retryAttempt c + 1) maybeCtx
122:
123: if attempt > 3
124: then do
125: say "[worker] Task failed 3 times, needs human intervention"
126: TaskCore.logActivity tid TaskCore.Failed (Just (toMetadata [("reason", "max_retries_exceeded")]))
127: TaskCore.updateTaskStatus tid TaskCore.Open []
128: else do
129: let currentReason = "attempt " <> tshow attempt <> ": commit_failed: " <> commitErr
130: let accumulatedReason = case maybeCtx of
131: Nothing -> currentReason
132: Just ctx -> TaskCore.retryReason ctx <> "\n" <> currentReason
133: TaskCore.setRetryContext
134: TaskCore.RetryContext
135: { TaskCore.retryTaskId = tid,
136: TaskCore.retryOriginalCommit = "",
137: TaskCore.retryConflictFiles = [],
138: TaskCore.retryAttempt = attempt,
139: TaskCore.retryReason = accumulatedReason,
140: TaskCore.retryNotes = maybeCtx +> TaskCore.retryNotes
141: }
142: TaskCore.logActivity tid TaskCore.Retrying (Just (toMetadata [("attempt", tshow attempt)]))
143: TaskCore.updateTaskStatus tid TaskCore.Open []
144: say ("[worker] Task reopened (attempt " <> tshow attempt <> "/3)")
145: NoChanges -> do
146: -- No changes = task already implemented, mark as Done
147: say "[worker] No changes to commit - task already done"
148: TaskCore.clearRetryContext tid
149: TaskCore.logActivity tid TaskCore.Completed (Just (toMetadata [("result", "no_changes")]))
150: TaskCore.updateTaskStatus tid TaskCore.Done []
151: say ("[worker] ✓ Task " <> tid <> " -> Done (no changes)")
152: unless quiet <| AgentLog.update (\s -> s {AgentLog.statusTask = Nothing})
153: CommitSuccess -> do
154: -- Commit succeeded, set to Review
155: TaskCore.logActivity tid TaskCore.Completed (Just (toMetadata [("result", "committed")]))
156: TaskCore.updateTaskStatus tid TaskCore.Review []
157: say ("[worker] ✓ Task " <> tid <> " -> Review")
158: unless quiet <| AgentLog.update (\s -> s {AgentLog.statusTask = Nothing})
159: Exit.ExitFailure code -> do
160: say ("[worker] Engine failed with code " <> tshow code)
161: TaskCore.logActivity tid TaskCore.Failed (Just (toMetadata [("exit_code", tshow code)]))
162: -- Don't set back to Open here - leave in InProgress for debugging
163: say "[worker] Task left in InProgress (engine failure)"
164:
165: -- | Run lint --fix to format and fix lint issues
166: runFormatters :: FilePath -> IO (Either Text ())
167: runFormatters repo = do
168: let cmd = (Process.proc "lint" ["--fix"]) {Process.cwd = Just repo}
169: (code, _, _) <- Process.readCreateProcessWithExitCode cmd ""
170: case code of
171: Exit.ExitSuccess -> pure (Right ())
172: Exit.ExitFailure _ -> pure (Right ()) -- lint --fix may exit non-zero but still fix things
173:
174: data CommitResult = CommitSuccess | NoChanges | CommitFailed Text
175: deriving (Show, Eq)
176:
177: -- | Try to commit, returning result
178: tryCommit :: FilePath -> Text -> IO CommitResult
179: tryCommit repo msg = do
180: -- Stage all changes
181: let addCmd = (Process.proc "git" ["add", "."]) {Process.cwd = Just repo}
182: (addCode, _, addErr) <- Process.readCreateProcessWithExitCode addCmd ""
183: case addCode of
184: Exit.ExitFailure _ -> pure <| CommitFailed (Text.pack addErr)
185: Exit.ExitSuccess -> do
186: -- Check for changes
187: let checkCmd = (Process.proc "git" ["diff", "--cached", "--quiet"]) {Process.cwd = Just repo}
188: (checkCode, _, _) <- Process.readCreateProcessWithExitCode checkCmd ""
189: case checkCode of
190: Exit.ExitSuccess -> pure NoChanges
191: Exit.ExitFailure 1 -> do
192: -- There are changes, commit them
193: let commitCmd = (Process.proc "git" ["commit", "-m", Text.unpack msg]) {Process.cwd = Just repo}
194: (commitCode, _, commitErr) <- Process.readCreateProcessWithExitCode commitCmd ""
195: case commitCode of
196: Exit.ExitSuccess -> pure CommitSuccess
197: Exit.ExitFailure _ -> pure <| CommitFailed (Text.pack commitErr)
198: Exit.ExitFailure c -> pure <| CommitFailed ("git diff failed with code " <> tshow c)
199:
200: -- | Run task using native Engine
201: -- Returns (ExitCode, output text, cost in cents)
202: runWithEngine :: Core.Worker -> FilePath -> TaskCore.Task -> IO (Exit.ExitCode, Text, Double)
203: runWithEngine worker repo task = do
204: -- Read API key from environment
205: maybeApiKey <- Env.lookupEnv "OPENROUTER_API_KEY"
206: case maybeApiKey of
207: Nothing -> pure (Exit.ExitFailure 1, "OPENROUTER_API_KEY not set", 0)
208: Just apiKey -> do
209: -- Check for retry context
210: maybeRetry <- TaskCore.getRetryContext (TaskCore.taskId task)
211:
212: -- Build the full prompt
213: let ns = fromMaybe "." (TaskCore.taskNamespace task)
214: let basePrompt = buildBasePrompt task ns repo
215:
216: -- Add retry context if present
217: let retryPrompt = buildRetryPrompt maybeRetry
218:
219: let prompt = basePrompt <> retryPrompt
220:
221: -- Read AGENTS.md
222: agentsMd <-
223: fmap (fromMaybe "") <| do
224: exists <- Directory.doesFileExist (repo </> "AGENTS.md")
225: if exists
226: then Just </ readFile (repo </> "AGENTS.md")
227: else pure Nothing
228:
229: -- Get relevant facts from the knowledge base
230: relevantFacts <- getRelevantFacts task
231: let factsSection = formatFacts relevantFacts
232:
233: -- Build system prompt
234: let systemPrompt =
235: prompt
236: <> "\n\nREPOSITORY GUIDELINES (AGENTS.md):\n"
237: <> agentsMd
238: <> factsSection
239:
240: -- Build user prompt from task comments
241: let userPrompt = formatTask task
242:
243: -- Select model based on task complexity (simple heuristic)
244: let model = selectModel task
245:
246: -- Generate session ID for event logging
247: sessionId <- TaskCore.generateSessionId
248: let tid = TaskCore.taskId task
249:
250: -- Helper to log events to DB
251: -- For text content, store as-is; for structured data, JSON-encode
252: let logEventText = TaskCore.insertAgentEvent tid sessionId
253: logEventJson eventType value = do
254: let contentJson = TE.decodeUtf8 (BSL.toStrict (Aeson.encode value))
255: TaskCore.insertAgentEvent tid sessionId eventType contentJson
256:
257: -- Build Engine config with callbacks
258: totalCostRef <- newIORef (0 :: Double)
259: let quiet = Core.workerQuiet worker
260: sayLog msg = if quiet then putText msg else AgentLog.log msg
261: engineCfg =
262: Engine.EngineConfig
263: { Engine.engineLLM =
264: Engine.defaultLLM
265: { Engine.llmApiKey = Text.pack apiKey
266: },
267: Engine.engineOnCost = \tokens cost -> do
268: modifyIORef' totalCostRef (+ cost)
269: sayLog <| "Cost: " <> tshow cost <> " cents (" <> tshow tokens <> " tokens)"
270: logEventJson "Cost" (Aeson.object [("tokens", Aeson.toJSON tokens), ("cents", Aeson.toJSON cost)]),
271: Engine.engineOnActivity = \activity -> do
272: sayLog <| "[engine] " <> activity,
273: Engine.engineOnToolCall = \toolName args -> do
274: sayLog <| "[tool] " <> toolName
275: logEventText "ToolCall" (toolName <> ": " <> args),
276: Engine.engineOnAssistant = \msg -> do
277: sayLog <| "[assistant] " <> Text.take 200 msg
278: logEventText "Assistant" msg,
279: Engine.engineOnToolResult = \toolName success output -> do
280: let statusStr = if success then "ok" else "failed"
281: sayLog <| "[result] " <> toolName <> " (" <> statusStr <> "): " <> Text.take 100 output
282: logEventText "ToolResult" output,
283: Engine.engineOnComplete = do
284: sayLog "[engine] Complete"
285: logEventText "Complete" "",
286: Engine.engineOnError = \err -> do
287: sayLog <| "[error] " <> err
288: logEventText "Error" err
289: }
290:
291: -- Build Agent config
292: let agentCfg =
293: Engine.AgentConfig
294: { Engine.agentModel = model,
295: Engine.agentTools = Tools.allTools,
296: Engine.agentSystemPrompt = systemPrompt,
297: Engine.agentMaxIterations = 100
298: }
299:
300: -- Run the agent
301: result <- Engine.runAgent engineCfg agentCfg userPrompt
302: totalCost <- readIORef totalCostRef
303:
304: case result of
305: Left err -> pure (Exit.ExitFailure 1, "Engine error: " <> err, totalCost)
306: Right agentResult -> do
307: let output = Engine.resultFinalMessage agentResult
308: pure (Exit.ExitSuccess, output, totalCost)
309:
310: -- | Build the base prompt for the agent
311: buildBasePrompt :: TaskCore.Task -> Text -> FilePath -> Text
312: buildBasePrompt task ns repo =
313: "You are a Worker Agent.\n"
314: <> "Your goal is to implement the following task:\n\n"
315: <> formatTask task
316: <> "\n\nCRITICAL INSTRUCTIONS:\n"
317: <> "1. Analyze the codebase to understand where to make changes.\n"
318: <> "2. Implement the changes by editing files.\n"
319: <> "3. BEFORE finishing, you MUST run: bild --test "
320: <> ns
321: <> "\n"
322: <> "4. Fix ALL errors from bild --test (including lint issues).\n"
323: <> "5. Keep running bild --test until it passes with no errors.\n"
324: <> "6. Do NOT update task status or manage git.\n"
325: <> "7. Only exit after bild --test passes.\n\n"
326: <> "IMPORTANT: The git commit will fail if lint finds issues.\n"
327: <> "You must fix all lint suggestions.\n\n"
328: <> "BUILD SYSTEM NOTES:\n"
329: <> "- Running 'bild --test "
330: <> ns
331: <> "' automatically tests ALL dependencies of that namespace\n"
332: <> "- You do NOT need to run bild --test on individual files - just the main namespace\n"
333: <> "- Once tests pass, do NOT re-run them unless you make more changes\n"
334: <> "- The 'lint' command will be run automatically during git commit via hooks\n"
335: <> "- You can run 'lint --fix' on changed files if needed, but it's optional\n\n"
336: <> "Context:\n"
337: <> "- Working directory: "
338: <> Text.pack repo
339: <> "\n"
340: <> "- Namespace: "
341: <> ns
342: <> "\n"
343:
344: -- | Build retry context prompt
345: buildRetryPrompt :: Maybe TaskCore.RetryContext -> Text
346: buildRetryPrompt Nothing = ""
347: buildRetryPrompt (Just ctx) =
348: "\n\n## RETRY CONTEXT (IMPORTANT)\n\n"
349: <> "This task was previously attempted but failed. Attempt: "
350: <> tshow (TaskCore.retryAttempt ctx)
351: <> "/3\n"
352: <> "Reason: "
353: <> TaskCore.retryReason ctx
354: <> "\n\n"
355: <> ( if null (TaskCore.retryConflictFiles ctx)
356: then ""
357: else
358: "Conflicting files from previous attempt:\n"
359: <> Text.unlines (map (" - " <>) (TaskCore.retryConflictFiles ctx))
360: <> "\n"
361: )
362: <> "Original commit: "
363: <> TaskCore.retryOriginalCommit ctx
364: <> "\n\n"
365: <> maybe "" (\notes -> "## HUMAN NOTES/GUIDANCE\n\n" <> notes <> "\n\n") (TaskCore.retryNotes ctx)
366: <> "INSTRUCTIONS FOR RETRY:\n"
367: <> "- The codebase has changed since your last attempt\n"
368: <> "- Re-implement this task on top of the CURRENT codebase\n"
369: <> "- If there were merge conflicts, the conflicting files may have been modified by others\n"
370: <> "- Review the current state of those files before making changes\n"
371:
372: -- | Select model based on task complexity (1-5 scale)
373: -- Uses OpenRouter model identifiers for Claude models
374: selectModel :: TaskCore.Task -> Text
375: selectModel task = selectModelByComplexity (TaskCore.taskComplexity task)
376:
377: -- | Select model based on complexity level
378: selectModelByComplexity :: Maybe Int -> Text
379: selectModelByComplexity Nothing = "anthropic/claude-sonnet-4.5"
380: selectModelByComplexity (Just 1) = "anthropic/claude-haiku-4.5"
381: selectModelByComplexity (Just 2) = "anthropic/claude-haiku-4.5"
382: selectModelByComplexity (Just 3) = "anthropic/claude-sonnet-4.5"
383: selectModelByComplexity (Just 4) = "anthropic/claude-sonnet-4.5"
384: selectModelByComplexity (Just 5) = "anthropic/claude-opus-4.5"
385: selectModelByComplexity (Just _) = "anthropic/claude-sonnet-4.5"
386:
387: formatTask :: TaskCore.Task -> Text
388: formatTask t =
389: "Task: "
390: <> TaskCore.taskId t
391: <> "\n"
392: <> "Title: "
393: <> TaskCore.taskTitle t
394: <> "\n"
395: <> "Type: "
396: <> Text.pack (show (TaskCore.taskType t))
397: <> "\n"
398: <> "Status: "
399: <> Text.pack (show (TaskCore.taskStatus t))
400: <> "\n"
401: <> "Priority: "
402: <> Text.pack (show (TaskCore.taskPriority t))
403: <> "\n"
404: <> maybe "" (\p -> "Parent: " <> p <> "\n") (TaskCore.taskParent t)
405: <> maybe "" (\ns -> "Namespace: " <> ns <> "\n") (TaskCore.taskNamespace t)
406: <> "Created: "
407: <> Text.pack (show (TaskCore.taskCreatedAt t))
408: <> "\n"
409: <> "Updated: "
410: <> Text.pack (show (TaskCore.taskUpdatedAt t))
411: <> "\n"
412: <> (if Text.null (TaskCore.taskDescription t) then "" else "Description:\n" <> TaskCore.taskDescription t <> "\n\n")
413: <> formatDeps (TaskCore.taskDependencies t)
414: <> formatComments (TaskCore.taskComments t)
415: where
416: formatDeps [] = ""
417: formatDeps deps = "\nDependencies:\n" <> Text.unlines (map formatDep deps)
418: formatDep dep = " - " <> TaskCore.depId dep <> " [" <> Text.pack (show (TaskCore.depType dep)) <> "]"
419: formatComments [] = ""
420: formatComments cs = "\nComments/Notes:\n" <> Text.unlines (map formatComment cs)
421: formatComment c = " [" <> Text.pack (show (TaskCore.commentCreatedAt c)) <> "] " <> TaskCore.commentText c
422:
423: formatCommitMessage :: TaskCore.Task -> Text -> Text
424: formatCommitMessage task agentOutput =
425: let tid = TaskCore.taskId task
426: subject = cleanSubject (TaskCore.taskTitle task)
427: body = cleanBody agentOutput
428: in if Text.null body
429: then subject <> "\n\nTask-Id: " <> tid
430: else subject <> "\n\n" <> body <> "\n\nTask-Id: " <> tid
431: where
432: cleanSubject s =
433: let trailingPunct = ['.', ':', '!', '?', ',', ';', ' ', '-']
434: stripped = Text.dropWhileEnd (`elem` trailingPunct) s
435: truncated = Text.take 72 stripped
436: noPunct = Text.dropWhileEnd (`elem` trailingPunct) truncated
437: capitalized = case Text.uncons noPunct of
438: Just (c, rest) -> Text.cons (toUpper c) rest
439: Nothing -> noPunct
440: in capitalized
441:
442: cleanBody :: Text -> Text
443: cleanBody output =
444: let stripped = Text.strip output
445: in if Text.null stripped
446: then ""
447: else
448: let lns = Text.lines stripped
449: cleaned = [Text.take 72 ln | ln <- lns]
450: in Text.intercalate "\n" cleaned
451:
452: -- | Get facts relevant to a task based on namespace/project
453: getRelevantFacts :: TaskCore.Task -> IO [TaskCore.Fact]
454: getRelevantFacts task = do
455: let namespace = fromMaybe "Omni" (TaskCore.taskNamespace task)
456: projectFacts <- Fact.getFactsByProject namespace
457: let sorted = List.sortBy (comparing (Down <. TaskCore.factConfidence)) projectFacts
458: pure (take 10 sorted)
459:
460: -- | Format facts for inclusion in the prompt
461: formatFacts :: [TaskCore.Fact] -> Text
462: formatFacts [] = ""
463: formatFacts facts =
464: Text.unlines
465: [ "\n\nKNOWLEDGE BASE FACTS:",
466: "(These are learned patterns/conventions from previous work)",
467: ""
468: ]
469: <> Text.unlines (map formatFact facts)
470:
471: -- | Format a single fact for the prompt
472: formatFact :: TaskCore.Fact -> Text
473: formatFact f =
474: "- "
475: <> TaskCore.factContent f
476: <> ( if null (TaskCore.factRelatedFiles f)
477: then ""
478: else " [" <> Text.intercalate ", " (TaskCore.factRelatedFiles f) <> "]"
479: )
Now let me check the existing directory structure and see if there are any examples of progress tracking:
ls -la _/llm/ 2>/dev/null | head -20 || echo "Directory does not exist yet"total 684 drwxr-xr-x 2 ben users 4096 Dec 1 07:52 . drwxr-xr-x 10 ben users 4096 Dec 1 07:51 .. -rw-r--r-- 1 ben users 1526 Nov 10 14:48 403.html -rw-r--r-- 1 ben users 9206 Nov 13 15:21 account_page_replacement.py -rw-r--r-- 1 ben users 38980 Nov 30 07:23 amp.log -rw-r--r-- 1 ben users 11562 Nov 8 17:16 beads-projects.md -rw-r--r-- 1 ben users 42654 Nov 14 10:28 BILD_PERFORMANCE_PLAN.md -rw-r--r-- 1 ben users 7194 Nov 26 16:59 BILD_PIPELINE_DESIGN.md -rw-r--r-- 1 ben users 15622 Nov 14 23:23 CONCURRENT_LOG_DESIGN.md -rw-r--r-- 1 ben users 2371 Dec 1 04:27 cost-reporting-fix-summary.md -rw-r--r-- 1 ben users 4015 Nov 25 23:09 DESIGN_Gerrit_Conflict_Handling.md -rw-r--r-- 1 ben users 689 Sep 5 16:11 error.txt -rw-r--r-- 1 ben users 1879 Nov 12 14:53 goinglive.md -rw-r--r-- 1 ben users 60794 Nov 26 14:20 jr-loop.log -rw-r--r-- 1 ben users 17862 Nov 24 11:05 jr.md -rw-r--r-- 1 ben users 333084 Jul 24 15:21 ludic.txt -rw-r--r-- 1 ben users 2217 Nov 15 09:56 NIXOS_INTEGRATION_DESIGN.md -rw-r--r-- 1 ben users 22444 Nov 14 14:02 PARALLEL_BUILDS_PLAN.md -rw-r--r-- 1 ben users 4347 Nov 26 23:33 PLAN_Agent_Architecture.md
Good, the _/llm/ directory exists. Now let me check the Task module to understand the RetryContext structure better:
data RetryContextOmni/Task/Core.hs @101-130101: data RetryContext = RetryContext
102: { retryTaskId :: Text,
103: retryOriginalCommit :: Text,
104: retryConflictFiles :: [Text],
105: retryAttempt :: Int,
106: retryReason :: Text, -- "merge_conflict" | "ci_failure" | "rejected"
107: retryNotes :: Maybe Text -- Human notes/guidance for intervention
108: }
109: deriving (Show, Eq, Generic)
110:
111: -- Activity stage for task_activity tracking
112: data ActivityStage = Claiming | Running | Reviewing | Retrying | Completed | Failed
113: deriving (Show, Eq, Read, Generic)
114:
115: -- Task activity log entry
116: data TaskActivity = TaskActivity
117: { activityId :: Maybe Int, -- NULL for new entries, set by DB
118: activityTaskId :: Text,
119: activityTimestamp :: UTCTime,
120: activityStage :: ActivityStage,
121: activityMessage :: Maybe Text,
122: activityMetadata :: Maybe Text, -- JSON for extra data
123: activityThreadUrl :: Maybe Text, -- Link to agent session (unused with native Engine)
124: activityStartedAt :: Maybe UTCTime, -- When work started
125: activityCompletedAt :: Maybe UTCTime, -- When work completed
126: activityCostCents :: Maybe Int, -- API cost in cents
127: activityTokensUsed :: Maybe Int -- Total tokens used
128: }
129: deriving (Show, Eq, Generic)
130:
Now let me understand the structure better. I need to:
1. Add progress file tracking (read/write progress files in _/llm/${taskId}-progress.md)
2. Update the prompt to enforce incremental workflow
3. Make verification more explicit
4. Avoid redundant testing
Let me start implementing these changes:
Omni/Agent/Worker.hsNow let me update the buildBasePrompt to emphasize incremental workflow:
Omni/Agent/Worker.hsNow I need to add the helper functions for progress file handling and the progress prompt builder:
Omni/Agent/Worker.hsNow I need to add a new tool for writing progress files. Let me check the Tools module:
Omni/Agent/Tools.hs1: {-# LANGUAGE DeriveGeneric #-}
2: {-# LANGUAGE OverloadedStrings #-}
3: {-# LANGUAGE NoImplicitPrelude #-}
4:
5: -- | Core coding tools for Jr agent.
6: --
7: -- Provides implementations for:
8: -- - readFile: Read file contents
9: -- - writeFile: Write file contents
10: -- - editFile: Search/replace edit
11: -- - runBash: Execute shell commands
12: -- - searchCodebase: Ripgrep wrapper for code search
13: --
14: -- All tools return structured JSON results.
15: --
16: -- : out omni-agent-tools
17: -- : dep aeson
18: -- : dep directory
19: module Omni.Agent.Tools
20: ( readFileTool,
21: writeFileTool,
22: editFileTool,
23: runBashTool,
24: searchCodebaseTool,
25: allTools,
26: ReadFileArgs (..),
27: WriteFileArgs (..),
28: EditFileArgs (..),
29: RunBashArgs (..),
30: SearchCodebaseArgs (..),
31: ToolResult (..),
32: main,
33: test,
34: )
35: where
36:
37: import Alpha
38: import Data.Aeson ((.!=), (.:), (.:?), (.=))
39: import qualified Data.Aeson as Aeson
40: import qualified Data.List as List
41: import qualified Data.Text as Text
42: import qualified Data.Text.IO as TextIO
43: import qualified Omni.Agent.Engine as Engine
44: import qualified Omni.Test as Test
45: import qualified System.Directory as Directory
46: import qualified System.Exit as Exit
47: import qualified System.Process as Process
48:
49: main :: IO ()
50: main = Test.run test
51:
52: test :: Test.Tree
53: test =
54: Test.group
55: "Omni.Agent.Tools"
56: [ Test.unit "readFileTool schema is valid" <| do
57: let schema = Engine.toolJsonSchema readFileTool
58: case schema of
59: Aeson.Object _ -> pure ()
60: _ -> Test.assertFailure "Schema should be an object",
61: Test.unit "writeFileTool schema is valid" <| do
62: let schema = Engine.toolJsonSchema writeFileTool
63: case schema of
64: Aeson.Object _ -> pure ()
65: _ -> Test.assertFailure "Schema should be an object",
66: Test.unit "editFileTool schema is valid" <| do
67: let schema = Engine.toolJsonSchema editFileTool
68: case schema of
69: Aeson.Object _ -> pure ()
70: _ -> Test.assertFailure "Schema should be an object",
71: Test.unit "runBashTool schema is valid" <| do
72: let schema = Engine.toolJsonSchema runBashTool
73: case schema of
74: Aeson.Object _ -> pure ()
75: _ -> Test.assertFailure "Schema should be an object",
76: Test.unit "searchCodebaseTool schema is valid" <| do
77: let schema = Engine.toolJsonSchema searchCodebaseTool
78: case schema of
79: Aeson.Object _ -> pure ()
80: _ -> Test.assertFailure "Schema should be an object",
81: Test.unit "allTools contains 5 tools" <| do
82: length allTools Test.@=? 5,
83: Test.unit "ReadFileArgs parses correctly" <| do
84: let json = Aeson.object ["path" .= ("test.txt" :: Text)]
85: case Aeson.fromJSON json of
86: Aeson.Success (args :: ReadFileArgs) -> readFilePath args Test.@=? "test.txt"
87: Aeson.Error e -> Test.assertFailure e,
88: Test.unit "WriteFileArgs parses correctly" <| do
89: let json = Aeson.object ["path" .= ("test.txt" :: Text), "content" .= ("hello" :: Text)]
90: case Aeson.fromJSON json of
91: Aeson.Success (args :: WriteFileArgs) -> do
92: writeFilePath args Test.@=? "test.txt"
93: writeFileContent args Test.@=? "hello"
94: Aeson.Error e -> Test.assertFailure e,
95: Test.unit "EditFileArgs parses correctly" <| do
96: let json =
97: Aeson.object
98: [ "path" .= ("test.txt" :: Text),
99: "old_str" .= ("foo" :: Text),
100: "new_str" .= ("bar" :: Text)
101: ]
102: case Aeson.fromJSON json of
103: Aeson.Success (args :: EditFileArgs) -> do
104: editFilePath args Test.@=? "test.txt"
105: editFileOldStr args Test.@=? "foo"
106: editFileNewStr args Test.@=? "bar"
107: Aeson.Error e -> Test.assertFailure e,
108: Test.unit "RunBashArgs parses correctly" <| do
109: let json = Aeson.object ["command" .= ("ls -la" :: Text)]
110: case Aeson.fromJSON json of
111: Aeson.Success (args :: RunBashArgs) -> runBashCommand args Test.@=? "ls -la"
112: Aeson.Error e -> Test.assertFailure e,
113: Test.unit "SearchCodebaseArgs parses correctly" <| do
114: let json = Aeson.object ["pattern" .= ("TODO" :: Text)]
115: case Aeson.fromJSON json of
116: Aeson.Success (args :: SearchCodebaseArgs) -> searchPattern args Test.@=? "TODO"
117: Aeson.Error e -> Test.assertFailure e,
118: Test.unit "ToolResult success JSON roundtrip" <| do
119: let result = ToolResult True "done" Nothing
120: case Aeson.decode (Aeson.encode result) of
121: Nothing -> Test.assertFailure "Failed to decode ToolResult"
122: Just decoded -> toolResultSuccess decoded Test.@=? True,
123: Test.unit "ToolResult failure JSON roundtrip" <| do
124: let result = ToolResult False "" (Just "error occurred")
125: case Aeson.decode (Aeson.encode result) of
126: Nothing -> Test.assertFailure "Failed to decode ToolResult"
127: Just decoded -> toolResultError decoded Test.@=? Just "error occurred",
128: Test.unit "readFileTool handles missing files" <| do
129: let args = Aeson.object ["path" .= ("/nonexistent/path/to/file.txt" :: Text)]
130: result <- Engine.toolExecute readFileTool args
131: case Aeson.fromJSON result of
132: Aeson.Success (tr :: ToolResult) -> do
133: toolResultSuccess tr Test.@=? False
134: isJust (toolResultError tr) Test.@=? True
135: Aeson.Error e -> Test.assertFailure e,
136: Test.unit "editFileTool handles no-match case" <| do
137: let args =
138: Aeson.object
139: [ "path" .= ("/nonexistent/file.txt" :: Text),
140: "old_str" .= ("needle" :: Text),
141: "new_str" .= ("replacement" :: Text)
142: ]
143: result <- Engine.toolExecute editFileTool args
144: case Aeson.fromJSON result of
145: Aeson.Success (tr :: ToolResult) -> toolResultSuccess tr Test.@=? False
146: Aeson.Error e -> Test.assertFailure e,
147: Test.unit "runBashTool captures exit codes" <| do
148: let args = Aeson.object ["command" .= ("exit 42" :: Text)]
149: result <- Engine.toolExecute runBashTool args
150: case Aeson.fromJSON result of
151: Aeson.Success (tr :: ToolResult) -> do
152: toolResultSuccess tr Test.@=? False
153: toolResultError tr Test.@=? Just "Exit code: 42"
154: Aeson.Error e -> Test.assertFailure e,
155: Test.unit "runBashTool captures stdout" <| do
156: let args = Aeson.object ["command" .= ("echo hello" :: Text)]
157: result <- Engine.toolExecute runBashTool args
158: case Aeson.fromJSON result of
159: Aeson.Success (tr :: ToolResult) -> do
160: toolResultSuccess tr Test.@=? True
161: ("hello" `Text.isInfixOf` toolResultOutput tr) Test.@=? True
162: Aeson.Error e -> Test.assertFailure e,
163: Test.unit "runBashTool validates cwd exists" <| do
164: let args =
165: Aeson.object
166: [ "command" .= ("echo test" :: Text),
167: "cwd" .= ("/nonexistent/path/that/does/not/exist" :: Text)
168: ]
169: result <- Engine.toolExecute runBashTool args
170: case Aeson.fromJSON result of
171: Aeson.Success (tr :: ToolResult) -> do
172: toolResultSuccess tr Test.@=? False
173: isJust (toolResultError tr) Test.@=? True
174: Aeson.Error e -> Test.assertFailure e,
175: Test.unit "searchCodebaseTool returns structured results" <| do
176: let args =
177: Aeson.object
178: [ "pattern" .= ("module" :: Text),
179: "path" .= ("." :: Text),
180: "max_results" .= (5 :: Int)
181: ]
182: result <- Engine.toolExecute searchCodebaseTool args
183: case Aeson.fromJSON result of
184: Aeson.Success (tr :: ToolResult) -> toolResultSuccess tr Test.@=? True
185: Aeson.Error e -> Test.assertFailure e
186: ]
187:
188: data ToolResult = ToolResult
189: { toolResultSuccess :: Bool,
190: toolResultOutput :: Text,
191: toolResultError :: Maybe Text
192: }
193: deriving (Show, Eq, Generic)
194:
195: instance Aeson.ToJSON ToolResult where
196: toJSON r =
197: Aeson.object
198: <| catMaybes
199: [ Just ("success" .= toolResultSuccess r),
200: Just ("output" .= toolResultOutput r),
201: ("error" .=) </ toolResultError r
202: ]
203:
204: instance Aeson.FromJSON ToolResult where
205: parseJSON =
206: Aeson.withObject "ToolResult" <| \v ->
207: (ToolResult </ (v .: "success"))
208: <*> (v .:? "output" .!= "")
209: <*> (v .:? "error")
210:
211: mkSuccess :: Text -> Aeson.Value
212: mkSuccess output = Aeson.toJSON <| ToolResult True output Nothing
213:
214: mkError :: Text -> Aeson.Value
215: mkError err = Aeson.toJSON <| ToolResult False "" (Just err)
216:
217: allTools :: [Engine.Tool]
218: allTools =
219: [ readFileTool,
220: writeFileTool,
221: editFileTool,
222: runBashTool,
223: searchCodebaseTool
224: ]
225:
226: data ReadFileArgs = ReadFileArgs
227: { readFilePath :: Text,
228: readFileStartLine :: Maybe Int,
229: readFileEndLine :: Maybe Int
230: }
231: deriving (Show, Eq, Generic)
232:
233: instance Aeson.FromJSON ReadFileArgs where
234: parseJSON =
235: Aeson.withObject "ReadFileArgs" <| \v ->
236: (ReadFileArgs </ (v .: "path"))
237: <*> (v .:? "start_line")
238: <*> (v .:? "end_line")
239:
240: readFileTool :: Engine.Tool
241: readFileTool =
242: Engine.Tool
243: { Engine.toolName = "read_file",
244: Engine.toolDescription = "Read the contents of a file. Can optionally read a specific line range.",
245: Engine.toolJsonSchema =
246: Aeson.object
247: [ "type" .= ("object" :: Text),
248: "properties"
249: .= Aeson.object
250: [ "path"
251: .= Aeson.object
252: [ "type" .= ("string" :: Text),
253: "description" .= ("Absolute path to the file to read" :: Text)
254: ],
255: "start_line"
256: .= Aeson.object
257: [ "type" .= ("integer" :: Text),
258: "description" .= ("Optional: first line to read (1-indexed)" :: Text)
259: ],
260: "end_line"
261: .= Aeson.object
262: [ "type" .= ("integer" :: Text),
263: "description" .= ("Optional: last line to read (1-indexed)" :: Text)
264: ]
265: ],
266: "required" .= (["path"] :: [Text])
267: ],
268: Engine.toolExecute = executeReadFile
269: }
270:
271: executeReadFile :: Aeson.Value -> IO Aeson.Value
272: executeReadFile v =
273: case Aeson.fromJSON v of
274: Aeson.Error e -> pure <| mkError (Text.pack e)
275: Aeson.Success args -> do
276: let path = Text.unpack (readFilePath args)
277: exists <- Directory.doesFileExist path
278: if exists
279: then do
280: content <- TextIO.readFile path
281: let allLines = Text.lines content
282: startIdx = maybe 0 (\n -> n - 1) (readFileStartLine args)
283: endIdx = maybe (length allLines) identity (readFileEndLine args)
284: selectedLines = take (endIdx - startIdx) (drop startIdx allLines)
285: numberedLines = zipWith formatLine [(startIdx + 1) ..] selectedLines
286: result = Text.unlines numberedLines
287: pure <| mkSuccess result
288: else pure <| mkError ("File not found: " <> readFilePath args)
289: where
290: formatLine :: Int -> Text -> Text
291: formatLine n line = Text.pack (show n) <> ": " <> line
292:
293: data WriteFileArgs = WriteFileArgs
294: { writeFilePath :: Text,
295: writeFileContent :: Text
296: }
297: deriving (Show, Eq, Generic)
298:
299: instance Aeson.FromJSON WriteFileArgs where
300: parseJSON =
301: Aeson.withObject "WriteFileArgs" <| \v ->
302: (WriteFileArgs </ (v .: "path"))
303: <*> (v .: "content")
304:
305: writeFileTool :: Engine.Tool
306: writeFileTool =
307: Engine.Tool
308: { Engine.toolName = "write_file",
309: Engine.toolDescription = "Create or overwrite a file with the given content.",
310: Engine.toolJsonSchema =
311: Aeson.object
312: [ "type" .= ("object" :: Text),
313: "properties"
314: .= Aeson.object
315: [ "path"
316: .= Aeson.object
317: [ "type" .= ("string" :: Text),
318: "description" .= ("Absolute path to the file to write" :: Text)
319: ],
320: "content"
321: .= Aeson.object
322: [ "type" .= ("string" :: Text),
323: "description" .= ("Content to write to the file" :: Text)
324: ]
325: ],
326: "required" .= (["path", "content"] :: [Text])
327: ],
328: Engine.toolExecute = executeWriteFile
329: }
330:
331: executeWriteFile :: Aeson.Value -> IO Aeson.Value
332: executeWriteFile v =
333: case Aeson.fromJSON v of
334: Aeson.Error e -> pure <| mkError (Text.pack e)
335: Aeson.Success args -> do
336: let path = Text.unpack (writeFilePath args)
337: let dir = takeDirectory path
338: dirExists <- Directory.doesDirectoryExist dir
339: if dirExists
340: then do
341: TextIO.writeFile path (writeFileContent args)
342: pure <| mkSuccess ("File written: " <> writeFilePath args)
343: else pure <| mkError ("Parent directory does not exist: " <> Text.pack dir)
344: where
345: takeDirectory :: FilePath -> FilePath
346: takeDirectory p =
347: let parts = Text.splitOn "/" (Text.pack p)
348: in Text.unpack (Text.intercalate "/" (List.init parts))
349:
350: data EditFileArgs = EditFileArgs
351: { editFilePath :: Text,
352: editFileOldStr :: Text,
353: editFileNewStr :: Text,
354: editFileReplaceAll :: Maybe Bool
355: }
356: deriving (Show, Eq, Generic)
357:
358: instance Aeson.FromJSON EditFileArgs where
359: parseJSON =
360: Aeson.withObject "EditFileArgs" <| \v ->
361: (EditFileArgs </ (v .: "path"))
362: <*> (v .: "old_str")
363: <*> (v .: "new_str")
364: <*> (v .:? "replace_all")
365:
366: editFileTool :: Engine.Tool
367: editFileTool =
368: Engine.Tool
369: { Engine.toolName = "edit_file",
370: Engine.toolDescription = "Edit a file by replacing old_str with new_str. By default replaces only the first occurrence unless replace_all is true.",
371: Engine.toolJsonSchema =
372: Aeson.object
373: [ "type" .= ("object" :: Text),
374: "properties"
375: .= Aeson.object
376: [ "path"
377: .= Aeson.object
378: [ "type" .= ("string" :: Text),
379: "description" .= ("Absolute path to the file to edit" :: Text)
380: ],
381: "old_str"
382: .= Aeson.object
383: [ "type" .= ("string" :: Text),
384: "description" .= ("The text to search for and replace" :: Text)
385: ],
386: "new_str"
387: .= Aeson.object
388: [ "type" .= ("string" :: Text),
389: "description" .= ("The replacement text" :: Text)
390: ],
391: "replace_all"
392: .= Aeson.object
393: [ "type" .= ("boolean" :: Text),
394: "description" .= ("If true, replace all occurrences; otherwise replace only the first" :: Text)
395: ]
396: ],
397: "required" .= (["path", "old_str", "new_str"] :: [Text])
398: ],
399: Engine.toolExecute = executeEditFile
400: }
401:
402: executeEditFile :: Aeson.Value -> IO Aeson.Value
403: executeEditFile v =
404: case Aeson.fromJSON v of
405: Aeson.Error e -> pure <| mkError (Text.pack e)
406: Aeson.Success args -> do
407: let path = Text.unpack (editFilePath args)
408: exists <- Directory.doesFileExist path
409: if exists
410: then do
411: content <- TextIO.readFile path
412: let oldStr = editFileOldStr args
413: newStr = editFileNewStr args
414: replaceAll = fromMaybe False (editFileReplaceAll args)
415: if Text.isInfixOf oldStr content
416: then do
417: let newContent =
418: if replaceAll
419: then Text.replace oldStr newStr content
420: else replaceFirst oldStr newStr content
421: TextIO.writeFile path newContent
422: let count =
423: if replaceAll
424: then Text.count oldStr content
425: else 1
426: pure <| mkSuccess ("Replaced " <> tshow count <> " occurrence(s)")
427: else pure <| mkError ("old_str not found in file: " <> editFilePath args)
428: else pure <| mkError ("File not found: " <> editFilePath args)
429:
430: replaceFirst :: Text -> Text -> Text -> Text
431: replaceFirst old new content =
432: case Text.breakOn old content of
433: (before, after) ->
434: if Text.null after
435: then content
436: else before <> new <> Text.drop (Text.length old) after
437:
438: data RunBashArgs = RunBashArgs
439: { runBashCommand :: Text,
440: runBashCwd :: Maybe Text,
441: runBashTimeout :: Maybe Int
442: }
443: deriving (Show, Eq, Generic)
444:
445: instance Aeson.FromJSON RunBashArgs where
446: parseJSON =
447: Aeson.withObject "RunBashArgs" <| \v ->
448: (RunBashArgs </ (v .: "command"))
449: <*> (v .:? "cwd")
450: <*> (v .:? "timeout")
451:
452: runBashTool :: Engine.Tool
453: runBashTool =
454: Engine.Tool
455: { Engine.toolName = "run_bash",
456: Engine.toolDescription = "Execute a shell command and return stdout/stderr.",
457: Engine.toolJsonSchema =
458: Aeson.object
459: [ "type" .= ("object" :: Text),
460: "properties"
461: .= Aeson.object
462: [ "command"
463: .= Aeson.object
464: [ "type" .= ("string" :: Text),
465: "description" .= ("The shell command to execute" :: Text)
466: ],
467: "cwd"
468: .= Aeson.object
469: [ "type" .= ("string" :: Text),
470: "description" .= ("Optional: working directory for the command" :: Text)
471: ],
472: "timeout"
473: .= Aeson.object
474: [ "type" .= ("integer" :: Text),
475: "description" .= ("Optional: timeout in seconds (default: 300)" :: Text)
476: ]
477: ],
478: "required" .= (["command"] :: [Text])
479: ],
480: Engine.toolExecute = executeRunBash
481: }
482:
483: executeRunBash :: Aeson.Value -> IO Aeson.Value
484: executeRunBash v =
485: case Aeson.fromJSON v of
486: Aeson.Error e -> pure <| mkError (Text.pack e)
487: Aeson.Success args -> do
488: let cmd = Text.unpack (runBashCommand args)
489: maybeCwd = runBashCwd args
490: cwdValid <- case maybeCwd of
491: Nothing -> pure True
492: Just cwd -> Directory.doesDirectoryExist (Text.unpack cwd)
493: if not cwdValid
494: then
495: pure
496: <| mkError
497: ("Working directory does not exist: " <> fromMaybe "" maybeCwd)
498: else do
499: let proc =
500: (Process.shell cmd)
501: { Process.cwd = Text.unpack </ maybeCwd
502: }
503: (exitCode, stdoutStr, stderrStr) <- Process.readCreateProcessWithExitCode proc ""
504: let output = Text.pack stdoutStr <> Text.pack stderrStr
505: case exitCode of
506: Exit.ExitSuccess ->
507: pure
508: <| Aeson.toJSON
509: <| ToolResult
510: { toolResultSuccess = True,
511: toolResultOutput = output,
512: toolResultError = Nothing
513: }
514: Exit.ExitFailure code ->
515: pure
516: <| Aeson.toJSON
517: <| ToolResult
518: { toolResultSuccess = False,
519: toolResultOutput = output,
520: toolResultError = Just ("Exit code: " <> tshow code)
521: }
522:
523: data SearchCodebaseArgs = SearchCodebaseArgs
524: { searchPattern :: Text,
525: searchPath :: Maybe Text,
526: searchGlob :: Maybe Text,
527: searchCaseSensitive :: Maybe Bool,
528: searchMaxResults :: Maybe Int
529: }
530: deriving (Show, Eq, Generic)
531:
532: instance Aeson.FromJSON SearchCodebaseArgs where
533: parseJSON =
534: Aeson.withObject "SearchCodebaseArgs" <| \v ->
535: (SearchCodebaseArgs </ (v .: "pattern"))
536: <*> (v .:? "path")
537: <*> (v .:? "glob")
538: <*> (v .:? "case_sensitive")
539: <*> (v .:? "max_results")
540:
541: searchCodebaseTool :: Engine.Tool
542: searchCodebaseTool =
543: Engine.Tool
544: { Engine.toolName = "search_codebase",
545: Engine.toolDescription = "Search the codebase using ripgrep. Returns matching lines with file paths and line numbers.",
546: Engine.toolJsonSchema =
547: Aeson.object
548: [ "type" .= ("object" :: Text),
549: "properties"
550: .= Aeson.object
551: [ "pattern"
552: .= Aeson.object
553: [ "type" .= ("string" :: Text),
554: "description" .= ("The regex pattern to search for" :: Text)
555: ],
556: "path"
557: .= Aeson.object
558: [ "type" .= ("string" :: Text),
559: "description" .= ("Optional: directory or file path to search in" :: Text)
560: ],
561: "glob"
562: .= Aeson.object
563: [ "type" .= ("string" :: Text),
564: "description" .= ("Optional: file glob pattern (e.g., '*.hs')" :: Text)
565: ],
566: "case_sensitive"
567: .= Aeson.object
568: [ "type" .= ("boolean" :: Text),
569: "description" .= ("Optional: case sensitive search (default: false)" :: Text)
570: ],
571: "max_results"
572: .= Aeson.object
573: [ "type" .= ("integer" :: Text),
574: "description" .= ("Optional: maximum number of results (default: 100)" :: Text)
575: ]
576: ],
577: "required" .= (["pattern"] :: [Text])
578: ],
579: Engine.toolExecute = executeSearchCodebase
580: }
581:
582: executeSearchCodebase :: Aeson.Value -> IO Aeson.Value
583: executeSearchCodebase v =
584: case Aeson.fromJSON v of
585: Aeson.Error e -> pure <| mkError (Text.pack e)
586: Aeson.Success args -> do
587: let pat = Text.unpack (searchPattern args)
588: maxRes = fromMaybe 100 (searchMaxResults args)
589: caseSensitive = fromMaybe False (searchCaseSensitive args)
590: baseArgs =
591: ["--line-number", "--no-heading", "--max-count=" <> show maxRes, pat]
592: caseArgs = ["--ignore-case" | not caseSensitive]
593: globArgs = maybe [] (\g -> ["--glob", Text.unpack g]) (searchGlob args)
594: pathArg = maybe ["."] (\p -> [Text.unpack p]) (searchPath args)
595: allArgs = caseArgs <> globArgs <> baseArgs <> pathArg
596: proc = Process.proc "rg" allArgs
597: (exitCode, stdoutStr, stderrStr) <- Process.readCreateProcessWithExitCode proc ""
598: case exitCode of
599: Exit.ExitSuccess ->
600: pure <| mkSuccess (Text.pack stdoutStr)
601: Exit.ExitFailure 1 ->
602: pure <| mkSuccess "No matches found"
603: Exit.ExitFailure code ->
604: pure <| mkError ("ripgrep failed with code " <> tshow code <> ": " <> Text.pack stderrStr)
I see that the tools module doesn't have a specific tool for writing progress files, but agents can already use write_file to write to _/llm/ directory. However, looking at the prompt instruction that mentions writing progress, I should make sure this is clear. The current tools are sufficient - the agent can use write_file to write the progress file.
Now let me build and test the changes:
cd /root/omni && bild --test Omni/Agent/Worker.hs 2>&1/bin/sh: line 1: cd: /root/omni: Not a directory
bild --test Omni/Agent/Worker.hs 2>&17[10000;10000H [1A[1G[2K[.] Omni/Agent/Worker.hs [1A[1G[2K[0m[+] Omni/Agent/Worker.hs[0m[1B[1A[1G[2K[0m[~] Omni/Agent/Worker.hs[0m[1B[1A[1G[2K[0m[38;5;2m[✓] Omni/Agent/Worker.hs[0m[1B
Great! The tests pass. Now let me write a progress file to document the changes I made for this task:
_/llm/t-203-progress.mdPerfect! Now let me run the tests one more time to make sure everything is still working correctly:
bild --test Omni/Agent/Worker.hs 2>&17[10000;10000H [1A[1G[2K[.] Omni/Agent/Worker.hs [1A[1G[2K[0m[+] Omni/Agent/Worker.hs[0m[1B[1A[1G[2K[0m[~] Omni/Agent/Worker.hs[0m[1B[1A[1G[2K[0m[38;5;2m[✓] Omni/Agent/Worker.hs[0m[1B
Excellent! The tests pass. Let me verify that there are no lint issues:
lint Omni/Agent/Worker.hs 2>&1[0m[38;5;7m[2Kinfo: lint: ormolu: 1 files [0m[0m[38;5;7m[2Kinfo: lint: hlint: 1 files [0m[0m[38;5;2m[2Kgood: lint: ormolu [0m[0m [0m[0m[38;5;2m[2Kgood: lint: hlint [0m[0m [0m
Perfect! No lint issues. Let me create a summary document to show what was implemented:
cat > _/llm/t-203-implementation-summary.md << 'EOF'
# Implementation Summary: t-203
## Overview
Successfully implemented progress file tracking and incremental workflow patterns for Jr Worker agent, based on industry best practices from Anthropic, OpenAI, and Amp.
## Changes Made to Omni/Agent/Worker.hs
### 1. Progress File Reading (Lines 344-356)
Added two new functions:
```haskell
-- | Read progress file for a task if it exists
readProgressFile :: FilePath -> Text -> IO (Maybe Text)
-- | Build progress context prompt
buildProgressPrompt :: Maybe Text -> Text
```
These functions enable the agent to:
- Read `_/llm/${taskId}-progress.md` at the start of each session
- Include progress context in the system prompt
- Understand what work has already been completed
### 2. Updated runWithEngine (Lines 202-220)
Modified the engine initialization to:
- Call `readProgressFile` to load existing progress
- Build progress prompt with `buildProgressPrompt`
- Include progress context in the final prompt: `basePrompt <> progressPrompt <> retryPrompt`
### 3. Enhanced Base Prompt (Lines 310-342)
Completely rewrote the prompt structure to emphasize incremental workflow:
**Key changes:**
- Instruction #1: "Read AGENTS.md and any existing progress file"
- Instruction #2: "Pick ONE specific change to implement (not everything at once)"
- Instruction #4: "Implement ONLY that one change"
- Instruction #8: "After tests pass, write progress to: _/llm/${taskId}-progress.md"
- Instruction #10: "Only exit after bild --test passes and progress is saved"
**New section added:**
```
INCREMENTAL WORKFLOW (IMPORTANT):
- DO NOT try to implement everything in one go
- Make ONE focused change, test it, save progress, then stop
- The task may be run multiple times to complete all changes
- Each session should leave the code in a clean, testable state
- If the task is already complete, just verify tests pass and note that in progress
```
**Enhanced BUILD SYSTEM NOTES:**
- Clarified to run `bild --test` on namespace ONCE (not individual files)
- Emphasized not to re-run tests unless more changes are made
## How It Works
### First Session
1. Agent reads AGENTS.md
2. No progress file exists yet
3. Agent picks ONE change to implement
4. Implements change, runs tests
5. Writes progress to `_/llm/${taskId}-progress.md`
6. Exits cleanly
### Subsequent Sessions
1. Agent reads AGENTS.md
2. Agent reads progress file showing what's done
3. Agent understands what remains
4. Agent picks NEXT logical change
5. Implements change, runs tests
6. Updates progress file
7. Exits cleanly
### Final Session
1. Agent reads progress showing all work done
2. Verifies tests still pass
3. Notes in progress that task is complete
4. Exits
## Benefits
### Reduced Token Usage
- Agent makes focused, incremental changes instead of trying to do everything at once
- Fewer iterations needed per change
- Less re-testing of passing code
### Better Code Quality
- Each change is isolated and tested
- Code left in clean, mergeable state after each session
- Easier to review changes
### Improved Reliability
- Progress tracking prevents "forgetting" what was done
- Explicit verification before completion
- Clear separation between sessions
### Follows Industry Best Practices
- **Anthropic**: Progress files across sessions, clean state
- **Claude Code**: Incremental workflow, verification before done
- **OpenAI Codex**: Explicit verification steps, task splitting
## Testing
All tests pass:
```bash
$ bild --test Omni/Agent/Worker.hs
[✓] Omni/Agent/Worker.hs
```
No lint issues:
```bash
$ lint Omni/Agent/Worker.hs
good: lint: ormolu
good: lint: hlint
```
## Example Progress File
Created `_/llm/t-203-progress.md` documenting this implementation as an example of the new workflow.
EOF
cat _/llm/t-203-implementation-summary.md
# Implementation Summary: t-203
## Overview
Successfully implemented progress file tracking and incremental workflow patterns for Jr Worker agent, based on industry best practices from Anthropic, OpenAI, and Amp.
## Changes Made to Omni/Agent/Worker.hs
### 1. Progress File Reading (Lines 344-356)
Added two new functions:
```haskell
-- | Read progress file for a task if it exists
readProgressFile :: FilePath -> Text -> IO (Maybe Text)
-- | Build progress context prompt
buildProgressPrompt :: Maybe Text -> Text
```
These functions enable the agent to:
- Read `_/llm/${taskId}-progress.md` at the start of each session
- Include progress context in the system prompt
- Understand what work has already been completed
### 2. Updated runWithEngine (Lines 202-220)
Modified the engine initialization to:
- Call `readProgressFile` to load existing progress
- Build progress prompt with `buildProgressPrompt`
- Include progress context in the final prompt: `basePrompt <> progressPrompt <> retryPrompt`
### 3. Enhanced Base Prompt (Lines 310-342)
Completely rewrote the prompt structure to emphasize incremental workflow:
**Key changes:**
- Instruction #1: "Read AGENTS.md and any existing progress file"
- Instruction #2: "Pick ONE specific change to implement (not everything at once)"
- Instruction #4: "Implement ONLY that one change"
- Instruction #8: "After tests pass, write progress to: _/llm/${taskId}-progress.md"
- Instruction #10: "Only exit after bild --test passes and progress is saved"
**New section added:**
```
INCREMENTAL WORKFLOW (IMPORTANT):
- DO NOT try to implement everything in one go
- Make ONE focused change, test it, save progress, then stop
- The task may be run multiple times to complete all changes
- Each session should leave the code in a clean, testable state
- If the task is already complete, just verify tests pass and note that in progress
```
**Enhanced BUILD SYSTEM NOTES:**
- Clarified to run `bild --test` on namespace ONCE (not individual files)
- Emphasized not to re-run tests unless more changes are made
## How It Works
### First Session
1. Agent reads AGENTS.md
2. No progress file exists yet
3. Agent picks ONE change to implement
4. Implements change, runs tests
5. Writes progress to `_/llm/${taskId}-progress.md`
6. Exits cleanly
### Subsequent Sessions
1. Agent reads AGENTS.md
2. Agent reads progress file showing what's done
3. Agent understands what remains
4. Agent picks NEXT logical change
5. Implements change, runs tests
6. Updates progress file
7. Exits cleanly
### Final Session
1. Agent reads progress showing all work done
2. Verifies tests still pass
3. Notes in progress that task is complete
4. Exits
## Benefits
### Reduced Token Usage
- Agent makes focused, incremental changes instead of trying to do everything at once
- Fewer iterations needed per change
- Less re-testing of passing code
### Better Code Quality
- Each change is isolated and tested
- Code left in clean, mergeable state after each session
- Easier to review changes
### Improved Reliability
- Progress tracking prevents "forgetting" what was done
- Explicit verification before completion
- Clear separation between sessions
### Follows Industry Best Practices
- **Anthropic**: Progress files across sessions, clean state
- **Claude Code**: Incremental workflow, verification before done
- **OpenAI Codex**: Explicit verification steps, task splitting
## Testing
All tests pass:
```bash
$ bild --test Omni/Agent/Worker.hs
[✓] Omni/Agent/Worker.hs
```
No lint issues:
```bash
$ lint Omni/Agent/Worker.hs
good: lint: ormolu
good: lint: hlint
```
## Example Progress File
Created `_/llm/t-203-progress.md` documenting this implementation as an example of the new workflow.
Perfect! Now let me run one final comprehensive test to make sure everything works:
bild --test Omni/Agent/Worker.hs 2>&1 && echo "=== All tests passed ===" 7[10000;10000H [1A[1G[2K[.] Omni/Agent/Worker.hs [1A[1G[2K[0m[+] Omni/Agent/Worker.hs[0m[1B[1A[1G[2K[0m[~] Omni/Agent/Worker.hs[0m[1B[1A[1G[2K[0m[38;5;2m[✓] Omni/Agent/Worker.hs[0m[1B=== All tests passed ===
Excellent! All tests pass. Let me verify the key changes one more time by reviewing the modified sections:
Omni/Agent/Worker.hs @200-230200: -- | Run task using native Engine 201: -- Returns (ExitCode, output text, cost in cents) 202: runWithEngine :: Core.Worker -> FilePath -> TaskCore.Task -> IO (Exit.ExitCode, Text, Double) 203: runWithEngine worker repo task = do 204: -- Read API key from environment 205: maybeApiKey <- Env.lookupEnv "OPENROUTER_API_KEY" 206: case maybeApiKey of 207: Nothing -> pure (Exit.ExitFailure 1, "OPENROUTER_API_KEY not set", 0) 208: Just apiKey -> do 209: -- Check for retry context 210: maybeRetry <- TaskCore.getRetryContext (TaskCore.taskId task) 211: 212: -- Read progress file if it exists 213: progressContent <- readProgressFile repo (TaskCore.taskId task) 214: 215: -- Build the full prompt 216: let ns = fromMaybe "." (TaskCore.taskNamespace task) 217: let basePrompt = buildBasePrompt task ns repo 218: 219: -- Add progress context if present 220: let progressPrompt = buildProgressPrompt progressContent 221: 222: -- Add retry context if present 223: let retryPrompt = buildRetryPrompt maybeRetry 224: 225: let prompt = basePrompt <> progressPrompt <> retryPrompt 226: 227: -- Read AGENTS.md 228: agentsMd <- 229: fmap (fromMaybe "") <| do 230: exists <- Directory.doesFileExist (repo </> "AGENTS.md")
Omni/Agent/Worker.hs @310-343310: case result of 311: Left err -> pure (Exit.ExitFailure 1, "Engine error: " <> err, totalCost) 312: Right agentResult -> do 313: let output = Engine.resultFinalMessage agentResult 314: pure (Exit.ExitSuccess, output, totalCost) 315: 316: -- | Build the base prompt for the agent 317: buildBasePrompt :: TaskCore.Task -> Text -> FilePath -> Text 318: buildBasePrompt task ns repo = 319: "You are a Worker Agent.\n" 320: <> "Your goal is to implement the following task:\n\n" 321: <> formatTask task 322: <> "\n\nCRITICAL INSTRUCTIONS:\n" 323: <> "1. Read AGENTS.md and any existing progress file for this task.\n" 324: <> "2. Pick ONE specific change to implement (not everything at once).\n" 325: <> "3. Analyze the codebase to understand where to make that change.\n" 326: <> "4. Implement ONLY that one change.\n" 327: <> "5. BEFORE finishing, you MUST run: bild --test " 328: <> ns 329: <> "\n" 330: <> "6. Fix ALL errors from bild --test (including lint issues).\n" 331: <> "7. Keep running bild --test until it passes with no errors.\n" 332: <> "8. After tests pass, write progress to: _/llm/" 333: <> TaskCore.taskId task 334: <> "-progress.md\n" 335: <> "9. Do NOT update task status or manage git.\n" 336: <> "10. Only exit after bild --test passes and progress is saved.\n\n" 337: <> "INCREMENTAL WORKFLOW (IMPORTANT):\n" 338: <> "- DO NOT try to implement everything in one go\n" 339: <> "- Make ONE focused change, test it, save progress, then stop\n" 340: <> "- The task may be run multiple times to complete all changes\n" 341: <> "- Each session should leave the code in a clean, testable state\n" 342: <> "- If the task is already complete, just verify tests pass and note that in progress\n\n" 343: <> "IMPORTANT: The git commit will fail if lint finds issues.\n"
Omni/Agent/Worker.hs @344-375344: <> "You must fix all lint suggestions.\n\n" 345: <> "BUILD SYSTEM NOTES:\n" 346: <> "- Running 'bild --test " 347: <> ns 348: <> "' automatically tests ALL dependencies of that namespace\n" 349: <> "- You do NOT need to run bild --test on individual files - just the main namespace ONCE\n" 350: <> "- Once tests pass, do NOT re-run them unless you make more changes\n" 351: <> "- The 'lint' command will be run automatically during git commit via hooks\n" 352: <> "- You can run 'lint --fix' on changed files if needed, but it's optional\n\n" 353: <> "Context:\n" 354: <> "- Working directory: " 355: <> Text.pack repo 356: <> "\n" 357: <> "- Namespace: " 358: <> ns 359: <> "\n" 360: 361: -- | Read progress file for a task if it exists 362: readProgressFile :: FilePath -> Text -> IO (Maybe Text) 363: readProgressFile repo taskId = do 364: let progressPath = repo </> "_" </> "llm" </> Text.unpack taskId <> "-progress.md" 365: exists <- Directory.doesFileExist progressPath 366: if exists 367: then Just </ readFile progressPath 368: else pure Nothing 369: 370: -- | Build progress context prompt 371: buildProgressPrompt :: Maybe Text -> Text 372: buildProgressPrompt Nothing = "" 373: buildProgressPrompt (Just progress) = 374: "\n\n## PROGRESS FROM PREVIOUS SESSIONS\n\n" 375: <> "This task has been worked on before. Here's what has been completed:\n\n"
Perfect! All changes are in place and working correctly. Let me create a final summary:
I have successfully implemented the improvements to Jr Worker agent structure with progress file tracking and incremental workflow. Here's what was accomplished:
1. Progress File Tracking
readProgressFile function to read _/llm/${taskId}-progress.mdbuildProgressPrompt function to include progress context in promptsrunWithEngine to load and include progress at the start of each session2. Incremental Workflow Enforcement
3. Explicit Verification
bild --test before completion4. Avoid Redundant Testing
bild --test on namespace ONCEbild --test Omni/Agent/Worker.hs - PASSED ✓lint Omni/Agent/Worker.hs - NO ISSUES ✓_/llm/t-203-progress.md - Progress file documenting this implementation_/llm/t-203-implementation-summary.md - Detailed summary of changesThe implementation follows industry best practices from Anthropic, OpenAI, and Amp, resulting in: