Agentd daemon pi.log writing is broken - handle closed immediately

t-695·WorkTask·
·
·
Created1 month ago·Updated1 month ago·pipeline runs →

Description

Edit

The deployed agentd daemon (PID 3077988, nix store path /nix/store/vf8n2ir0mb301rlng7n0m4aq0l43mdxk-agentd) fails to write to pi.log files immediately after creating them.

Symptoms:

  • All pi.log files under /var/log/agentd/<run-id>/ are 0 bytes
  • journalctl shows: Failed to write pi log: .../pi.log: hPutStr: illegal operation (handle is closed)
  • Errors start within 2 seconds of agent spawn

Impact:

  • The agents web UI event stream shows "no events yet" for all agents
  • Agent summary/status still works (comes from daemon DB API, not logs)

Root cause: The log file handle opened in spawnPiAgentInner is being closed by something before the readPiOutput async can use it. The systemd service has ProtectSystem=strict with ReadWritePaths=/var/log/agentd, so permissions are fine (the file IS created). Possible GC or file descriptor issue in the deployed version.

Workaround: The agents web UI now shows the agent's summary (from daemon API) in a RESPONSE section, which provides basic send/receive feedback without requiring log files.

Fix needed: Debug why the log handle is closed. May require redeploying agentd with additional logging or a different handle management approach.

Timeline (2)

💬[human]1 month ago

Fixed. pi.log handle bug resolved by using path-based append-per-line instead of persistent handle. Also added messages table to agentd DB for conversation history (user/assistant messages stored and served via GET /agents/:id/messages API). Web UI now renders chat thread from DB.