readyblockedneeds helpallepicskb

Fund: Live hledger data integration

t-654·WorkTask·
·
·
·Omni/Fund/Web.hs
Created1 month ago·Updated1 week ago·pipeline runs →

Description

Edit

Wire live hledger data into the Fund dashboard (Omni/Fund/Web.hs) so projections use actual balance sheet values instead of hardcoded defaults.

Background

The Fund dashboard (Omni/Fund/Web.hs) currently serves an all-client-side HTML page with hardcoded JSON defaults (btcStack, btcPrice, strdPosition, salary, etc). The goal is to add a live data endpoint that reads from the user's hledger journals and pre-seeds the dashboard with real values on page load.

Ben's hledger journals live in ~/fund/ (ledger.journal as root, includes many sub-journals). There is already a Haskell hledger query module at Omni/Agent/Tools/Hledger.hs that shells out to the hledger binary and returns parsed results.

What to build

1. New module: Omni/Fund/Ledger.hs

  • Functions to extract key financial metrics from hledger:
  • btcStack (BTC total across all asset accounts)
  • cashUsd (cash balance across as:*:cash accounts)
  • creditBalance (total liabilities li:*)
  • monthlyIncome (avg monthly income from in:* last 3 months)
  • monthlyNeeds (avg monthly from ex:*:need last 3 months)
  • monthlyWants (avg monthly from ex:*:want last 3 months)
  • Use the hledger binary (same approach as Omni/Agent/Tools/Hledger.hs) or hledger-lib if already a dep
  • Caching required: journals are large. Cache results in-process (IORef or MVar with TTL ~5 min) so repeated page loads do not re-run hledger each time.

2. New route in Omni/Fund/Web.hs: GET /fund/live

  • Returns JSON with live metrics (same shape as the 'defaults' object)
  • The Lucid page should fetch /fund/live on load (via fetch()) and merge into defaults, overriding only keys present in the response
  • If /fund/live fails or is slow, fall back to hardcoded defaults gracefully

3. Routing: The Fund app is wired into the main Omni web server. Check Omni/Web.hs or the main app entry point for route composition and add /fund/live there.

Key files

  • Omni/Fund/Web.hs -- the dashboard server (main target)
  • Omni/Agent/Tools/Hledger.hs -- existing hledger shell-out utilities to reference/reuse
  • ~/fund/ledger.journal -- root journal file (pass to hledger -f)
  • Omni/Web.hs or Main.hs -- check for route composition

Notes

  • Account naming: ex (expenses), as (assets), li (liabilities), in (income)
  • BTC amounts may be in BTC or sats depending on account; normalize to BTC
  • Shell out to hledger binary rather than linking hledger-lib unless already a dep
  • The caching layer is important -- hledger with many journals can take 1-2 seconds

Timeline (56)

💬[human]1 month ago

To start port over the dashboard view ~/fund/bin/hledger-overview.hs

The main goal is to reproduce that TUI dashboard in a web app, and then later add the model and projections that we already have in Omni/Fund

🔄[system]Open → InProgress1 month ago
🔄[system]InProgress → Open1 month ago
💬[system]1 month ago

Pipeline: agent died without producing a commit, resetting

🔄[system]Open → InProgress1 month ago
🔄[human]InProgress → Open1 month ago
💬[human]1 month ago

Pipeline scheduler: failed to spawn agentd run

🔄[system]Open → InProgress1 month ago
🔄[human]InProgress → Open1 month ago
💬[human]1 month ago

Pipeline scheduler: failed to spawn agentd run

🔄[system]Open → InProgress1 month ago
💬[human]1 month ago

Pipeline scheduler: started run=pipeline-omni-fund-web-hs-t-654-1771554726 domain=Omni/Fund/Web.hs

🔄[human]InProgress → Open1 month ago
💬[human]1 month ago

Pipeline scheduler reset stale run pipeline-omni-fund-web-hs-t-654-1771554726 (agentd reported running but no active container). Returning task to Open for requeue.

🔄[system]Open → InProgress1 month ago
💬[human]1 month ago

Pipeline scheduler: started run=pipeline-omni-fund-web-hs-t-654-1771559702 domain=Omni/Fund/Web.hs

🔄[human]InProgress → Review1 month ago
💬[human]1 month ago

Pipeline scheduler: run=pipeline-omni-fund-web-hs-t-654-1771559702 domain=Omni/Fund/Web.hs status=failed cost=0c error=OAuth login failed: OAuth login requires an interactive terminal. Please run this command in a terminal session, not in headless mode. (fund-spend=failed)

🔄[human]Review → Open1 month ago
💬[human]1 month ago

Reopened: prior Review transition came from pipeline auth failure, not successful task execution. Re-queued for normal processing.

🔄[system]Open → InProgress1 month ago
💬[human]1 month ago

Pipeline scheduler: started run=pipeline-omni-fund-web-hs-t-654-1771560025 domain=Omni/Fund/Web.hs

🔄[human]InProgress → Review1 month ago
💬[human]1 month ago

Pipeline scheduler: run=pipeline-omni-fund-web-hs-t-654-1771560025 domain=Omni/Fund/Web.hs status=done cost=74c (fund-spend=failed)

🔄[human]Review → InProgress1 month ago
💬[human]1 month ago

Restarted with table-first web port of hledger overview. Added Omni/Fund/Overview.hs (runs ~/fund/bin/hledger-overview.hs via nix-shell, strips ANSI, parses sections/rows, 5m cache + stale fallback), Omni/Fund/OverviewWeb.hs (renders parsed output as HTML tables), and routed Omni/Web/Core.hs so /fund serves overview tables while /fund/model preserves existing chart dashboard. Verified with bild Omni/Web.hs and curl 200 on /fund + /fund/model.

💬[human]1 month ago

Deployed current Fund table-first port via Omni/Ide/ship.sh Omni/Web.hs. Ship succeeded and updated web service manifest; deployer message indicates pickup in <5 min.

💬[human]1 month ago

Added /fund to shared top navigation in Omni/Web/Nav.hs and set Fund overview page active nav state (Omni/Fund/OverviewWeb.hs uses sharedShell active='fund'). Deployed via Omni/Ide/ship.sh Omni/Web.hs; web service now running new store path and /fund HTML includes nav href '/fund'.

💬[human]1 month ago

Implemented phase-1 Fund IA redesign: /fund (overview), /fund/balances, /fund/metrics, /fund/investments, /fund/raw with Fund subnav and KPI cards. Added lightweight Chart.js visualizations per page (allocation doughnut, expenses trend line, custody doughnut) while keeping full section tables for trust/debuggability. Deployed via ship.sh; verified web.service running new store path and all routes return 200 including /fund/model.

💬[human]1 month ago

Implemented typed snapshot API surface and wired routing live. Added Omni/Fund/Snapshot.hs (typed JSON snapshot from parsed overview), exposed /fund/snapshot in Omni/Fund/OverviewWeb.hs, and updated Omni/Fund/Web.hs router so /fund/* serves OverviewWeb while /fund/model keeps the existing stress model. Fixed clean-build blockers in Omni/Newsreader/Extractor.hs (User-Agent bytes, signature/shadowing/import warnings) and Omni/Newsreader/Ingest.hs (Alpha bind operator). Verified typecheck/lint, built with bild Omni/Web.hs, deployed via Omni/Ide/ship.sh Omni/Web.hs. Live checks on :8079: /fund 200 (overview), /fund/model 200 (model), /fund/snapshot 200 application/json.

💬[human]1 month ago

Investigated report of stale/old Fund web behavior. Root cause: live and ava worktrees were diverged for web/fund code (ava missing Omni/Web.hs and new Omni/Fund/{Overview,OverviewWeb,Snapshot}.hs; Omni/Fund/Web.hs still old single-page model handler). Synced those files from live -> /home/ben/omni/ava, then deployed web from ava worktree using CODEROOT=/home/ben/omni/ava Omni/Ide/ship.sh Omni/Web.hs. Deployer rolled web.service to /nix/store/7cd4r7zbfmh6vff8i89xhgbdji1icra3-web/bin/web. Verified locally on :8079: /fund title=Fund Overview, /fund/model title=Fund Model — Stress Dashboard, /fund/snapshot=200 application/json with keys [fetchedAt,isStale,summary,charts,sections]. Filed follow-up t-684 for durable deploy/worktree drift guardrails.

💬[human]1 month ago

Addressed user nits and redeployed from ava worktree. Changes: (1) /fund 'updated' timestamp now renders in Eastern time using Omni.Time.formatEastern (e.g. 2026-02-23 22:56:33 ET) in Omni/Fund/OverviewWeb.hs. (2) Added top-nav /fund link in Omni/Web/Style.hs sharedNav. Deployed with CODEROOT=/home/ben/omni/ava Omni/Ide/ship.sh Omni/Web.hs; web.service now running /nix/store/lzgplll8icp9h90js34r9jsiv703qbkl-web/bin/web --port 8079. Verified via ts.net endpoint that updated ET timestamp and fund nav link are present. Filed follow-up t-686 for reported /fund backend slowness investigation.

💬[human]1 month ago

Completed the 4 requested next steps and redeployed web. Implementation details: (1) Added Omni/Fund/Ledger.hs typed live backend using hledger with metrics btcStack/cashUsd/creditBalanceUsd/monthlyIncomeUsd/monthlyNeedsUsd/monthlyWantsUsd, plus 5m in-process cache and stale-cache background refresh. (2) Added /fund/live route in Omni/Fund/Web.hs returning typed JSON with metadata + metrics + model override defaults, including top-level default-compatible keys (btcStack,cash,salaryNet,expensesNeeds,expensesWants,creditBalance). (3) Updated /fund/model client JS to fetch /fund/live asynchronously on load and merge override keys into DEFAULTS with graceful fallback. (4) Latency work: overview and live paths now refresh in background when cache is stale instead of blocking requests; live backend runs all hledger queries in one nix-shell invocation; model render no longer waits on live fetch. Also fixed a runtime regression discovered during rollout (GHC interface mismatch when running hledger-overview under service env) by sanitizing GHC_PACKAGE_PATH/GHC_ENVIRONMENT in Omni/Fund/Overview.hs. Deployed from ava worktree with CODEROOT and verified web.service path /nix/store/2zalghi5minxhwglw6w3829f9avzsxg6-web/bin/web. Live checks (ts.net and localhost): /fund 200, /fund/model 200, /fund/live 200 JSON, /fund/snapshot 200; /fund shows ET timestamp and fund top-nav item.

🔄[human]InProgress → Review1 month ago
💬[human]1 week ago

Ava verified: last comment from human author documents shipping/deployment with concrete evidence. Moving to Verified.

🔄[human]Review → Verified1 week ago