readyblockedneeds helpallepicskb

Bug: ship/push may report manifest update without persisting

t-690·WorkTask·
·
·
Created1 month ago·Updated1 month ago·pipeline runs →

Dependencies

Description

Edit

ship.sh/push.sh reported successful manifest update for web, but S3 manifest remained on old storePath until a manual deploy-manifest update command was run. deployer.timer was healthy; issue appears in manifest update/push path (possibly silent S3 put failure).

Timeline (2)

💬[human]1 month ago

Correction after deeper investigation: deployer.service being inactive(dead) is expected (oneshot timer unit), not the root issue. deployer.timer runs every ~5m and logs confirm it was healthy.

Observed actual failure mode:

  • ship.sh run reported success and 'updating manifest', but manifest generation and web artifact storePath remained unchanged in S3 (still old path).
  • Verified with deploy-manifest show: web artifact stayed at /nix/store/ri865...
  • Manual command succeeded immediately:

/home/ben/omni/ava/_/nix/Omni/Deploy/Manifest.hs/bin/deploy-manifest update web /nix/store/vd77... 7977e032

  • After manual update, deployer.timer picked up change and rolled web.service to /nix/store/vd77... automatically.

So current actionable bug is: push/ship can report successful manifest update while manifest may remain unchanged (possible silent S3 put failure path).