Proactive memory deduplication and linking

t-262·WorkTask·
·
·
Created2 months ago·Updated4 weeks ago

Description

Edit

Detect duplicate memories at store time using embedding similarity. Auto-link or merge duplicates. Surface duplicates for agent review. Could run as background job or at remember-time.

Timeline (6)

🔄[human]Open → InProgress4 weeks ago
💬[human]4 weeks ago

Implementation complete. Commit 1a7ebab.

Key features: 1. Memories with similarity >= 0.95 are merged (content combined, confidence boosted) 2. Memories with similarity 0.85-0.95 are linked bidirectionally via related_ids 3. New StoreResult type exposes dedup outcome (StoredNew, MergedWith, StoredLinked) 4. Schema migration adds related_ids column to existing tables

The API is backward compatible - storeMemoryWithTags still returns Memory. Use storeMemoryWithDedup for explicit dedup result handling.

🔄[human]InProgress → Done4 weeks ago
💬[human]4 weeks ago

Ran one-time deduplication on production memory.db:

  • Before: 109 memories
  • After: 90 memories
  • Deleted: 19 true duplicates (>=0.95 similarity)
  • Linked: 10 related pairs (0.85-0.95 similarity)

Backup at /home/ben/memory.db.bak.20260124-173926