Memory System

Create Omni/Agent/Memory.hs module for cross-agent, multi-user shared memory with vector similarity search.

Context

This is the foundation for a multi-agent system where different agents (Telegram bot, researcher, coder, etc.) share knowledge about users. Example: User tells Telegram bot 'I'm an AI engineer' -> later, researcher agent searching for papers should recall this context.

Design Decisions

• Use sqlite-vss extension for vector similarity search
• Users identified by Telegram ID initially (email added later)
• All memories shared across agents, but private per user (users can't see each other's memories)
• Separate database file: memory.db (not tasks.db)

Schema (memory.db)

CREATE TABLE users (
  id TEXT PRIMARY KEY,              -- UUID
  telegram_id INTEGER UNIQUE,       -- Primary identifier initially
  email TEXT UNIQUE,                -- Added later for email interface
  name TEXT NOT NULL,
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE memories (
  id TEXT PRIMARY KEY,              -- UUID
  user_id TEXT NOT NULL REFERENCES users(id),
  content TEXT NOT NULL,
  embedding BLOB,                   -- float32 vector for sqlite-vss
  source_agent TEXT NOT NULL,       -- 'telegram', 'coder', etc.
  source_session TEXT,              -- Session UUID
  source_context TEXT,              -- How this was learned
  confidence REAL DEFAULT 0.8,
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  last_accessed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  tags TEXT                         -- JSON array
);

CREATE VIRTUAL TABLE memories_vss USING vss0(embedding(1536));
CREATE INDEX idx_memories_user ON memories(user_id);

Types

data User = User
  { userId :: UUID
  , userTelegramId :: Maybe Int64
  , userEmail :: Maybe Text
  , userName :: Text
  , userCreatedAt :: UTCTime
  }

data Memory = Memory
  { memoryId :: UUID
  , memoryUserId :: UUID
  , memoryContent :: Text
  , memoryEmbedding :: Maybe (Vector Float)
  , memorySource :: MemorySource
  , memoryConfidence :: Double
  , memoryCreatedAt :: UTCTime
  , memoryLastAccessedAt :: UTCTime
  , memoryTags :: [Text]
  }

data MemorySource = MemorySource
  { sourceAgent :: Text
  , sourceSession :: Maybe UUID
  , sourceContext :: Text
  }

Core Functions

-- User management
createUser :: Text -> Maybe Int64 -> IO User
getUserByTelegramId :: Int64 -> IO (Maybe User)
getOrCreateUserByTelegramId :: Int64 -> Text -> IO User

-- Memory operations
storeMemory :: UUID -> Text -> MemorySource -> IO Memory
recallMemories :: UUID -> Text -> Int -> IO [Memory]  -- semantic search by user
forgetMemory :: UUID -> IO ()
getAllMemoriesForUser :: UUID -> IO [Memory]

-- Embedding (via Ollama)
embedText :: Text -> IO (Vector Float)

Embedding Integration

• Call Ollama API: POST http://localhost:11434/api/embeddings
• Request body: {"model": "nomic-embed-text", "prompt": "text to embed"}
• Response: {"embedding": [0.1, 0.2, ...]}
• Store as BLOB in memories table
• Use sqlite-vss for similarity search

Agent Integration

Create wrapper for agent loop:

runAgentWithMemory :: User -> AgentConfig -> Text -> IO AgentResult
runAgentWithMemory user config prompt = do
  memories <- recallMemories (userId user) prompt 10
  let memoryContext = formatMemoriesForPrompt memories
  let enhancedPrompt = agentSystemPrompt config <> "\n\n## Known about this user\n" <> memoryContext
  runAgent config { agentSystemPrompt = enhancedPrompt } prompt

Tools for Agents

rememberTool :: UUID -> Tool  -- takes user ID, returns Tool that stores memories
recallTool :: UUID -> Tool    -- takes user ID, returns Tool that queries memories

Files to Create

• Omni/Agent/Memory.hs - main module with types and core functions
• Omni/Agent/Memory/Embedding.hs - Ollama embedding client

Dependencies

• sqlite-vss Nix package (check if available, may need to add to Bild.nix)
• Depends on t-247 (Provider) for Ollama embedding calls

Testing

• bild --test Omni/Agent/Memory.hs
• Test user CRUD
• Test memory storage and retrieval
• Test embedding generation (requires local Ollama with nomic-embed-text)
• Test similarity search

Reference

• See Omni/Agent/PLAN.md section 4.3 for full design
• See Omni/Task/Core.hs for similar SQLite patterns