Architecture Evolution¶

// Theme context with persistence
const ThemeProvider = ({ children }) => {
  const [theme, setTheme] = useState<'light' | 'dark' | 'system'>(() => {
    return localStorage.getItem('claude-mem-theme') || 'system';
  });

  useEffect(() => {
    localStorage.setItem('claude-mem-theme', theme);
  }, [theme]);

  return (
    <ThemeContext.Provider value={{ theme, setTheme }}>
      {children}
    </ThemeContext.Provider>
  );
};

GET /                    # Serves viewer HTML
GET /stream              # SSE real-time updates
GET /api/prompts         # Paginated user prompts
GET /api/observations    # Paginated observations
GET /api/summaries       # Paginated session summaries
GET /api/stats           # Database statistics
GET /api/settings        # User settings
POST /api/settings       # Save settings

src/ui/viewer/
├── components/
│   ├── Header.tsx          # Navigation + stats
│   ├── Sidebar.tsx         # Project filter
│   ├── Feed.tsx            # Infinite scroll
│   └── cards/
│       ├── ObservationCard.tsx
│       ├── PromptCard.tsx
│       ├── SummaryCard.tsx
│       └── SkeletonCard.tsx
├── hooks/
│   ├── useSSE.ts           # Real-time events
│   ├── usePagination.ts    # Infinite scroll
│   ├── useSettings.ts      # Persistence
│   └── useStats.ts         # Statistics
└── utils/
    ├── merge.ts            # Data deduplication
    └── format.ts           # Display formatting

// esbuild bundles everything into single HTML file
esbuild.build({
  entryPoints: ['src/ui/viewer/index.tsx'],
  bundle: true,
  outfile: 'plugin/ui/viewer.html',
  loader: { '.tsx': 'tsx', '.woff2': 'dataurl' },
  define: { 'process.env.NODE_ENV': '"production"' },
});

// Check version marker before installing
const currentVersion = getPackageVersion();
const installedVersion = readFileSync('.install-version', 'utf-8');

if (currentVersion !== installedVersion) {
  // Only install if version changed
  await runNpmInstall();
  writeFileSync('.install-version', currentVersion);
}

// Health check endpoint
app.get('/health', (req, res) => {
  res.json({
    status: 'ok',
    uptime: process.uptime(),
    port: WORKER_PORT,
    memory: process.memoryUsage(),
  });
});

// Smart worker startup
async function ensureWorkerHealthy() {
  const healthy = await isWorkerHealthy(1000);
  if (!healthy) {
    await startWorker();
    await waitForWorkerHealth(10000);
  }
}

┌─────────────────────────────────────────────────────────┐
│                    HYBRID SEARCH                         │
│                                                          │
│  Text Query → SQLite FTS5 (keyword matching)            │
│                      ↓                                   │
│            Chroma Vector Search (semantic)               │
│                      ↓                                   │
│              Merge + Re-rank Results                     │
└─────────────────────────────────────────────────────────┘

// Chroma-backed semantic search
search_observations({
  query: "authentication bug",
  useSemanticSearch: true  // Uses Chroma
});

// Falls back to FTS5 if Chroma unavailable

9+ MCP tools registered at session start:
- search_observations
- find_by_type
- find_by_file
- find_by_concept
- get_recent_context
- get_observation
- get_session
- get_prompt
- help

Problems:
- Overlapping operations (search_observations vs find_by_type)
- Complex parameter schemas (~2,500 tokens in tool definitions)
- No built-in workflow guidance
- High cognitive load for Claude (which tool to use?)
- Code size: ~2,718 lines in mcp-server.ts

4 MCP tools following 3-layer workflow:

1. important_workflow - Workflow documentation (always visible)
   "3-LAYER WORKFLOW (ALWAYS FOLLOW):
    1. search(query) → Get index with IDs
    2. timeline(anchor=ID) → Get context
    3. get_observations([IDs]) → Fetch details
    NEVER fetch full details without filtering first."

2. search - Layer 1: Get index with IDs (~50-100 tokens/result)
3. timeline - Layer 2: Get chronological context
4. get_observations - Layer 3: Fetch full details (~500-1,000 tokens/result)

Benefits:
- Progressive disclosure enforced by tool structure
- No overlapping operations
- Simple schemas (additionalProperties: true)
- Clear workflow pattern
- Code size: ~312 lines in mcp-server.ts (88% reduction)
- ~10x token savings

// Complex parameter schemas
{
  name: "search_observations",
  inputSchema: {
    type: "object",
    properties: {
      query: { type: "string", description: "..." },
      type: { type: "array", items: { enum: [...] } },
      format: { enum: ["index", "full"] },
      limit: { type: "number", minimum: 1, maximum: 100 },
      // ... many more parameters
    }
  }
}

// Simple schemas with workflow guidance
{
  name: "search",
  description: "Step 1: Search memory. Returns index with IDs.",
  inputSchema: {
    type: "object",
    properties: {},
    additionalProperties: true  // Accept any parameters
  }
}

Traditional: Fetch 20 observations upfront
→ 10,000-20,000 tokens
→ Only 2 observations relevant (90% waste)

3-Layer Workflow:
→ search (20 results): ~1,000-2,000 tokens
→ timeline around relevant IDs: ~300-600 tokens
→ Review context, select 3 relevant IDs
→ get_observations (3 IDs): ~1,500-3,000 tokens
→ Total: ~2,800-5,600 tokens

PostToolUse Hook → Save raw tool outputs → Retrieve everything on startup

SessionStart loaded:
- 150 file read operations
- 80 grep searches
- 45 bash commands
- Total: ~35,000 tokens
- Relevant to current task: ~500 tokens (1.4%)

PostToolUse Hook → Queue observation → SDK Worker → AI compression → Store insights

❌ Old: Load 50 observations (8,500 tokens)
✅ New: Show index of 50 observations (800 tokens)
        Agent fetches 2-3 relevant ones (300 tokens)
        Total: 1,100 tokens vs 8,500 tokens

// ❌ Wrong assumption
UserPromptSubmit → Capture session ID once → Use forever

// ✅ Actual behavior
Turn 1: session_abc123
Turn 2: session_def456
Turn 3: session_ghi789

// Capture from system init message
for await (const msg of response) {
  if (msg.type === 'system' && msg.subtype === 'init') {
    sdkSessionId = msg.session_id;
    await updateSessionId(sessionId, sdkSessionId);
  }
}

// ❌ Aggressive: Kill worker immediately
SessionEnd → DELETE /worker/session → Worker stops

// ✅ Graceful: Let worker finish
SessionEnd → Mark session complete → Worker finishes → Exit naturally

// v3: Aggressive
async function sessionEnd(sessionId: string) {
  await fetch(`http://localhost:37777/sessions/${sessionId}`, {
    method: 'DELETE'
  });
}

// v4: Graceful
async function sessionEnd(sessionId: string) {
  await db.run(
    'UPDATE sdk_sessions SET completed_at = ? WHERE id = ?',
    [Date.now(), sessionId]
  );
}

Claude Code session → Create SDK session per observation → 100+ SDK sessions

Claude Code session → ONE long-running SDK session → Streaming input

// ✅ Streaming Input Mode
async function* messageGenerator(): AsyncIterable<UserMessage> {
  // Initial prompt
  yield {
    role: "user",
    content: "You are a memory assistant..."
  };

  // Then continuously yield observations
  while (session.status === 'active') {
    const observations = await pollQueue();
    for (const obs of observations) {
      yield {
        role: "user",
        content: formatObservation(obs)
      };
    }
    await sleep(1000);
  }
}

const response = query({
  prompt: messageGenerator(),
  options: { maxTurns: 1000 }
});

┌─────────────────────────────────────────────────────────┐
│              CLAUDE CODE SESSION                         │
│  User → Claude → Tools (Read, Edit, Write, Bash)        │
│                    ↓                                     │
│              PostToolUse Hook                            │
│              (queues observation)                        │
└─────────────────────────────────────────────────────────┘
                     ↓ SQLite queue
┌─────────────────────────────────────────────────────────┐
│              SDK WORKER PROCESS                          │
│  ONE streaming session per Claude Code session          │
│                                                          │
│  AsyncIterable<UserMessage>                             │
│    → Yields observations from queue                     │
│    → SDK compresses via AI                              │
│    → Parses XML responses                               │
│    → Stores in database                                 │
└─────────────────────────────────────────────────────────┘
                     ↓ SQLite storage
┌─────────────────────────────────────────────────────────┐
│              NEXT SESSION                                │
│  SessionStart Hook                                       │
│    → Queries database                                    │
│    → Returns progressive disclosure index               │
│    → Agent fetches details via MCP                      │
└─────────────────────────────────────────────────────────┘

-- Simple, flat structure
CREATE TABLE observations (
  id INTEGER PRIMARY KEY,
  session_id TEXT,
  text TEXT,
  created_at INTEGER
);

-- Rich, structured schema
CREATE TABLE observations (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  session_id TEXT NOT NULL,
  project TEXT NOT NULL,

  -- Progressive disclosure metadata
  title TEXT NOT NULL,
  subtitle TEXT,
  type TEXT NOT NULL,  -- decision, bugfix, feature, etc.

  -- Content
  narrative TEXT NOT NULL,
  facts TEXT,  -- JSON array

  -- Searchability
  concepts TEXT,  -- JSON array of tags
  files_read TEXT,  -- JSON array
  files_modified TEXT,  -- JSON array

  -- Timestamps
  created_at TEXT NOT NULL,
  created_at_epoch INTEGER NOT NULL,

  FOREIGN KEY(session_id) REFERENCES sdk_sessions(id)
);

-- FTS5 for full-text search
CREATE VIRTUAL TABLE observations_fts USING fts5(
  title, subtitle, narrative, facts, concepts,
  content=observations
);

-- Auto-sync triggers
CREATE TRIGGER observations_ai AFTER INSERT ON observations BEGIN
  INSERT INTO observations_fts(rowid, title, subtitle, narrative, facts, concepts)
  VALUES (new.id, new.title, new.subtitle, new.narrative, new.facts, new.concepts);
END;

// Multiple short SDK sessions
app.post('/process', async (req, res) => {
  const response = await query({
    prompt: buildPrompt(req.body),
    options: { maxTurns: 1 }
  });

  for await (const msg of response) {
    // Process single observation
  }

  res.json({ success: true });
});

// ONE long-running SDK session
async function runWorker(sessionId: string) {
  const response = query({
    prompt: messageGenerator(),  // AsyncIterable
    options: { maxTurns: 1000 }
  });

  for await (const msg of response) {
    if (msg.type === 'text') {
      parseObservations(msg.content);
      parseSummaries(msg.content);
    }
  }
}

# Hook output contained:
npm WARN deprecated ...
npm WARN deprecated ...
{"hookSpecificOutput": {"additionalContext": "..."}}

{
  "command": "npm install --loglevel=silent && node context-hook.js"
}

#!/usr/bin/env node
#!/usr/bin/env node  // ← Duplicate!

// Rest of code...

// Remove shebangs from source files
// Let esbuild add them during build

// ❌ Vulnerable
const results = db.query(
  `SELECT * FROM observations_fts WHERE observations_fts MATCH '${userQuery}'`
);

userQuery = "'; DROP TABLE observations; --"

// ✅ Safe: Use parameterized queries
const results = db.query(
  'SELECT * FROM observations_fts WHERE observations_fts MATCH ?',
  [userQuery]
);

INSERT INTO sdk_sessions (claude_session_id, user_prompt, ...)
VALUES ('abc123', NULL, ...)  -- ❌ user_prompt is NOT NULL

// Allow NULL user_prompts
user_prompt: input.prompt ?? null

-- Before
user_prompt TEXT NOT NULL

-- After
user_prompt TEXT  -- Nullable

for (const obs of observations) {
  db.run(`INSERT INTO observations (...) VALUES (?, ?, ...)`, [obs.id, obs.text, ...]);
}

const stmt = db.prepare(`INSERT INTO observations (...) VALUES (?, ?, ...)`);
for (const obs of observations) {
  stmt.run([obs.id, obs.text, ...]);
}
stmt.finalize();

// Manual full-text search
const results = db.query(
  `SELECT * FROM observations WHERE text LIKE '%${query}%'`
);

// FTS5 virtual table
const results = db.query(
  `SELECT * FROM observations_fts WHERE observations_fts MATCH ?`,
  [query]
);

// Always return full observations
search_observations({ query: "hooks" });
// Returns: 5,000 tokens

// Default to index format
search_observations({ query: "hooks", format: "index" });
// Returns: 200 tokens

// Fetch full only when needed
search_observations({ query: "hooks", format: "full", limit: 1 });
// Returns: 150 tokens

SessionStart({ source: "startup" }):
  → Show last 10 sessions (normal)

SessionStart({ source: "resume" }):
  → Show only current session (minimal)

SessionStart({ source: "compact" }):
  → Show last 20 sessions (comprehensive)

// Use embeddings to pre-sort index by semantic relevance
search_observations({
  query: "authentication bug",
  sort: "relevance"  // Based on embeddings
});

// Cross-project pattern recognition
search_observations({
  query: "API rate limiting",
  projects: ["api-gateway", "user-service", "billing-service"]
});

// Team-shared observations (optional)
createObservation({
  title: "Rate limit: 100 req/min",
  scope: "team"  // vs "user"
});

cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem-v3-backup.db

cd ~/.claude/plugins/marketplaces/thedotmack
git pull

/plugin update claude-mem

# Start Claude Code
claude

# Check that context is injected
# (Should see progressive disclosure index with v5 viewer link)

# Open viewer UI (v5.1.0+)
open http://localhost:37777

# Submit a prompt and watch real-time updates in viewer

# View memory stream in browser (v5.1.0+)
open http://localhost:37777

# Toggle theme (v5.1.2+)
# Click theme button in viewer header

# Check worker health
npm run worker:status
curl http://localhost:37777/health