2 Commits

Author	SHA1	Message	Date
Liu Yuan	15a70ac45c	feat(seahorse): implement short-term memory engine (LCM) (#2285) ... * feat(seahorse): implement short-term memory engine of seahorse Add pkg/seahorse/ module implementing a SQLite-backed DAG-based summary hierarchy for context management, ported from lossless-claw's LCM design: - types.go + short_constants.go: core types (Message, Summary, Conversation, ContextItem) and configuration constants (fanout, token targets, thresholds) - migration.go: idempotent DB schema with FTS5 trigram tokenizer for CJK - store.go: full SQLite CRUD (conversations, messages, summaries DAG, context_items with ordinal gap numbering, FTS5 search) - short_engine.go: Engine lifecycle (NewEngine, Ingest, Assemble, Compact), session pattern filtering (ignore/stateless glob→regex compilation), per-session mutex via sync.Map - short_assembler.go: budget-aware context assembly with fresh tail protection (32 messages), oldest-first eviction, summary XML formatting, RebuildContextItems - short_compaction.go: leaf compaction (messages→summary) and condensed compaction (summaries→higher-level summary), 3-level LLM escalation, CompactUntilUnder for emergency overflow - short_retrieval.go: lookupByID, FTS5/LIKE search, recursive expand with token cap - context_seahorse.go: agent.ContextManager adapter, registered as "seahorse", provider↔seahorse message type conversion (ToolCalls, tool_result) * fix(seahorse): correct 3 adapter bugs in context management - TokenCount: use full message (Content+ToolCalls+Media) instead of Content-only - Empty Content: rebuild Content from tool_result Parts when stored empty - Duplicate summaries: summaries only in Summary field, not in History messages - Grep: fix SearchResult.Snippet→Content for summaries - Schema: fix FTS5 SQL uses VIRTUAL TABLE not TEMP TABLE - TestFTS5SQLConstants: verify FTS5 SQL syntax correctness - Test: fix flaky TestCompactLeaf * fix(agent): ingest steering messages into seahorse SQLite Steering messages were only persisted to session JSONL but not ingested into seahorse SQLite, causing them to be missing from context assembly. Added `ts.ingestMessage(turnCtx, al, pm)` call in the steering message injection block alongside the existing JSONL persistence. Test: TestSeahorseSteeringMessageIngested verifies steering messages appear in seahorse SQLite DB after being processed. * fix(seahorse): address 3 blocking bugs from code review - Fix resequenceContextItemsTx scan error handling (store.go:850) Changed `return err` to `return scanErr` to properly propagate scan errors instead of returning nil (which silently corrupts data) - Fix sql.NullString for INTEGER column (store.go:847) Changed `mid` from sql.NullString to sql.NullInt64 since message_id is INTEGER in schema. Removed unnecessary strconv.ParseInt call. - Fix compactCondensed fallback deleting non-candidate items Added ReplaceContextItemsWithSummary method for per-item deletion when candidates are not contiguous in ordinal space. Optimized to use range deletion when candidates are consecutive. * fix(seahorse): pass Budget to Compact for correct condensed threshold Issue #4 from PR review: When Budget was not passed to seahorse.Compact, it defaulted to `tokensBefore * 0.75`, making `tokensBefore > budget` always true and causing condensed compaction to trigger unnecessarily. Changes: - context_seahorse.go: Forward Budget from CompactRequest to CompactInput - loop.go: Pass Budget (ContextWindow) in all 3 Compact calls - Add test verifying condensed is skipped when tokens < threshold - Fix lint issues in store.go and store_test.go * fix(seahorse): add mutex for assembler lazy initialization Issue #5 from PR review: The check-then-create pattern for e.assembler was a data race when multiple goroutines called Assemble() concurrently: if e.assembler == nil { e.assembler = &Assembler{...} } Changes: - Add assemblerMu sync.Mutex to Engine struct - Add initAssemblerOnce() using double-checked locking (same pattern as initCompactionOnce) - Add TestAssemblerLazyInitRace to verify thread-safety * fix(seahorse): handle non-consecutive depths in selectShallowestCondensationCandidate Issue #8 from PR review: the loop iterated depth 0, 1, 2... assuming consecutive keys, but break when key was missing caused deeper depths to never be checked. Fix: collect all existing depth keys, sort, then iterate in order. * fix(seahorse): wrap DeleteMessagesAfterID and appendContextItems in transactions - DeleteMessagesAfterID: wrap all DELETE operations in a transaction for atomicity, remove redundant manual FTS delete (handled by trigger) - appendContextItems: use transaction to fix read-then-write race condition - Add GetMaxOrdinalTx and resolveItemTokenCountTx for transaction-scoped queries - Remove unused resolveItemTokenCount function Fixes PR review issues 6 and 7. * fix(seahorse): derive readable content from Parts and cap CompactUntilUnder iterations - Derive readable content from MessageParts in AddMessageWithParts so FTS5 indexing and summary formatting can access tool call information - formatMessagesForSummary and truncateSummary now fall back to Parts when Content is empty, fixing blank summaries for Part-based messages - Add MaxCompactIterations (20) to prevent CompactUntilUnder infinite loops; exceeded iterations are logged as warnings	2026-04-05 09:05:16 +08:00
is-Xiaoen	26f623ed32	feat(session): integrate JSONL persistence into agent loop (#1170) ... * feat(session): add SessionStore interface and JSONL backend adapter Extract a SessionStore interface from the methods the agent loop uses (AddMessage, GetHistory, SetSummary, TruncateHistory, Save, etc.). Both SessionManager and the new JSONLBackend satisfy this interface, allowing the persistence layer to be swapped transparently. JSONLBackend wraps memory.Store and maps its error-returning API to the fire-and-forget contract that the agent loop expects — write errors are logged, reads return empty defaults on failure. Save() triggers compaction to reclaim space after logical truncation. Part of #1169 * test(session): add JSONLBackend integration tests 8 tests covering the full SessionStore contract through the JSONL backend: message roundtrip, tool calls, summary, truncation with compaction, history replacement, empty sessions, session isolation, and the complete summarization flow (SetSummary → TruncateHistory → Save). Includes compile-time interface satisfaction checks for both SessionManager and JSONLBackend. Part of #1169 * feat(agent): wire JSONL session store into agent loop Replace the concrete *SessionManager field with the SessionStore interface and initialize the JSONL backend by default. Legacy .json session files are auto-migrated on first startup. Falls back to SessionManager if the JSONL store cannot be initialized. The agent loop code (loop.go) requires zero changes — all method calls work identically through the interface. Closes #1169 * fix(session): propagate compact error from Save Save() was swallowing the error returned by Compact and always returning nil. Callers checking Save's return value would never see a compaction failure. Return the error directly so the agent loop can log or handle it as needed. * feat(session): add Close to SessionStore interface Add Close() error to SessionStore so callers can release resources through the interface. JSONLBackend already had Close; this adds a no-op implementation to SessionManager for compatibility. * fix(session): close session stores on shutdown and harden migration - Add Close() to AgentInstance, AgentRegistry, and AgentLoop so JSONL file handles are released during gateway shutdown and CLI exit. - Fall back to SessionManager when migration fails, preventing a split state where some sessions live in JSONL and others remain in JSON. - Add defer agentLoop.Close() in the CLI agent command path. - Document SessionStore interface methods (fire-and-forget contract).	2026-03-10 15:14:09 +08:00