db123/DivaCode
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: restore detail in plan.md — schema SQL, table summary, principles, pgvector future, enriched phase descriptions

Browse Source
This commit is contained in:
db123-ai
2026-06-10 01:08:08 +03:30
parent cd37d01cae
commit 9903c7b12b
1 changed files with 239 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

321
plan.md
View File

@@ -2,6 +2,7 @@
**Language:** Go **Language:** Go
**Inference:** llama.cpp (HTTP server), OpenAI-compatible API planned **Inference:** llama.cpp (HTTP server), OpenAI-compatible API planned
**Storage:** SQLite (primary), PostgreSQL + pgvector (future)
**Inspiration:** OpenHer, Agent Diva, nanobot **Inspiration:** OpenHer, Agent Diva, nanobot
--- ---
@@ -10,6 +11,7 @@
**Builds:** yes **Builds:** yes
**Runs:** yes (requires llama.cpp on :8080) **Runs:** yes (requires llama.cpp on :8080)
**Known issues:** **Known issues:**
- Prompt template is raw text — model dumps reasoning/self-correction meta-text instead of just the response - Prompt template is raw text — model dumps reasoning/self-correction meta-text instead of just the response
@@ -32,6 +34,9 @@ User
┌─────┴──────────────────────────────┐ ┌─────┴──────────────────────────────┐
│ Core Agent Loop │ │ Core Agent Loop │
│ (internal/agent/) │ │ (internal/agent/) │
│ - Message routing │
│ - Tool invocation │
│ - Context management │
└─────┬──────────────────────────────┘ └─────┬──────────────────────────────┘
│ │
┌─────┴──────────────────────────────┐ ┌─────┴──────────────────────────────┐
@@ -47,7 +52,9 @@ User
┌─────┴──────────────────────────────┐ ┌─────┴──────────────────────────────┐
│ Memory Layer │ │ Memory Layer │
│ (internal/memory/) │ │ (internal/memory/) │
│ ├─ RAG store (keywords → vectors) │ │ ├─ Short-term (ephemeral context) │
│ ├─ Long-term (SQLite → pgvector) │
│ ├─ Episodic / Diary (sqlite) │
│ └─ Self-memory (promises,strats) │ │ └─ Self-memory (promises,strats) │
└─────┬──────────────────────────────┘ └─────┬──────────────────────────────┘
│ │
@@ -61,6 +68,10 @@ Skills / Tools:
├─ web_fetch ├─ web_fetch
├─ web_search (stub) ├─ web_search (stub)
└─ MCP protocol (future) └─ MCP protocol (future)
├─ Obsidian vault
├─ Gitea
├─ Actual Budget
└─ Custom MCP tools
``` ```
--- ---
@@ -76,6 +87,8 @@ Skills / Tools:
- [x] `internal/llama/client.go` — HTTP client for `POST /completion`, `POST /embedding`, `GET /health` - [x] `internal/llama/client.go` — HTTP client for `POST /completion`, `POST /embedding`, `GET /health`
- [x] `.env.example`, `Makefile`, `.gitignore`, `Dockerfile`, `docker-compose.yml` - [x] `.env.example`, `Makefile`, `.gitignore`, `Dockerfile`, `docker-compose.yml`
- [x] `AGENTS.md` with architecture, commands, conventions - [x] `AGENTS.md` with architecture, commands, conventions
- [x] `pkg/types/types.go` — Message, Role, Mood, ToolCall types
- [x] `pkg/personality/traits.go` — TraitValue, RelationshipMetrics, DiaryEntry, Observation types
--- ---
@@ -83,173 +96,313 @@ Skills / Tools:
**Status: COMPLETE** (minor issues remain) **Status: COMPLETE** (minor issues remain)
- [x] `internal/agent/agent.go` — message → memory search → prompt → llama → save → respond - [x] `internal/agent/agent.go` — message loop: receive → memory search → prompt → llama → save → respond
- [x] `internal/agent/scheduler.go` — tick every 15m, check mood/diary/auto-message - [x] `internal/agent/scheduler.go` — background goroutine, configurable tick (default 15m), mood check, diary trigger, autonomous message queue
- [x] `cmd/divad/main.go` — wires everything: config, DB, llama, TUI, scheduler, Matrix, API - [x] `cmd/divad/main.go` — wires everything: config, DB, llama health, TUI, scheduler, Matrix, API
- [x] Conversation persistence to SQLite (`conversations`, `messages` tables) - [x] Conversation persistence to SQLite (`conversations`, `messages` tables with indexes)
- [x] Bubble Tea TUI with viewport + input bar - [x] Bubble Tea TUI with viewport + input bar, Ctrl+C/Ctrl+Q quit
- [ ] Prompt template is wrong — uses raw headers, no chat format. Model outputs meta-text - [x] HTTP API (std lib `net/http`): `GET /health`, `POST /v1/chat`, `POST /v1/conversations`
- [ ] Prompt template is raw headers — model outputs self-correction/next-action meta-text instead of just the response
- [ ] No token tracking — context window grows unbounded - [ ] No token tracking — context window grows unbounded
- [ ] TUI doesn't stream tokens (waits for full completion) - [ ] TUI doesn't stream tokens (waits for full completion)
**Schema (`internal/storage/migrations.go`):**
```sql
CREATE TABLE conversations (
id TEXT PRIMARY KEY,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE messages (
id TEXT PRIMARY KEY,
conversation_id TEXT NOT NULL REFERENCES conversations(id),
role TEXT NOT NULL CHECK(role IN ('user','assistant','system','tool')),
content TEXT NOT NULL,
tool_call TEXT,
token_count INTEGER,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_messages_conversation ON messages(conversation_id);
CREATE TABLE context_state (
conversation_id TEXT PRIMARY KEY REFERENCES conversations(id),
token_count INTEGER NOT NULL DEFAULT 0,
summary TEXT
);
```
--- ---
## Phase 2 — Prompt Templating & Format Fix ## Phase 2 — Prompt Templating & Format Fix
**Goal:** Fix the model output. Swap raw prompt text for a configurable chat template so the model only returns the assistant response. **Goal:** Fix model output. Swap raw prompt text for a configurable chat template so the model only returns the assistant response, not reasoning meta-text.
- [ ] `internal/llama/template.go` — chat template formatter: - [ ] `internal/llama/template.go` — chat template formatter:
- ChatML (`<|im_start|>system...<|im_end|>`) - ChatML: `<|im_start|>system\n...<|im_end|>\n<|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n`
- Llama 3 instruct (`<|begin_of_text|><|start_header_id|>system<|end_header_id|>`) - Llama 3 instruct: `<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n...<|eot_id|>`
- Configurable via env `CHAT_TEMPLATE=chatml` (default) - Configurable via env `CHAT_TEMPLATE=chatml` (default)
- [ ] Refactor `companion/promptbuilder.go` to output structured segments, then let template.go wrap them - [ ] Refactor `companion/promptbuilder.go` to output structured segments (system context, personality, relationship, conversation history, user message)
- [ ] Strip system prompt of raw formatting — let the template handle structure - [ ] Let template.go wrap those segments in the chosen format
- [ ] Test: "hi" produces just "Hello!" not paragraphs of self-correction - [ ] Strip system prompt of raw formatting headers — let template handle structure
- [ ] Add `stop` tokens per template (`<|im_end|>`, `<|eot_id|>`)
- [ ] **Test:** "hi" produces just "Hello!" not paragraphs of self-correction
--- ---
## Phase 3 — Long-Term Memory (RAG) ## Phase 3 — Long-Term Memory (RAG)
**Goal:** Embedding-based retrieval so the agent recalls across sessions. **Goal:** Embedding-based retrieval so the agent recalls facts across sessions.
- [ ] `internal/memory/embed.go` — call llama.cpp `POST /embedding` to get vectors - [ ] `internal/memory/embed.go` — call llama.cpp `POST /embedding` to generate vectors
- [ ] `internal/memory/store.go` — upgrade `Search()` from keyword `LIKE` to cosine similarity - [ ] SQLite vector storage: store embeddings as BLOB in `memories` table
- [ ] Store embeddings as BLOB in `memories` table - [ ] `internal/memory/store.go` — upgrade `Search()` from keyword `LIKE` to cosine similarity scan
- [ ] Temporal decay — older memories scored lower - [ ] Temporal decay — older memories scored lower (weight = importance \* recency_factor)
- [ ] Importance scoring — "remember this" / emotional content boosts score - [ ] Importance scoring — user flags ("remember this"), emotional content, repeat mentions boost score
- [ ] **Test:** Store info, restart, ask about stored info → recall
```sql
CREATE TABLE memories (
id TEXT PRIMARY KEY,
content TEXT NOT NULL,
embedding BLOB, -- float32 vector
source TEXT, -- 'conversation', 'reflection', 'user_input'
importance REAL DEFAULT 0.5, -- 0.0–1.0
created_at TEXT NOT NULL DEFAULT (datetime('now')),
last_accessed TEXT
);
CREATE INDEX IF NOT EXISTS idx_memories_importance ON memories(importance);
CREATE INDEX IF NOT EXISTS idx_memories_created ON memories(created_at);
```
- [ ] **Future:** Migrate to PostgreSQL + pgvector for production-scale vector search
- [ ] **Test:** Store info, restart agent, ask about stored info → verify recall
--- ---
## Phase 4 — OpenAI-Compatible Adapter ## Phase 4 — OpenAI-Compatible Adapter
**Goal:** Swap inference to any OpenAI-compatible API (OpenAI, vLLM, Ollama, etc.). **Goal:** Swap inference to any OpenAI-compatible API (OpenAI, vLLM, Ollama, Groq, etc.).
- [ ] `internal/llama/provider.go` — interface: - [ ] `internal/llama/provider.go` — interface:
```go
type Provider interface { ```go
Complete(ctx, *Request) (*Response, error) type Provider interface {
Embed(ctx, string) ([]float64, error) Complete(ctx context.Context, req *CompletionRequest) (*CompletionResponse, error)
Embed(ctx context.Context, content string) ([]float64, error)
Health() error Health() error
} }
``` ```
- [ ] Rename `internal/llama/` → `internal/inference/` - [ ] Rename `internal/llama/` → `internal/inference/`
- [ ] `llama.go` — implements Provider (existing code) - [ ] `llama.go` — implements Provider (existing llama.cpp client wrapped)
- [ ] `openai.go` — implements Provider via OpenAI-compatible REST API - [ ] `openai.go` — implements Provider via OpenAI-compatible REST API (`/v1/chat/completions`, `/v1/embeddings`)
- [ ] Config: `INFERENCE_PROVIDER=llama|openai`, `OPENAI_API_KEY`, `OPENAI_BASE_URL` - [ ] Config: `INFERENCE_PROVIDER=llama|openai`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `OPENAI_MODEL`
- [ ] **Test:** Both providers produce same interface, switchable via env - [ ] Both providers produce same interface, switchable via env var
- [ ] **Test:** Same conversation, swap provider, verify behavior is consistent
--- ---
## Phase 5 — Episodic Diary & Reflection ## Phase 5 — Episodic Diary & Reflection
**Status: CODE DONE** — needs integration into the scheduled pipeline. **Goal:** Daily diary entries + periodic self-reflection for narrative continuity. The diary is the emotional continuity layer.
**Status: CODE DONE** — needs LLM integration for real summaries.
- [x] `internal/companion/reflection.go` — `GenerateDiary`, `RecentObservations`, `AddObservation` - [x] `internal/companion/reflection.go` — `GenerateDiary`, `RecentObservations`, `AddObservation`
- [x] `diary_entries` table with date, summary, mood, topics, observations - [x] `diary_entries` table:
- [x] `observations` table with confidence, category, applied flag
- [ ] Connect scheduler's diary trigger to actual LLM-generated summaries (currently writes stub text) ```sql
CREATE TABLE diary_entries (
id TEXT PRIMARY KEY,
date TEXT NOT NULL, -- YYYY-MM-DD
summary TEXT NOT NULL,
mood TEXT,
topics TEXT, -- JSON array
observations TEXT, -- JSON array
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
```
- [x] `observations` table:
```sql
CREATE TABLE observations (
id TEXT PRIMARY KEY,
content TEXT NOT NULL,
confidence REAL DEFAULT 0.5,
category TEXT, -- 'topic_interest', 'interaction_style', 'mood_pattern'
applied INTEGER DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
```
- [ ] Connect scheduler's nightly diary trigger to LLM-generated summaries (currently writes stub text)
- [ ] Reflection → Personality update pipeline: - [ ] Reflection → Personality update pipeline:
- Observations accumulate → periodic analysis → bounded trait drift - Diary entries accumulate → periodic analysis → trait drift candidates
- Max 0.5% change/day, max 5%/month - Approval process: observations → candidates → bounded update
- Max 0.5% trait change/day, max 5%/month — slow evolution feels human, fast feels schizophrenic
- [ ] **Test:** Simulate conversations across multiple "days", verify diary entries, verify trait drift remains within bounds
--- ---
## Phase 6 — MCP Tools & Integrations ## Phase 6 — MCP Tools & Integrations
**Goal:** Extensible tool system for web search, file access, external services.
**Status: PARTIAL** **Status: PARTIAL**
- [x] `internal/tools/registry.go` — register/list/execute tools - [x] `internal/tools/registry.go` — register/list/execute tools with parameter definitions
- [x] `internal/tools/builtin.go` — `web_fetch`, `web_search` (stub) - [x] `internal/tools/builtin.go` — `web_fetch` (working), `web_search` (stub)
- [ ] MCP protocol client (`internal/tools/mcp.go`) — stdio/TCP, list tools, execute, parse results - [ ] MCP protocol client (`internal/tools/mcp.go`):
- [ ] Tool call loop in agent: LLM decides → agent executes → result appended → LLM finalizes - Connect to MCP servers via stdio or TCP
- [ ] Plugins: Obsidian vault, Gitea, Actual Budget - List available tools with parameters
- Execute tool calls, parse results into context
- [ ] Tool call loop in agent:
- LLM decides to call tool → agent executes → result appended to context → LLM generates final response
- [ ] MCP-managed integrations:
- Obsidian vault (via `enquire-mcp` or custom)
- Gitea/Forgejo operations
- Actual Budget (spending tracking, categorization)
- [ ] **Test:** `web_fetch` tool, verify agent can search, summarize, cite sources
--- ---
## Phase 7 — Matrix Integration ## Phase 7 — Matrix Integration
**Goal:** Agent lives as a Matrix bot, accessible via DM. Matrix feels like `@diva:homeserver.tld` instead of `Bot #4321`.
**Status: STUB** **Status: STUB**
- [x] `internal/channel/matrix.go` — config + start stub - [x] `internal/channel/matrix.go` — config + start stub (disabled by default)
- [ ] Full Matrix bot with `mautrix-go`: - [ ] Full Matrix bot with `mautrix-go`:
- Listen for invitations, join rooms - Listen for invitations, join rooms
- Handle DMs with typing indicators - Handle DMs with typing indicators, read receipts
- Message parsing & sending
- Bot identity: `@diva:homeserver.tld` - Bot identity: `@diva:homeserver.tld`
- [ ] Sync TUI and Matrix state - [ ] Config: homeserver URL, bot user token in env
- [ ] **Test:** Send DM, verify response, verify cross-session memory - [ ] TUI and Matrix run concurrently — messages sync between both
- [ ] **Test:** Send DM to bot, verify response, verify cross-session memory
--- ---
## Phase 8 — Self-Memory & Identity ## Phase 8 — Self-Memory & Identity
**Goal:** Agent remembers its own state — promises, strategies, mistakes. "Bidirectional memory."
**Status: CODE DONE** **Status: CODE DONE**
- [x] `internal/memory/self_memory.go` — promises, strategies - [x] `internal/memory/self_memory.go`:
- [x] `agent_promises` table (id, promise, context, fulfilled, deadline)
- [x] `agent_strategies` table (id, situation, strategy, outcome) ```go
- [ ] Wire into Prompt Composer: "You promised to X" context type Promise struct {
- [ ] **Test:** "What did you promise me?" → recall ID string
Promise string
Context string
Fulfilled bool
CreatedAt time.Time
Deadline *time.Time
}
type Strategy struct {
ID string
Situation string
Strategy string
Outcome string
CreatedAt time.Time
}
```
- [x] `agent_promises` table (id, promise, context, fulfilled, created_at, deadline)
- [x] `agent_strategies` table (id, situation, strategy, outcome, created_at)
- [ ] Wire into Prompt Composer: recent promises, relevant strategies added to context
- [ ] **Test:** "What did you promise me yesterday?" → verify recall from self-memory
--- ---
## Phase 9 — Safety Guardrails & Identity Stability ## Phase 9 — Safety Guardrails & Identity Stability
**Goal:** Prevent personality drift, maintain core safety. **Goal:** Prevent personality drift, maintain core safety rules.
- [ ] `internal/companion/safety.go` — core identity (stable) vs state (dynamic) - [ ] `internal/companion/safety.go`:
- [ ] Personality safety: bounded traits, safety overrides personality, no tool access changes - Core Identity (never changes): values, communication style, boundaries, role definition
- [ ] Prompt injection protection: user input wrapped in delimiters, strict template sections - State (dynamic): mood, interests, current projects, relationship opinions
- Separation: identity config (env/read-only) vs state (SQLite/writable)
- [ ] Personality safety rules:
- Traits bounded within configured min/max at all times
- Core safety rules override any personality state
- Permission policies cannot be modified by personality
- No tool access changes via personality evolution
- [ ] Prompt injection protection:
- User messages wrapped in separator tokens
- System prompt templates with strict, non-overridable sections
- Input sanitization at the agent boundary
- [ ] **Test:** Attempt prompt injection, verify guardrails hold
--- ---
## Phase 10 — Polish: Streaming TUI, Token Tracking, Observability ## Phase 10 — Polish: Streaming TUI, Token Tracking, Observability
**Goal:** Production-quality UX. **Goal:** Production-quality UX and operations.
- [ ] Token streaming in TUI (char-by-char from llama.cpp SSE or chunked response) - [ ] Token streaming in TUI — char-by-char from llama.cpp SSE or chunked response
- [ ] Context window manager — track tokens, sliding window when full, auto-summarize old messages - [ ] Context window manager:
- Track total token count per conversation
- Sliding window when context exceeds limit
- Auto-summarize oldest messages to stay within context
- [ ] `POST /v1/chat` streaming via SSE in API mode - [ ] `POST /v1/chat` streaming via SSE in API mode
- [ ] Structured JSON logging via `log/slog` (already done — custom handler with `[LEVEL] [TIME] [FILE:LINE]`)
- [ ] Prometheus metrics (`GET /metrics`) - [ ] Prometheus metrics (`GET /metrics`)
- [ ] Graceful shutdown — drain in-flight requests, save state - [ ] Graceful shutdown — drain in-flight requests, save agent state
- [ ] **Test:** Full restart cycle, verify all state restored
--- ---
## Phase 11 — Future Enhancements (Optional) ## Phase 11 — Future Enhancements (Optional)
- [ ] Voice: whisper.cpp (STT) + piper/XTTS (TTS) - [ ] Voice interface: whisper.cpp (STT) + piper/XTTS (TTS)
- [ ] VRM/Live2D avatar rendering - [ ] VRM/Live2D avatar rendering (like Open-LLM-VTuber, Soul of Waifu)
- [ ] Multi-user with per-user relationship tracking - [ ] Multi-user support with per-user relationship tracking
- [ ] Multi-room Matrix - [ ] Multi-room Matrix support
- [ ] Plugin system - [ ] Plugin system for community-contributed tools
- [ ] WebUI dashboard - [ ] WebUI dashboard (like nanobot's WebUI)
- [ ] Home Assistant integration - [ ] Mobile companion via Matrix (any Matrix client works)
- [ ] Home Assistant integration for physical automation
- [ ] PostgreSQL + pgvector migration for production-scale vector search
--- ---
## Database Schema (15 tables across 2 DBs) ## Database Summary
`divacode.db`: Two separate SQLite databases (clean separation of concerns):
- `conversations`, `messages`, `context_state`, `memories` | Database | Purpose | Tables |
| -------------- | --------------- | ------------------------------------------------------- |
| `divacode.db` | Core agent data | `conversations`, `messages`, `context_state` |
| `divacode.db` | Memory | `memories` |
| `companion.db` | Personality | `personality_traits` |
| `companion.db` | Relationship | `relationship_metrics`, `shared_topics`, `inside_jokes` |
| `companion.db` | Reflection | `diary_entries`, `observations` |
| `companion.db` | Self-memory | `agent_promises`, `agent_strategies` |
`companion.db`: **Future:** `divacode.db` may be replaced by PostgreSQL + pgvector for production deployments with many users or large memory stores.
- `personality_traits`, `relationship_metrics`, `shared_topics`, `inside_jokes`
- `diary_entries`, `observations`
- `agent_promises`, `agent_strategies`
--- ---
## Key Principles ## Key Principles
1. **Smallest loop first** — get message → memory → llm → response working before adding complexity 1. **Build the smallest loop first** — get message → memory → LLM → response → diary → reflection working before adding complexity
2. **Personality is data, not prompt text** — traits in SQLite, composed dynamically 2. **Personality is data, not prompt text** — traits stored in SQLite, composed dynamically by the prompt builder
3. **Memory ≠ Personality** — factual memory separate from relationship/emotional state 3. **Memory ≠ Personality** — factual memory (Engram-style RAG) is separate from relationship/emotional state
4. **Slow evolution** — max 0.5% trait change/day, 5%/month 4. **Slow evolution** — max 0.5% trait change/day, max 5%/month. Fast evolution feels schizophrenic, slow feels human
5. **Core Identity stable** — never allow personality to override safety rules 5. **Core Identity is stable** — identity.yaml (or env config) never changes much; state.json (SQLite) changes constantly
6. **Diary before reflection** — nightly diary is the emotional continuity layer 6. **Diary before reflection** — the nightly diary is the emotional continuity layer, more important than vector search
7. **Autonomous messaging** — scheduler makes it feel alive, not the LLM 7. **Autonomous messaging** — the scheduler is what makes the character feel alive, not the LLM intelligence
8. **No config libs** — std lib `os.Getenv` only 8. **No secrets in code** — all config via env vars, `.env` in `.gitignore`
9. **No HTTP routers** — std lib `net/http` only 9. **No config libraries** — std lib `os.Getenv` only, never `caarlos0/env` or similar
10. **Error wrapping** — `fmt.Errorf("context: %w", err)`, never discard 10. **No HTTP routers** — std lib `net/http` only, no third-party routers (chi, gorilla, etc.)
11. **Error handling** — wrap errors with `fmt.Errorf("context: %w", err)`, never silent discards
12. **Test as you go** — `go test -race ./...` before every push
13. **Output readable code** — explain _why_, never _what_. No comments for self-documenting code