427 lines
18 KiB
Markdown
427 lines
18 KiB
Markdown
# DivaCode — AI Being Platform Action Plan
|
||
|
||
**Language:** Go
|
||
**Inference:** llama.cpp (HTTP server), OpenAI-compatible API planned
|
||
**Storage:** SQLite (primary), PostgreSQL + pgvector (future)
|
||
**Inspiration:** OpenHer, Agent Diva, nanobot
|
||
|
||
---
|
||
|
||
## Current Status (2026-06-10)
|
||
|
||
**Builds:** yes
|
||
**Runs:** yes (requires llama.cpp on :8080)
|
||
|
||
**Known issues:**
|
||
|
||
- Prompt template is raw text — model dumps reasoning/self-correction meta-text instead of just the response
|
||
- No chat template configurability (ChatML, Llama 3 instruct, etc.)
|
||
- TUI is basic (no streaming, no word-wrap polish)
|
||
- Memory store uses keyword `LIKE` search, not real embeddings
|
||
- OpenAI-compatible adapter not yet built
|
||
- Matrix adapter is a stub
|
||
|
||
---
|
||
|
||
## Architecture Overview
|
||
|
||
```
|
||
User
|
||
│
|
||
├── TUI (Bubble Tea)
|
||
└── Matrix (future)
|
||
│
|
||
┌─────┴──────────────────────────────┐
|
||
│ Core Agent Loop │
|
||
│ (internal/agent/) │
|
||
│ - Message routing │
|
||
│ - Tool invocation │
|
||
│ - Context management │
|
||
└─────┬──────────────────────────────┘
|
||
│
|
||
┌─────┴──────────────────────────────┐
|
||
│ Companion Layer │
|
||
│ (internal/companion/) │
|
||
│ ├─ Personality (5 bounded traits) │
|
||
│ ├─ Relationship (familiarity,trust)│
|
||
│ ├─ Reflection (diary,observations)│
|
||
│ ├─ Mood (6 states + energy) │
|
||
│ └─ Prompt Composer (dynamic ctx) │
|
||
└─────┬──────────────────────────────┘
|
||
│
|
||
┌─────┴──────────────────────────────┐
|
||
│ Memory Layer │
|
||
│ (internal/memory/) │
|
||
│ ├─ Short-term (ephemeral context) │
|
||
│ ├─ Long-term (SQLite → pgvector) │
|
||
│ ├─ Episodic / Diary (sqlite) │
|
||
│ └─ Self-memory (promises,strats) │
|
||
└─────┬──────────────────────────────┘
|
||
│
|
||
┌─────┴──────────────────────────────┐
|
||
│ Inference (internal/llama/) │
|
||
│ ├─ llama.cpp HTTP client │
|
||
│ └─ OpenAI-compatible (planned) │
|
||
└────────────────────────────────────┘
|
||
|
||
Skills / Tools:
|
||
├─ web_fetch
|
||
├─ web_search (stub)
|
||
└─ MCP protocol (future)
|
||
├─ Obsidian vault
|
||
├─ Gitea
|
||
├─ Actual Budget
|
||
└─ Custom MCP tools
|
||
```
|
||
|
||
---
|
||
|
||
## Phase 0 — Foundation & Scaffolding
|
||
|
||
**Status: COMPLETE**
|
||
|
||
- [x] Go module: `git.db123.ir/db123/divacode`
|
||
- [x] Project structure per Holy Code Bible conventions
|
||
- [x] `internal/config/` — std lib `os.Getenv` only
|
||
- [x] `internal/storage/` — `modernc.org/sqlite` (pure Go, no CGO), auto-creates `data/` dir
|
||
- [x] `internal/llama/client.go` — HTTP client for `POST /completion`, `POST /embedding`, `GET /health`
|
||
- [x] `.env.example`, `Makefile`, `.gitignore`, `Dockerfile`, `docker-compose.yml`
|
||
- [x] `AGENTS.md` with architecture, commands, conventions
|
||
- [x] `pkg/types/types.go` — Message, Role, Mood, ToolCall types
|
||
- [x] `pkg/personality/traits.go` — TraitValue, RelationshipMetrics, DiaryEntry, Observation types
|
||
|
||
---
|
||
|
||
## Phase 1 — Core Agent Loop
|
||
|
||
**Status: COMPLETE** (minor issues remain)
|
||
|
||
- [x] `internal/agent/agent.go` — message loop: receive → memory search → prompt → llama → save → respond
|
||
- [x] `internal/agent/scheduler.go` — background goroutine, configurable tick (default 15m), mood check, diary trigger, autonomous message queue
|
||
- [x] `cmd/divad/main.go` — wires everything: config, DB, llama health, TUI, scheduler, Matrix, API
|
||
- [x] Conversation persistence to SQLite (`conversations`, `messages` tables with indexes)
|
||
- [x] Bubble Tea TUI with viewport + input bar, Ctrl+C/Ctrl+Q quit
|
||
- [x] HTTP API (std lib `net/http`): `GET /health`, `POST /v1/chat`, `POST /v1/conversations`
|
||
- [ ] Prompt template is raw headers — model outputs self-correction/next-action meta-text instead of just the response
|
||
- [ ] No token tracking — context window grows unbounded
|
||
- [ ] TUI doesn't stream tokens (waits for full completion)
|
||
|
||
**Schema (`internal/storage/migrations.go`):**
|
||
|
||
```sql
|
||
CREATE TABLE conversations (
|
||
id TEXT PRIMARY KEY,
|
||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||
);
|
||
|
||
CREATE TABLE messages (
|
||
id TEXT PRIMARY KEY,
|
||
conversation_id TEXT NOT NULL REFERENCES conversations(id),
|
||
role TEXT NOT NULL CHECK(role IN ('user','assistant','system','tool')),
|
||
content TEXT NOT NULL,
|
||
tool_call TEXT,
|
||
token_count INTEGER,
|
||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||
);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_messages_conversation ON messages(conversation_id);
|
||
|
||
CREATE TABLE context_state (
|
||
conversation_id TEXT PRIMARY KEY REFERENCES conversations(id),
|
||
token_count INTEGER NOT NULL DEFAULT 0,
|
||
summary TEXT
|
||
);
|
||
```
|
||
|
||
---
|
||
|
||
## Phase 2 — Prompt Templating & Reasoning-Aware TUI
|
||
|
||
**Goal:** Fix model output with proper chat templates. Parse reasoning/thoughts from the response and render them beautifully in the TUI — dimmed, collapsible, like OpenCode.
|
||
|
||
### Chat Template Engine
|
||
|
||
- [ ] `internal/llama/template.go` — chat template formatter:
|
||
- ChatML: `<|im_start|>system\n...<|im_end|>\n<|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n`
|
||
- Llama 3 instruct: `<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n...<|eot_id|>`
|
||
- DeepSeek R1: with `...` support
|
||
- Configurable via env `CHAT_TEMPLATE=chatml` (default)
|
||
- [ ] Refactor `companion/promptbuilder.go` to output structured segments (system context, personality, relationship, conversation history, user message)
|
||
- [ ] Let template.go wrap those segments in the chosen format
|
||
- [ ] Add `stop` tokens per template (`<|im_end|>`, `<|eot_id|>`)
|
||
- [ ] **Test:** "hi" produces just the response, no meta-text
|
||
|
||
### Reasoning Parser
|
||
|
||
- [ ] `internal/agent/reasoning.go` — parse `...`, `...`, or custom tags from model output
|
||
- [ ] Split response into `Reasoning` (thoughts) and `Content` (final answer)
|
||
- [ ] Store both separately in the `messages` table
|
||
- [ ] Feed only `Content` back into context window (reasoning is discarded after display)
|
||
|
||
### TUI Enhancement
|
||
|
||
- [ ] Render reasoning in a collapsible section:
|
||
- Header: "💭 (or `...`) with reasoning snippet
|
||
- On click/expand: show full reasoning in dimmed/italic style (`lipgloss.Color("#6B7280")`)
|
||
- [ ] Render final answer in normal assistant style (green/bold)
|
||
- [ ] Reasoning hidden by default, toggle with a keybinding (e.g., `tab`)
|
||
- [ ] **Test:** Send a prompt that triggers reasoning, verify collapsible display
|
||
|
||
---
|
||
|
||
## Phase 3 — Long-Term Memory (RAG)
|
||
|
||
**Goal:** Embedding-based retrieval so the agent recalls facts across sessions.
|
||
|
||
- [ ] `internal/memory/embed.go` — call llama.cpp `POST /embedding` to generate vectors
|
||
- [ ] SQLite vector storage: store embeddings as BLOB in `memories` table
|
||
- [ ] `internal/memory/store.go` — upgrade `Search()` from keyword `LIKE` to cosine similarity scan
|
||
- [ ] Temporal decay — older memories scored lower (weight = importance \* recency_factor)
|
||
- [ ] Importance scoring — user flags ("remember this"), emotional content, repeat mentions boost score
|
||
|
||
```sql
|
||
CREATE TABLE memories (
|
||
id TEXT PRIMARY KEY,
|
||
content TEXT NOT NULL,
|
||
embedding BLOB, -- float32 vector
|
||
source TEXT, -- 'conversation', 'reflection', 'user_input'
|
||
importance REAL DEFAULT 0.5, -- 0.0–1.0
|
||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||
last_accessed TEXT
|
||
);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_memories_importance ON memories(importance);
|
||
CREATE INDEX IF NOT EXISTS idx_memories_created ON memories(created_at);
|
||
```
|
||
|
||
- [ ] **Future:** Migrate to PostgreSQL + pgvector for production-scale vector search
|
||
- [ ] **Test:** Store info, restart agent, ask about stored info → verify recall
|
||
|
||
---
|
||
|
||
## Phase 4 — OpenAI-Compatible Adapter
|
||
|
||
**Goal:** Swap inference to any OpenAI-compatible API (OpenAI, vLLM, Ollama, Groq, etc.).
|
||
|
||
- [ ] `internal/llama/provider.go` — interface:
|
||
|
||
```go
|
||
type Provider interface {
|
||
Complete(ctx context.Context, req *CompletionRequest) (*CompletionResponse, error)
|
||
Embed(ctx context.Context, content string) ([]float64, error)
|
||
Health() error
|
||
}
|
||
```
|
||
|
||
- [ ] Rename `internal/llama/` → `internal/inference/`
|
||
- [ ] `llama.go` — implements Provider (existing llama.cpp client wrapped)
|
||
- [ ] `openai.go` — implements Provider via OpenAI-compatible REST API (`/v1/chat/completions`, `/v1/embeddings`)
|
||
- [ ] Config: `INFERENCE_PROVIDER=llama|openai`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `OPENAI_MODEL`
|
||
- [ ] Both providers produce same interface, switchable via env var
|
||
- [ ] **Test:** Same conversation, swap provider, verify behavior is consistent
|
||
|
||
---
|
||
|
||
## Phase 5 — Episodic Diary & Reflection
|
||
|
||
**Goal:** Daily diary entries + periodic self-reflection for narrative continuity. The diary is the emotional continuity layer.
|
||
|
||
**Status: CODE DONE** — needs LLM integration for real summaries.
|
||
|
||
- [x] `internal/companion/reflection.go` — `GenerateDiary`, `RecentObservations`, `AddObservation`
|
||
- [x] `diary_entries` table:
|
||
|
||
```sql
|
||
CREATE TABLE diary_entries (
|
||
id TEXT PRIMARY KEY,
|
||
date TEXT NOT NULL, -- YYYY-MM-DD
|
||
summary TEXT NOT NULL,
|
||
mood TEXT,
|
||
topics TEXT, -- JSON array
|
||
observations TEXT, -- JSON array
|
||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||
);
|
||
```
|
||
|
||
- [x] `observations` table:
|
||
|
||
```sql
|
||
CREATE TABLE observations (
|
||
id TEXT PRIMARY KEY,
|
||
content TEXT NOT NULL,
|
||
confidence REAL DEFAULT 0.5,
|
||
category TEXT, -- 'topic_interest', 'interaction_style', 'mood_pattern'
|
||
applied INTEGER DEFAULT 0,
|
||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||
);
|
||
```
|
||
|
||
- [ ] Connect scheduler's nightly diary trigger to LLM-generated summaries (currently writes stub text)
|
||
- [ ] Reflection → Personality update pipeline:
|
||
- Diary entries accumulate → periodic analysis → trait drift candidates
|
||
- Approval process: observations → candidates → bounded update
|
||
- Max 0.5% trait change/day, max 5%/month — slow evolution feels human, fast feels schizophrenic
|
||
- [ ] **Test:** Simulate conversations across multiple "days", verify diary entries, verify trait drift remains within bounds
|
||
|
||
---
|
||
|
||
## Phase 6 — MCP Tools & Integrations
|
||
|
||
**Goal:** Extensible tool system for web search, file access, external services.
|
||
|
||
**Status: PARTIAL**
|
||
|
||
- [x] `internal/tools/registry.go` — register/list/execute tools with parameter definitions
|
||
- [x] `internal/tools/builtin.go` — `web_fetch` (working), `web_search` (stub)
|
||
- [ ] MCP protocol client (`internal/tools/mcp.go`):
|
||
- Connect to MCP servers via stdio or TCP
|
||
- List available tools with parameters
|
||
- Execute tool calls, parse results into context
|
||
- [ ] Tool call loop in agent:
|
||
- LLM decides to call tool → agent executes → result appended to context → LLM generates final response
|
||
- [ ] MCP-managed integrations:
|
||
- Obsidian vault (via `enquire-mcp` or custom)
|
||
- Gitea/Forgejo operations
|
||
- Actual Budget (spending tracking, categorization)
|
||
- [ ] **Test:** `web_fetch` tool, verify agent can search, summarize, cite sources
|
||
|
||
---
|
||
|
||
## Phase 7 — Matrix Integration
|
||
|
||
**Goal:** Agent lives as a Matrix bot, accessible via DM. Matrix feels like `@diva:homeserver.tld` instead of `Bot #4321`.
|
||
|
||
**Status: STUB**
|
||
|
||
- [x] `internal/channel/matrix.go` — config + start stub (disabled by default)
|
||
- [ ] Full Matrix bot with `mautrix-go`:
|
||
- Listen for invitations, join rooms
|
||
- Handle DMs with typing indicators, read receipts
|
||
- Message parsing & sending
|
||
- Bot identity: `@diva:homeserver.tld`
|
||
- [ ] Config: homeserver URL, bot user token in env
|
||
- [ ] TUI and Matrix run concurrently — messages sync between both
|
||
- [ ] **Test:** Send DM to bot, verify response, verify cross-session memory
|
||
|
||
---
|
||
|
||
## Phase 8 — Self-Memory & Identity
|
||
|
||
**Goal:** Agent remembers its own state — promises, strategies, mistakes. "Bidirectional memory."
|
||
|
||
**Status: CODE DONE**
|
||
|
||
- [x] `internal/memory/self_memory.go`:
|
||
|
||
```go
|
||
type Promise struct {
|
||
ID string
|
||
Promise string
|
||
Context string
|
||
Fulfilled bool
|
||
CreatedAt time.Time
|
||
Deadline *time.Time
|
||
}
|
||
|
||
type Strategy struct {
|
||
ID string
|
||
Situation string
|
||
Strategy string
|
||
Outcome string
|
||
CreatedAt time.Time
|
||
}
|
||
```
|
||
|
||
- [x] `agent_promises` table (id, promise, context, fulfilled, created_at, deadline)
|
||
- [x] `agent_strategies` table (id, situation, strategy, outcome, created_at)
|
||
- [ ] Wire into Prompt Composer: recent promises, relevant strategies added to context
|
||
- [ ] **Test:** "What did you promise me yesterday?" → verify recall from self-memory
|
||
|
||
---
|
||
|
||
## Phase 9 — Safety Guardrails & Identity Stability
|
||
|
||
**Goal:** Prevent personality drift, maintain core safety rules.
|
||
|
||
- [ ] `internal/companion/safety.go`:
|
||
- Core Identity (never changes): values, communication style, boundaries, role definition
|
||
- State (dynamic): mood, interests, current projects, relationship opinions
|
||
- Separation: identity config (env/read-only) vs state (SQLite/writable)
|
||
- [ ] Personality safety rules:
|
||
- Traits bounded within configured min/max at all times
|
||
- Core safety rules override any personality state
|
||
- Permission policies cannot be modified by personality
|
||
- No tool access changes via personality evolution
|
||
- [ ] Prompt injection protection:
|
||
- User messages wrapped in separator tokens
|
||
- System prompt templates with strict, non-overridable sections
|
||
- Input sanitization at the agent boundary
|
||
- [ ] **Test:** Attempt prompt injection, verify guardrails hold
|
||
|
||
---
|
||
|
||
## Phase 10 — Polish: Streaming TUI, Token Tracking, Observability
|
||
|
||
**Goal:** Production-quality UX and operations.
|
||
|
||
- [ ] Token streaming in TUI — char-by-char from llama.cpp SSE or chunked response
|
||
- [ ] Context window manager:
|
||
- Track total token count per conversation
|
||
- Sliding window when context exceeds limit
|
||
- Auto-summarize oldest messages to stay within context
|
||
- [ ] `POST /v1/chat` streaming via SSE in API mode
|
||
- [ ] Structured JSON logging via `log/slog` (already done — custom handler with `[LEVEL] [TIME] [FILE:LINE]`)
|
||
- [ ] Prometheus metrics (`GET /metrics`)
|
||
- [ ] Graceful shutdown — drain in-flight requests, save agent state
|
||
- [ ] **Test:** Full restart cycle, verify all state restored
|
||
|
||
---
|
||
|
||
## Phase 11 — Future Enhancements (Optional)
|
||
|
||
- [ ] Voice interface: whisper.cpp (STT) + piper/XTTS (TTS)
|
||
- [ ] VRM/Live2D avatar rendering (like Open-LLM-VTuber, Soul of Waifu)
|
||
- [ ] Multi-user support with per-user relationship tracking
|
||
- [ ] Multi-room Matrix support
|
||
- [ ] Plugin system for community-contributed tools
|
||
- [ ] WebUI dashboard (like nanobot's WebUI)
|
||
- [ ] Mobile companion via Matrix (any Matrix client works)
|
||
- [ ] Home Assistant integration for physical automation
|
||
- [ ] PostgreSQL + pgvector migration for production-scale vector search
|
||
|
||
---
|
||
|
||
## Database Summary
|
||
|
||
Two separate SQLite databases (clean separation of concerns):
|
||
|
||
| Database | Purpose | Tables |
|
||
| -------------- | --------------- | ------------------------------------------------------- |
|
||
| `divacode.db` | Core agent data | `conversations`, `messages`, `context_state` |
|
||
| `divacode.db` | Memory | `memories` |
|
||
| `companion.db` | Personality | `personality_traits` |
|
||
| `companion.db` | Relationship | `relationship_metrics`, `shared_topics`, `inside_jokes` |
|
||
| `companion.db` | Reflection | `diary_entries`, `observations` |
|
||
| `companion.db` | Self-memory | `agent_promises`, `agent_strategies` |
|
||
|
||
**Future:** `divacode.db` may be replaced by PostgreSQL + pgvector for production deployments with many users or large memory stores.
|
||
|
||
---
|
||
|
||
## Key Principles
|
||
|
||
1. **Build the smallest loop first** — get message → memory → LLM → response → diary → reflection working before adding complexity
|
||
2. **Personality is data, not prompt text** — traits stored in SQLite, composed dynamically by the prompt builder
|
||
3. **Memory ≠ Personality** — factual memory (Engram-style RAG) is separate from relationship/emotional state
|
||
4. **Slow evolution** — max 0.5% trait change/day, max 5%/month. Fast evolution feels schizophrenic, slow feels human
|
||
5. **Core Identity is stable** — identity.yaml (or env config) never changes much; state.json (SQLite) changes constantly
|
||
6. **Diary before reflection** — the nightly diary is the emotional continuity layer, more important than vector search
|
||
7. **Autonomous messaging** — the scheduler is what makes the character feel alive, not the LLM intelligence
|
||
8. **No secrets in code** — all config via env vars, `.env` in `.gitignore`
|
||
9. **No config libraries** — std lib `os.Getenv` only, never `caarlos0/env` or similar
|
||
10. **No HTTP routers** — std lib `net/http` only, no third-party routers (chi, gorilla, etc.)
|
||
11. **Error handling** — wrap errors with `fmt.Errorf("context: %w", err)`, never silent discards
|
||
12. **Test as you go** — `go test -race ./...` before every push
|
||
13. **Output readable code** — explain _why_, never _what_. No comments for self-documenting code
|