# DivaCode — AI Being Platform Action Plan

**Language:** Go  
**Inference:** llama.cpp (HTTP server), OpenAI-compatible API planned  
**Storage:** SQLite (primary), PostgreSQL + pgvector (future)  
**Inspiration:** OpenHer, Agent Diva, nanobot

---

## Current Status (2026-06-10)

**Builds:** yes  
**Runs:** yes (requires llama.cpp on :8080)

**Known issues:**

- Prompt template is raw text — model dumps reasoning/self-correction meta-text instead of just the response
- No chat template configurability (ChatML, Llama 3 instruct, etc.)
- TUI is basic (no streaming, no word-wrap polish)
- Memory store uses keyword `LIKE` search, not real embeddings
- OpenAI-compatible adapter not yet built
- Matrix adapter is a stub

---

## Architecture Overview

```
User
  │
  ├── TUI (Bubble Tea)
  └── Matrix (future)
        │
  ┌─────┴──────────────────────────────┐
  │        Core Agent Loop             │
  │  (internal/agent/)                 │
  │  - Message routing                 │
  │  - Tool invocation                 │
  │  - Context management              │
  └─────┬──────────────────────────────┘
        │
  ┌─────┴──────────────────────────────┐
  │        Companion Layer             │
  │  (internal/companion/)             │
  │  ├─ Personality (5 bounded traits) │
  │  ├─ Relationship (familiarity,trust)│
  │  ├─ Reflection (diary,observations)│
  │  ├─ Mood (6 states + energy)       │
  │  └─ Prompt Composer (dynamic ctx)  │
  └─────┬──────────────────────────────┘
        │
  ┌─────┴──────────────────────────────┐
  │         Memory Layer               │
  │  (internal/memory/)                │
  │  ├─ Short-term (ephemeral context) │
  │  ├─ Long-term (SQLite → pgvector)  │
  │  ├─ Episodic / Diary (sqlite)      │
  │  └─ Self-memory (promises,strats)  │
  └─────┬──────────────────────────────┘
        │
  ┌─────┴──────────────────────────────┐
  │  Inference (internal/llama/)       │
  │  ├─ llama.cpp HTTP client          │
  │  └─ OpenAI-compatible (planned)    │
  └────────────────────────────────────┘

Skills / Tools:
  ├─ web_fetch
  ├─ web_search (stub)
  └─ MCP protocol (future)
       ├─ Obsidian vault
       ├─ Gitea
       ├─ Actual Budget
       └─ Custom MCP tools
```

---

## Phase 0 — Foundation & Scaffolding

**Status: COMPLETE**

- [x] Go module: `git.db123.ir/db123/divacode`
- [x] Project structure per Holy Code Bible conventions
- [x] `internal/config/` — std lib `os.Getenv` only
- [x] `internal/storage/` — `modernc.org/sqlite` (pure Go, no CGO), auto-creates `data/` dir
- [x] `internal/llama/client.go` — HTTP client for `POST /completion`, `POST /embedding`, `GET /health`
- [x] `.env.example`, `Makefile`, `.gitignore`, `Dockerfile`, `docker-compose.yml`
- [x] `AGENTS.md` with architecture, commands, conventions
- [x] `pkg/types/types.go` — Message, Role, Mood, ToolCall types
- [x] `pkg/personality/traits.go` — TraitValue, RelationshipMetrics, DiaryEntry, Observation types

---

## Phase 1 — Core Agent Loop

**Status: COMPLETE** (minor issues remain)

- [x] `internal/agent/agent.go` — message loop: receive → memory search → prompt → llama → save → respond
- [x] `internal/agent/scheduler.go` — background goroutine, configurable tick (default 15m), mood check, diary trigger, autonomous message queue
- [x] `cmd/divad/main.go` — wires everything: config, DB, llama health, TUI, scheduler, Matrix, API
- [x] Conversation persistence to SQLite (`conversations`, `messages` tables with indexes)
- [x] Bubble Tea TUI with viewport + input bar, Ctrl+C/Ctrl+Q quit
- [x] HTTP API (std lib `net/http`): `GET /health`, `POST /v1/chat`, `POST /v1/conversations`
- [ ] Prompt template is raw headers — model outputs self-correction/next-action meta-text instead of just the response
- [ ] No token tracking — context window grows unbounded
- [ ] TUI doesn't stream tokens (waits for full completion)

**Schema (`internal/storage/migrations.go`):**

```sql
CREATE TABLE conversations (
    id          TEXT PRIMARY KEY,
    created_at  TEXT NOT NULL DEFAULT (datetime('now')),
    updated_at  TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE TABLE messages (
    id              TEXT PRIMARY KEY,
    conversation_id TEXT NOT NULL REFERENCES conversations(id),
    role            TEXT NOT NULL CHECK(role IN ('user','assistant','system','tool')),
    content         TEXT NOT NULL,
    tool_call       TEXT,
    token_count     INTEGER,
    created_at      TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX IF NOT EXISTS idx_messages_conversation ON messages(conversation_id);

CREATE TABLE context_state (
    conversation_id TEXT PRIMARY KEY REFERENCES conversations(id),
    token_count     INTEGER NOT NULL DEFAULT 0,
    summary         TEXT
);
```

---

## Phase 2 — Prompt Templating & Reasoning-Aware TUI

**Goal:** Fix model output with proper chat templates. Parse reasoning/thoughts from the response and render them beautifully in the TUI — dimmed, collapsible, like OpenCode.

### Chat Template Engine

- [x] `internal/llama/template.go` — chat template formatter:
  - ChatML: `<|im_start|>system\n...<|im_end|>\n<|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n`
  - Llama 3 instruct: `<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n...<|eot_id|>`
  - Configurable via env `CHAT_TEMPLATE=chatml|llama3|none` (default: chatml)
- [x] Refactor `companion/promptbuilder.go` to output structured `BuildResult`
- [x] Let template.go wrap those segments in the chosen format
- [x] Add `stop` tokens per template (`<|im_end|>`, `<|eot_id|>`)

### Reasoning Parser

- [x] `internal/agent/reasoning.go` — parse `<think>`, `<reasoning>`, `<reason>`, `<thinking>` tags
- [x] Split response into `Reasoning` (thoughts) and `Content` (final answer)
- [x] Store both separately in the `messages` table
- [x] Feed only `Content` back into context window (reasoning is discarded after display)

### TUI Enhancement

- [x] Render reasoning in a collapsible section (dimmed/italic, `lipgloss.Color("#6B7280")`)
- [x] Render final answer in normal assistant style (green/bold)
- [x] Reasoning hidden by default, toggle with `tab` key
- [x] Mouse support (`tea.WithMouseCellMotion()` + viewport scroll)

### Command Palette & Themes

- [ ] `internal/channel/palette.go` — fuzzy-findable command palette overlay (Ctrl+P):
  - Modal popup with search/filter input
  - Renders a filtered list of commands with descriptions
  - Enter to execute, Esc to dismiss
- [ ] `/help` — show available commands with descriptions
- [ ] `/new` — start a new conversation (alias: `/clear`)
- [ ] `/exit` — quit the TUI (alias: `/quit`, `/q`)
- [ ] `/thinking` — toggle visibility of reasoning/thinking blocks in the conversation
- [ ] `/themes` — list and switch between color themes
- [ ] `/sessions` — list and switch between conversations (alias: `/resume`)
- [ ] `internal/channel/theme.go` — theme engine:
  - JSON-based theme config with `defs` + `theme` color map
  - Built-in themes: `default`, `tokyonight`, `catppuccin`, `gruvbox`, `nord`, `matrix`
  - Uses `THEME` env var or `/themes` command to switch
  - Hues: `primary`, `secondary`, `accent`, `text`, `textMuted`, `background`, `border`, `success`, `error`
- [ ] `internal/channel/sessions.go` — session management:
  - List existing conversations from SQLite
  - Switch agent's active conversation
  - Show per-session message count and last activity

---

## Phase 3 — Long-Term Memory (RAG)

**Goal:** Embedding-based retrieval so the agent recalls facts across sessions.

- [ ] `internal/memory/embed.go` — call llama.cpp `POST /embedding` to generate vectors
- [ ] SQLite vector storage: store embeddings as BLOB in `memories` table
- [ ] `internal/memory/store.go` — upgrade `Search()` from keyword `LIKE` to cosine similarity scan
- [ ] Temporal decay — older memories scored lower (weight = importance \* recency_factor)
- [ ] Importance scoring — user flags ("remember this"), emotional content, repeat mentions boost score

```sql
CREATE TABLE memories (
    id            TEXT PRIMARY KEY,
    content       TEXT NOT NULL,
    embedding     BLOB,              -- float32 vector
    source        TEXT,              -- 'conversation', 'reflection', 'user_input'
    importance    REAL DEFAULT 0.5,  -- 0.0–1.0
    created_at    TEXT NOT NULL DEFAULT (datetime('now')),
    last_accessed TEXT
);

CREATE INDEX IF NOT EXISTS idx_memories_importance ON memories(importance);
CREATE INDEX IF NOT EXISTS idx_memories_created ON memories(created_at);
```

- [ ] **Future:** Migrate to PostgreSQL + pgvector for production-scale vector search
- [ ] **Test:** Store info, restart agent, ask about stored info → verify recall

---

## Phase 4 — OpenAI-Compatible Adapter

**Goal:** Swap inference to any OpenAI-compatible API (OpenAI, vLLM, Ollama, Groq, etc.).

- [ ] `internal/llama/provider.go` — interface:

```go
type Provider interface {
    Complete(ctx context.Context, req *CompletionRequest) (*CompletionResponse, error)
    Embed(ctx context.Context, content string) ([]float64, error)
    Health() error
}
```

- [ ] Rename `internal/llama/` → `internal/inference/`
- [ ] `llama.go` — implements Provider (existing llama.cpp client wrapped)
- [ ] `openai.go` — implements Provider via OpenAI-compatible REST API (`/v1/chat/completions`, `/v1/embeddings`)
- [ ] Config: `INFERENCE_PROVIDER=llama|openai`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `OPENAI_MODEL`
- [ ] Both providers produce same interface, switchable via env var
- [ ] **Test:** Same conversation, swap provider, verify behavior is consistent

---

## Phase 5 — Episodic Diary & Reflection

**Goal:** Daily diary entries + periodic self-reflection for narrative continuity. The diary is the emotional continuity layer.

**Status: CODE DONE** — needs LLM integration for real summaries.

- [x] `internal/companion/reflection.go` — `GenerateDiary`, `RecentObservations`, `AddObservation`
- [x] `diary_entries` table:

```sql
CREATE TABLE diary_entries (
    id            TEXT PRIMARY KEY,
    date          TEXT NOT NULL,         -- YYYY-MM-DD
    summary       TEXT NOT NULL,
    mood          TEXT,
    topics        TEXT,                  -- JSON array
    observations  TEXT,                  -- JSON array
    created_at    TEXT NOT NULL DEFAULT (datetime('now'))
);
```

- [x] `observations` table:

```sql
CREATE TABLE observations (
    id          TEXT PRIMARY KEY,
    content     TEXT NOT NULL,
    confidence  REAL DEFAULT 0.5,
    category    TEXT,              -- 'topic_interest', 'interaction_style', 'mood_pattern'
    applied     INTEGER DEFAULT 0,
    created_at  TEXT NOT NULL DEFAULT (datetime('now'))
);
```

- [ ] Connect scheduler's nightly diary trigger to LLM-generated summaries (currently writes stub text)
- [ ] Reflection → Personality update pipeline:
  - Diary entries accumulate → periodic analysis → trait drift candidates
  - Approval process: observations → candidates → bounded update
  - Max 0.5% trait change/day, max 5%/month — slow evolution feels human, fast feels schizophrenic
- [ ] **Test:** Simulate conversations across multiple "days", verify diary entries, verify trait drift remains within bounds

---

## Phase 6 — MCP Tools & Integrations

**Goal:** Extensible tool system for web search, file access, external services.

**Status: PARTIAL**

- [x] `internal/tools/registry.go` — register/list/execute tools with parameter definitions
- [x] `internal/tools/builtin.go` — `web_fetch` (working), `web_search` (stub)
- [ ] MCP protocol client (`internal/tools/mcp.go`):
  - Connect to MCP servers via stdio or TCP
  - List available tools with parameters
  - Execute tool calls, parse results into context
- [ ] Tool call loop in agent:
  - LLM decides to call tool → agent executes → result appended to context → LLM generates final response
- [ ] MCP-managed integrations:
  - Obsidian vault (via `enquire-mcp` or custom)
  - Gitea/Forgejo operations
  - Actual Budget (spending tracking, categorization)
- [ ] **Test:** `web_fetch` tool, verify agent can search, summarize, cite sources

---

## Phase 7 — Matrix Integration

**Goal:** Agent lives as a Matrix bot, accessible via DM. Matrix feels like `@diva:homeserver.tld` instead of `Bot #4321`.

**Status: STUB**

- [x] `internal/channel/matrix.go` — config + start stub (disabled by default)
- [ ] Full Matrix bot with `mautrix-go`:
  - Listen for invitations, join rooms
  - Handle DMs with typing indicators, read receipts
  - Message parsing & sending
  - Bot identity: `@diva:homeserver.tld`
- [ ] Config: homeserver URL, bot user token in env
- [ ] TUI and Matrix run concurrently — messages sync between both
- [ ] **Test:** Send DM to bot, verify response, verify cross-session memory

---

## Phase 8 — Self-Memory & Identity

**Goal:** Agent remembers its own state — promises, strategies, mistakes. "Bidirectional memory."

**Status: CODE DONE**

- [x] `internal/memory/self_memory.go`:

```go
type Promise struct {
    ID        string
    Promise   string
    Context   string
    Fulfilled bool
    CreatedAt time.Time
    Deadline  *time.Time
}

type Strategy struct {
    ID        string
    Situation string
    Strategy  string
    Outcome   string
    CreatedAt time.Time
}
```

- [x] `agent_promises` table (id, promise, context, fulfilled, created_at, deadline)
- [x] `agent_strategies` table (id, situation, strategy, outcome, created_at)
- [ ] Wire into Prompt Composer: recent promises, relevant strategies added to context
- [ ] **Test:** "What did you promise me yesterday?" → verify recall from self-memory

---

## Phase 9 — Safety Guardrails & Identity Stability

**Goal:** Prevent personality drift, maintain core safety rules.

- [ ] `internal/companion/safety.go`:
  - Core Identity (never changes): values, communication style, boundaries, role definition
  - State (dynamic): mood, interests, current projects, relationship opinions
  - Separation: identity config (env/read-only) vs state (SQLite/writable)
- [ ] Personality safety rules:
  - Traits bounded within configured min/max at all times
  - Core safety rules override any personality state
  - Permission policies cannot be modified by personality
  - No tool access changes via personality evolution
- [ ] Prompt injection protection:
  - User messages wrapped in separator tokens
  - System prompt templates with strict, non-overridable sections
  - Input sanitization at the agent boundary
- [ ] **Test:** Attempt prompt injection, verify guardrails hold

---

## Phase 10 — Polish: Streaming TUI, Token Tracking, Observability

**Goal:** Production-quality UX and operations.

- [ ] Token streaming in TUI — char-by-char from llama.cpp SSE or chunked response
- [ ] Context window manager:
  - Track total token count per conversation
  - Sliding window when context exceeds limit
  - Auto-summarize oldest messages to stay within context
- [ ] `POST /v1/chat` streaming via SSE in API mode
- [ ] Structured JSON logging via `log/slog` (already done — custom handler with `[LEVEL] [TIME] [FILE:LINE]`)
- [ ] Prometheus metrics (`GET /metrics`)
- [ ] Graceful shutdown — drain in-flight requests, save agent state
- [ ] **Test:** Full restart cycle, verify all state restored

---

## Phase 11 — Future Enhancements (Optional)

- [ ] Voice interface: whisper.cpp (STT) + piper/XTTS (TTS)
- [ ] VRM/Live2D avatar rendering (like Open-LLM-VTuber, Soul of Waifu)
- [ ] Multi-user support with per-user relationship tracking
- [ ] Multi-room Matrix support
- [ ] Plugin system for community-contributed tools
- [ ] WebUI dashboard (like nanobot's WebUI)
- [ ] Mobile companion via Matrix (any Matrix client works)
- [ ] Home Assistant integration for physical automation
- [ ] PostgreSQL + pgvector migration for production-scale vector search

---

## Database Summary

Two separate SQLite databases (clean separation of concerns):

| Database       | Purpose         | Tables                                                  |
| -------------- | --------------- | ------------------------------------------------------- |
| `divacode.db`  | Core agent data | `conversations`, `messages`, `context_state`            |
| `divacode.db`  | Memory          | `memories`                                              |
| `companion.db` | Personality     | `personality_traits`                                    |
| `companion.db` | Relationship    | `relationship_metrics`, `shared_topics`, `inside_jokes` |
| `companion.db` | Reflection      | `diary_entries`, `observations`                         |
| `companion.db` | Self-memory     | `agent_promises`, `agent_strategies`                    |

**Future:** `divacode.db` may be replaced by PostgreSQL + pgvector for production deployments with many users or large memory stores.

---

## Key Principles

1. **Build the smallest loop first** — get message → memory → LLM → response → diary → reflection working before adding complexity
2. **Personality is data, not prompt text** — traits stored in SQLite, composed dynamically by the prompt builder
3. **Memory ≠ Personality** — factual memory (Engram-style RAG) is separate from relationship/emotional state
4. **Slow evolution** — max 0.5% trait change/day, max 5%/month. Fast evolution feels schizophrenic, slow feels human
5. **Core Identity is stable** — identity.yaml (or env config) never changes much; state.json (SQLite) changes constantly
6. **Diary before reflection** — the nightly diary is the emotional continuity layer, more important than vector search
7. **Autonomous messaging** — the scheduler is what makes the character feel alive, not the LLM intelligence
8. **No secrets in code** — all config via env vars, `.env` in `.gitignore`
9. **No config libraries** — std lib `os.Getenv` only, never `caarlos0/env` or similar
10. **No HTTP routers** — std lib `net/http` only, no third-party routers (chi, gorilla, etc.)
11. **Error handling** — wrap errors with `fmt.Errorf("context: %w", err)`, never silent discards
12. **Test as you go** — `go test -race ./...` before every push
13. **Output readable code** — explain _why_, never _what_. No comments for self-documenting code