DivaCode/plan.md

DivaCode — AI Being Platform Action Plan

Language: Go
Inference: llama.cpp (HTTP server), OpenAI-compatible API planned
Inspiration: OpenHer, Agent Diva, nanobot

---

Current Status (2026-06-10)

Builds: yes
Runs: yes (requires llama.cpp on :8080)
Known issues:

• Prompt template is raw text — model dumps reasoning/self-correction meta-text instead of just the response
• No chat template configurability (ChatML, Llama 3 instruct, etc.)
• TUI is basic (no streaming, no word-wrap polish)
• Memory store uses keyword LIKE search, not real embeddings
• OpenAI-compatible adapter not yet built
• Matrix adapter is a stub

---

Architecture Overview

User
  │
  ├── TUI (Bubble Tea)
  └── Matrix (future)
        │
  ┌─────┴──────────────────────────────┐
  │        Core Agent Loop             │
  │  (internal/agent/)                 │
  └─────┬──────────────────────────────┘
        │
  ┌─────┴──────────────────────────────┐
  │        Companion Layer             │
  │  (internal/companion/)             │
  │  ├─ Personality (5 bounded traits) │
  │  ├─ Relationship (familiarity,trust)│
  │  ├─ Reflection (diary,observations)│
  │  ├─ Mood (6 states + energy)       │
  │  └─ Prompt Composer (dynamic ctx)  │
  └─────┬──────────────────────────────┘
        │
  ┌─────┴──────────────────────────────┐
  │         Memory Layer               │
  │  (internal/memory/)                │
  │  ├─ RAG store (keywords → vectors) │
  │  └─ Self-memory (promises,strats)  │
  └─────┬──────────────────────────────┘
        │
  ┌─────┴──────────────────────────────┐
  │  Inference (internal/llama/)       │
  │  ├─ llama.cpp HTTP client          │
  │  └─ OpenAI-compatible (planned)    │
  └────────────────────────────────────┘

Skills / Tools:
  ├─ web_fetch
  ├─ web_search (stub)
  └─ MCP protocol (future)

---

Phase 0 — Foundation & Scaffolding

Status: COMPLETE

• Go module: git.db123.ir/db123/divacode
• Project structure per Holy Code Bible conventions
• internal/config/ — std lib os.Getenv only
• internal/storage/ — modernc.org/sqlite (pure Go, no CGO), auto-creates data/ dir
• internal/llama/client.go — HTTP client for POST /completion, POST /embedding, GET /health
• .env.example, Makefile, .gitignore, Dockerfile, docker-compose.yml
• AGENTS.md with architecture, commands, conventions

---

Phase 1 — Core Agent Loop

Status: COMPLETE (minor issues remain)

• internal/agent/agent.go — message → memory search → prompt → llama → save → respond
• internal/agent/scheduler.go — tick every 15m, check mood/diary/auto-message
• cmd/divad/main.go — wires everything: config, DB, llama, TUI, scheduler, Matrix, API
• Conversation persistence to SQLite (conversations, messages tables)
• Bubble Tea TUI with viewport + input bar
• Prompt template is wrong — uses raw headers, no chat format. Model outputs meta-text
• No token tracking — context window grows unbounded
• TUI doesn't stream tokens (waits for full completion)

---

Phase 2 — Prompt Templating & Format Fix

Goal: Fix the model output. Swap raw prompt text for a configurable chat template so the model only returns the assistant response.

• internal/llama/template.go — chat template formatter:
  • ChatML (<|im_start|>system...<|im_end|>)
  • Llama 3 instruct (<|begin_of_text|><|start_header_id|>system<|end_header_id|>)
  • Configurable via env CHAT_TEMPLATE=chatml (default)
• Refactor companion/promptbuilder.go to output structured segments, then let template.go wrap them
• Strip system prompt of raw formatting — let the template handle structure
• Test: "hi" produces just "Hello!" not paragraphs of self-correction

---

Phase 3 — Long-Term Memory (RAG)

Goal: Embedding-based retrieval so the agent recalls across sessions.

• internal/memory/embed.go — call llama.cpp POST /embedding to get vectors
• internal/memory/store.go — upgrade Search() from keyword LIKE to cosine similarity
• Store embeddings as BLOB in memories table
• Temporal decay — older memories scored lower
• Importance scoring — "remember this" / emotional content boosts score
• Test: Store info, restart, ask about stored info → recall

---

Phase 4 — OpenAI-Compatible Adapter

Goal: Swap inference to any OpenAI-compatible API (OpenAI, vLLM, Ollama, etc.).

• internal/llama/provider.go — interface:

  type Provider interface {
      Complete(ctx, *Request) (*Response, error)
      Embed(ctx, string) ([]float64, error)
      Health() error
  }
  

• Rename internal/llama/ → internal/inference/
• llama.go — implements Provider (existing code)
• openai.go — implements Provider via OpenAI-compatible REST API
• Config: INFERENCE_PROVIDER=llama|openai, OPENAI_API_KEY, OPENAI_BASE_URL
• Test: Both providers produce same interface, switchable via env

---

Phase 5 — Episodic Diary & Reflection

Status: CODE DONE — needs integration into the scheduled pipeline.

• internal/companion/reflection.go — GenerateDiary, RecentObservations, AddObservation
• diary_entries table with date, summary, mood, topics, observations
• observations table with confidence, category, applied flag
• Connect scheduler's diary trigger to actual LLM-generated summaries (currently writes stub text)
• Reflection → Personality update pipeline:
  • Observations accumulate → periodic analysis → bounded trait drift
  • Max 0.5% change/day, max 5%/month

---

Phase 6 — MCP Tools & Integrations

Status: PARTIAL

• internal/tools/registry.go — register/list/execute tools
• internal/tools/builtin.go — web_fetch, web_search (stub)
• MCP protocol client (internal/tools/mcp.go) — stdio/TCP, list tools, execute, parse results
• Tool call loop in agent: LLM decides → agent executes → result appended → LLM finalizes
• Plugins: Obsidian vault, Gitea, Actual Budget

---

Phase 7 — Matrix Integration

Status: STUB

• internal/channel/matrix.go — config + start stub
• Full Matrix bot with mautrix-go:
  • Listen for invitations, join rooms
  • Handle DMs with typing indicators
  • Bot identity: @diva:homeserver.tld
• Sync TUI and Matrix state
• Test: Send DM, verify response, verify cross-session memory

---

Phase 8 — Self-Memory & Identity

Status: CODE DONE

• internal/memory/self_memory.go — promises, strategies
• agent_promises table (id, promise, context, fulfilled, deadline)
• agent_strategies table (id, situation, strategy, outcome)
• Wire into Prompt Composer: "You promised to X" context
• Test: "What did you promise me?" → recall

---

Phase 9 — Safety Guardrails & Identity Stability

Goal: Prevent personality drift, maintain core safety.

• internal/companion/safety.go — core identity (stable) vs state (dynamic)
• Personality safety: bounded traits, safety overrides personality, no tool access changes
• Prompt injection protection: user input wrapped in delimiters, strict template sections

---

Phase 10 — Polish: Streaming TUI, Token Tracking, Observability

Goal: Production-quality UX.

• Token streaming in TUI (char-by-char from llama.cpp SSE or chunked response)
• Context window manager — track tokens, sliding window when full, auto-summarize old messages
• POST /v1/chat streaming via SSE in API mode
• Prometheus metrics (GET /metrics)
• Graceful shutdown — drain in-flight requests, save state

---

Phase 11 — Future Enhancements (Optional)

• Voice: whisper.cpp (STT) + piper/XTTS (TTS)
• VRM/Live2D avatar rendering
• Multi-user with per-user relationship tracking
• Multi-room Matrix
• Plugin system
• WebUI dashboard
• Home Assistant integration

---

Database Schema (15 tables across 2 DBs)

divacode.db:

• conversations, messages, context_state, memories

companion.db:

• personality_traits, relationship_metrics, shared_topics, inside_jokes
• diary_entries, observations
• agent_promises, agent_strategies

---

Key Principles

1. Smallest loop first — get message → memory → llm → response working before adding complexity
2. Personality is data, not prompt text — traits in SQLite, composed dynamically
3. Memory ≠ Personality — factual memory separate from relationship/emotional state
4. Slow evolution — max 0.5% trait change/day, 5%/month
5. Core Identity stable — never allow personality to override safety rules
6. Diary before reflection — nightly diary is the emotional continuity layer
7. Autonomous messaging — scheduler makes it feel alive, not the LLM
8. No config libs — std lib os.Getenv only
9. No HTTP routers — std lib net/http only
10. Error wrapping — fmt.Errorf("context: %w", err), never discard