db123/DivaCode

Fork 0

Files

db123-ai b8b030ef06

docs: add comprehensive 13-phase action plan for DivaCode

2026-06-10 09:50:14 +03:30

19 KiB

Raw Blame History

DivaCode — AI Being Platform Action Plan

Language: Go
Inference: llama.cpp (via go-llama.cpp bindings or llama.cpp server API)
Inspiration: OpenHer (AI Being), Agent Diva (modular agent architecture), nanobot (lightweight design)

Architecture Overview

User
  │
  ├── TUI Terminal (Bubble Tea / gum)
  └── Matrix Adapter (future)
        │
  ┌─────┴──────────────────────────────────┐
  │           Core Agent Loop              │
  │  (cmd/divad)                           │
  │  - Message routing                     │
  │  - MCP tool invocation                 │
  │  - Tool execution & response           │
  └─────┬──────────────────────────────────┘
        │
  ┌─────┴──────────────────────────────────┐
  │        Companion Layer                 │
  │  (internal/companion/)                 │
  │  ├─ Personality Engine (traits, drift) │
  │  ├─ Relationship Engine (metrics)      │
  │  ├─ Reflection Engine (diary, summar.) │
  │  └─ Prompt Composer (dynamic context)  │
  └─────┬──────────────────────────────────┘
        │
  ┌─────┴──────────────────────────────────┐
  │         Memory Layer                   │
  │  (internal/memory/)                    │
  │  ├─ Short-term (ephemeral context)     │
  │  ├─ Long-term (SQLite vector store)    │
  │  ├─ Episodic / Diary (sqlite)          │
  │  └─ Self-memory (agent's own state)    │
  └─────┬──────────────────────────────────┘
        │
  ┌─────┴──────────────────────────────────┐
  │         llama.cpp Inference            │
  │  (internal/llama/)                     │
  │  - llama.cpp server HTTP API client    │
  │  - Context window management           │
  │  - Token streaming                     │
  └────────────────────────────────────────┘

Skill / Tool Layer (MCP):
  ├── Web search / fetch
  ├── Obsidian vault access
  ├── Gitea integration
  ├── Budget tracking (Actual)
  └── Custom MCP tools

Phase 0 — Foundation & Scaffolding

Goal: Project skeleton, Go module, llama.cpp server connectivity.

Initialize Go module: git.db123.ir/db123/divacode
Project structure (per Holy Code Bible conventions):

divacode/
├── cmd/
│   └── divad/              # Main agent daemon (TUI entrypoint)
├── internal/
│   ├── companion/           # Personality, relationship, reflection
│   ├── memory/              # Memory stores & retrieval
│   ├── llama/               # llama.cpp client
│   ├── agent/               # Core agent loop
│   ├── tools/               # MCP tool definitions
│   ├── config/              # Config loading
│   └── storage/             # SQLite helpers
├── pkg/                     # Reusable shared library
├── migrations/              # SQLite schema migrations
├── .env.example
├── Makefile
├── go.mod / go.sum
└── README.md

config/ package: load .env with caarlos0/env or joho/godotenv
storage/ package: SQLite wrapper with modernc.org/sqlite (pure Go, no CGO)
Run llama.cpp server in subprocess or connect to existing: llama-server -m model.gguf --port 8080
internal/llama/ client: HTTP API wrapper for llama.cpp server (POST /completion, GET /health)
Test: ping llama.cpp health endpoint, send a test completion

Go dependencies seeded:

modernc.org/sqlite — pure-Go SQLite
github.com/charmbracelet/bubbletea — TUI framework
github.com/charmbracelet/bubbles — TUI components
github.com/charmbracelet/lipgloss — TUI styling
github.com/caarlos0/env/v11 — config from env

Phase 1 — Core Agent Loop

Goal: A minimal conversational loop with llama.cpp, context management, and a TUI chat interface.

internal/agent/ — core loop:
- Receives user input → appends to context window
- Calls llama.cpp with context + system prompt
- Streams tokens back to TUI
- Appends assistant response to context
Context window manager:
- Track total token count
- Implement sliding window / summarization when context exceeds limit
cmd/divad/main.go — Bubble Tea TUI:
- Chat view (message list): user messages right-aligned, assistant left-aligned
- Input bar at bottom
- Scrollable message history
- Hotkey: Ctrl+C quit, Ctrl+L clear context
System prompt loaded from config (config.toml or env var)
Persist conversation history to SQLite (conversations table, messages table)
Test: Start divad, type messages, verify responses stream correctly, verify persistence across restarts

Schema (migrations/001_init.up.sql):

CREATE TABLE conversations (
    id          TEXT PRIMARY KEY,          -- UUID v7
    created_at  TEXT NOT NULL DEFAULT (datetime('now')),
    updated_at  TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE TABLE messages (
    id              TEXT PRIMARY KEY,
    conversation_id TEXT NOT NULL REFERENCES conversations(id),
    role            TEXT NOT NULL CHECK(role IN ('user','assistant','system')),
    content         TEXT NOT NULL,
    token_count     INTEGER,
    created_at      TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE TABLE context_state (
    conversation_id TEXT PRIMARY KEY REFERENCES conversations(id),
    token_count     INTEGER NOT NULL DEFAULT 0,
    summary         TEXT
);

Phase 2 — Personality Engine

Goal: Stable personality traits stored in SQLite, loaded dynamically as part of system prompt.

Based on dumpeddata.md personality architecture.

internal/companion/personality.go:
- Trait struct: Humor, Curiosity, Playfulness, Formality, Affection (float64 0.0–1.0)
- Load/save from SQLite (personality_traits table)
- Bounded ranges: Humor 0.6 ± 0.2, Curiosity 0.8 ± 0.1 (prevents drift)
personality_traits table:

CREATE TABLE personality_traits (
    id        TEXT PRIMARY KEY,
    trait     TEXT NOT NULL UNIQUE,
    value     REAL NOT NULL DEFAULT 0.5,
    min_value REAL NOT NULL DEFAULT 0.0,
    max_value REAL NOT NULL DEFAULT 1.0,
    updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);

Prompt Composer (internal/companion/promptbuilder.go):
- Reads current traits + relationship state + recent memory
- Generates dynamic system prompt section:

Current personality:
- Curious (0.82)
- Moderately playful (0.44)
- Technical (0.70)

Interaction preferences:
- Use concise language
- Ask follow-up questions
- Reference past conversations naturally

Integrate into agent loop: prompt composer modifies system prompt per turn
Test: Verify traits are persisted, loaded, and reflected in prompt

Phase 3 — Long-Term Memory (RAG)

Goal: Embedding-based retrieval for factual memory across conversations.

internal/memory/:
- Embedding subpackage: call llama.cpp embedding endpoint (POST /embedding) to generate vectors
- SQLite vector storage: store embeddings as BLOBs or use sqlite-vec extension
- Memory entry schema:

CREATE TABLE memories (
    id          TEXT PRIMARY KEY,
    content     TEXT NOT NULL,
    embedding   BLOB,              -- float32 vector
    source      TEXT,              -- 'conversation', 'reflection', 'user_input'
    importance  REAL DEFAULT 0.5,  -- 0.0–1.0
    created_at  TEXT NOT NULL DEFAULT (datetime('now')),
    last_accessed TEXT
);

Memory retrieval:
- On each user message, search top-K similar memories via cosine similarity
- Include retrieved memories in context window
- Implement temporal decay (older memories scored lower)
Importance scoring: messages flagged as important (user says "remember this", emotional content) get higher importance
Test: Store memories, restart agent, ask about stored info, verify recall

Phase 4 — Episodic Diary & Reflection Engine

Goal: Daily diary entries + periodic self-reflection for narrative continuity.

Based on dumpeddata.md: "The diary becomes the emotional continuity layer."

internal/companion/reflection.go:
- Diary entry at end of each conversation session (or nightly):
  - Summarize topics discussed
  - Note user mood/emotions
  - Record what the agent learned
- diary_entries table:

CREATE TABLE diary_entries (
    id          TEXT PRIMARY KEY,
    date        TEXT NOT NULL,         -- YYYY-MM-DD
    summary     TEXT NOT NULL,
    mood        TEXT,
    topics      TEXT,                  -- JSON array
    observations TEXT,                 -- JSON array
    created_at  TEXT NOT NULL DEFAULT (datetime('now'))
);

Reflection Engine:
- Periodic analysis (every N conversations or daily)
- Examines recent diary entries for patterns
- Generates observations:
  - "User discussed Go development frequently."
  - "User responds positively to light humor."
- Observations stored in observations table

CREATE TABLE observations (
    id          TEXT PRIMARY KEY,
    content     TEXT NOT NULL,
    confidence  REAL DEFAULT 0.5,
    category    TEXT,              -- 'topic_interest', 'interaction_style', 'mood_pattern'
    created_at  TEXT NOT NULL DEFAULT (datetime('now')),
    applied     INTEGER DEFAULT 0  -- whether fed into personality update
);

Reflection → Personality update pipeline:
- Observations accumulate → periodic analysis → trait drift candidates → bounded approval → personality update
- Max 0.5% trait change per day, max 5% per month
Test: Simulate conversations across multiple "days", verify diary entries, verify trait drift remains bounded

Phase 5 — Relationship Engine

Goal: Track relationship dynamics separate from factual memory.

Based on dumpeddata.md relationship engine design.

internal/companion/relationship.go:
- Metrics: Familiarity, Trust, SharedTopics, InsideJokes
- Stored in relationship_metrics table:

CREATE TABLE relationship_metrics (
    id          TEXT PRIMARY KEY,
    metric      TEXT NOT NULL UNIQUE,
    value       REAL NOT NULL DEFAULT 0.0,
    updated_at  TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE TABLE shared_topics (
    id      TEXT PRIMARY KEY,
    topic   TEXT NOT NULL UNIQUE,
    count   INTEGER NOT NULL DEFAULT 1,
    last_discussed TEXT
);

CREATE TABLE inside_jokes (
    id      TEXT PRIMARY KEY,
    joke    TEXT NOT NULL,
    context TEXT,
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

Update rules:
- Familiarity increases with conversation frequency & duration
- Trust increases on successful interactions, decreases on ignored requests
- SharedTopics tracked via keyword extraction from conversation
- InsideJokes manually taggable or auto-detected via repetition
Relationship state fed into Prompt Composer for dynamic relationship context
Test: Verify relationship metrics update and persist

Phase 6 — Autonomous Behavior (Scheduler)

Goal: Agent-initiated actions — messaging first, diary writing, periodic reflection.

Based on dumpeddata.md: "The missing piece: agency."

internal/agent/scheduler.go:
- Background goroutine with configurable tick interval (default: 15 min)
- On each tick, evaluate:
  - Current mood (derived from recent interactions)
  - Time since last conversation
  - Unresolved topics
  - Loneliness / boredom score
- Decision tree:

if time_since_last_contact > 48h && trust > 0.6:
    → send "Haven't heard from you in a while..." message
if mood == "curious" && unfinished_topics exists:
    → send follow-up question
if energy < 20:
    → stay silent (don't respond immediately)

Mood system (internal/companion/mood.go):
- Mood derivation from recent interactions + relationship state
- Moods: curious, playful, thoughtful, annoyed, tired, neutral
- Mood influences response style in prompt
Energy system: depletes with activity, recovers with idle time
Autonomous diary: scheduled nightly entry generation
Test: Simulate 48h idle, verify agent initiates contact

Phase 7 — MCP Tools & Integrations

Goal: Extensible tool system for web search, file access, external services.

internal/tools/mcp.go — MCP protocol client:
- Connect to MCP servers via stdio or TCP
- List available tools
- Execute tool calls
- Parse results into context
Built-in tools (no MCP server needed):
- web_search — search via configurable search API
- web_fetch — fetch URL content
- note — save/read from local notes
MCP-managed tools (external servers):
- Obsidian vault access (via enquire-mcp or custom)
- Gitea/Forgejo operations
- Actual Budget integration
Tool call loop in agent:
- LLM decides to call tool → agent executes → result appended to context → LLM generates final response
Test: web_search tool, verify agent can search, summarize, cite

Phase 8 — Matrix Integration

Goal: Agent lives as a Matrix bot, accessible via DM.

Matrix bot with mautrix-go (Go Matrix client library)
internal/channel/matrix.go:
- Listen for invitations, join rooms
- Handle DMs (1-on-1)
- Typing indicators, read receipts
- Message parsing & sending
Bot identity: @diva:your-homeserver.tld
Config: homeserver URL, bot user token in env
TUI and Matrix run concurrently — messages sync between both
Test: Send DM to bot, verify response, verify cross-session memory

Phase 9 — Self-Memory & Identity

Goal: Agent remembers its own state, promises, strategies, and mistakes.

Based on dumpeddata.md: "Add self-memory — the agent remembers itself as well as the user."

internal/memory/self_memory.go:
- agent_promises table:

CREATE TABLE agent_promises (
    id            TEXT PRIMARY KEY,
    promise       TEXT NOT NULL,
    context       TEXT,
    fulfilled     INTEGER DEFAULT 0,
    created_at    TEXT NOT NULL DEFAULT (datetime('now')),
    deadline      TEXT
);

agent_strategies table: what worked / didn't work

CREATE TABLE agent_strategies (
    id          TEXT PRIMARY KEY,
    situation   TEXT NOT NULL,
    strategy    TEXT NOT NULL,
    outcome     TEXT,              -- 'success', 'failure', 'neutral'
    created_at  TEXT NOT NULL DEFAULT (datetime('now'))
);

agent_mistakes table: errors and learnings
Integrate into Prompt Composer: recent promises, relevant strategies
Test: Ask agent "What did you promise me yesterday?" → verify recall

Phase 10 — Safety Guardrails & Identity Stability

Goal: Prevent personality drift, maintain core safety rules.

Based on dumpeddata.md safety rules and Holy Code Bible security laws.

internal/companion/safety.go:
- Core Identity (never changes): values, communication style, boundaries, role definition
- State (dynamic): mood, interests, current projects, relationship opinions
- Separation: identity.yaml (read-only) vs state.json (writable)
Personality safety rules:
- Traits bounded within configured min/max
- Core safety rules override any personality state
- Permission policies cannot be modified by personality
- No tool access changes via personality
Prompt injection protection:
- User messages wrapped in separator tokens
- System prompt templates with strict sections
- Regular expression filters for prompt injection patterns
Test: Attempt prompt injection, verify guardrails hold

Phase 11 — Polish, Deployment & Automation

Goal: Production-ready deployment, backup, observability.

Systemd service (or Docker Compose) for:
- divad (agent daemon with TUI + Matrix)
- llama.cpp server
- (Optional) external MCP servers
Makefile targets:
- make build — build binary
- make run — run with default config
- make test — run all tests
- make lint — golangci-lint run
- make vet — go vet ./...
Automated backups:
- SQLite databases: divacode.db, companion.db
- Llama model files
- Cron job: 0 3 * * * tar -czf backup-$(date +\%F).tar.gz /path/to/data
Logging: structured JSON logs via log/slog
Metrics: Prometheus endpoint (GET /metrics) via prometheus/client_golang
Config hardening: .env.example, required vars validation at startup
Test: Full restart cycle, verify all state restored

Phase 12 — Future Enhancements (Optional)

Voice interface: whisper.cpp (STT) + piper (TTS)
VRM/Live2D avatar rendering (like Open-LLM-VTuber, Soul of Waifu)
Multi-user support with per-user relationship tracking
Multi-room Matrix support
Plugin system for community-contributed tools
WebUI dashboard (like nanobot's WebUI)
Mobile companion via Matrix (any Matrix client works)
Home Assistant integration for physical automation

Database Summary

Database	Purpose	Tables
`divacode.db`	Core agent data	`conversations`, `messages`, `context_state`
`divacode.db`	Memory	`memories`
`companion.db`	Personality	`personality_traits`
`companion.db`	Relationship	`relationship_metrics`, `shared_topics`, `inside_jokes`
`companion.db`	Reflection	`diary_entries`, `observations`
`companion.db`	Self-memory	`agent_promises`, `agent_strategies`, `agent_mistakes`

Key Principles (from dumpeddata.md & Holy Code Bible)

Build the smallest loop first — Matrix → Memory → LLM → Diary → Reflection. Get that working before adding complexity.
Personality is data, not prompt text — traits stored in SQLite, composed dynamically.
Memory ≠ Personality — factual memory (Engram-style) is separate from relationship/emotional state.
Slow evolution — max 0.5% trait change/day, max 5%/month. Fast evolution feels schizophrenic.
Core Identity is stable — identity.yaml never changes much; state.json changes constantly.
Diary before reflection — the nightly diary is the emotional continuity layer, more important than vector search.
Autonomous messaging — the scheduler is what makes the character feel alive, not the LLM intelligence.
No secrets in code — all config via env vars, .env in .gitignore.
Error handling — wrap errors with fmt.Errorf("...: %w"), never discard.
Test as you go — go test -race ./... before every push.

19 KiB Raw Blame History Unescape Escape