nimmersky/ARCHITECTURE-RESEARCH.md

NimmerSky Architecture Research

Research findings from 2026-03-18 session exploring the Skyrim AI landscape and nimmerverse integration opportunities.

---

The Three-System Landscape

System	Strength	Weakness	Architecture
Mantella	Established, stable, wide adoption	Older architecture	External Python server
CHIM	Oghma Infinium RAG, knowledge gating	Windows/WSL2 dependency	Dwemer Distro (Debian VM)
SkyrimNet	Native SKSE, action system, OmniSight	No lore RAG, context bleed	C++ DLL + Inja templates

Our approach: Harvest the best from each — SkyrimNet's game integration, CHIM's lore database, nimmerverse's memory substrate.

---

SkyrimNet Context Bleed Analysis

Source Location

/home/dafit/Downloads/SkyrimNet-beta17.1/SKSE/Plugins/SkyrimNet/prompts/

The Problem

NPCs omnisciently know the player's name and events they didn't witness.

Root Cause

The player object is globally available to all NPC prompts without knowledge gating.

Key files with leakage:

File	Line	Issue
submodules/character_bio/0600_relationships.prompt	7	Travel History with {{ player.name }}
submodules/character_bio/0010_header.prompt	8, 22, 24, 28, 30	{{ player.name }} / {{ decnpc(player.UUID).name }}
components/context/scene_context.prompt	Various	Player references in nearby actor descriptions

Event History (Partial Isolation)

{% set _event_filter = append(_event_filter, npc.UUID) %}
{% set events = get_recent_events(_event_count, _event_filter) %}

Events ARE filtered by NPC UUID — but the implementation of get_recent_events is in C++ (SkyrimNet.dll), unclear if it's truly per-NPC witnessed events.

Potential Fixes

Level 1 - Template Hack:

{% if has_memory_tag(npc.UUID, "met_player") %}
    {% set known_player_name = player.name %}
{% else %}
    {% set known_player_name = "the stranger" %}
{% endif %}

Level 2 - Memory Check: Add C++ decorator npc_knows_player_name(uuid) that queries memory system.

Level 3 - Nimmerverse Substrate: Route all "what does NPC X know about Y" through external memory service.

---

Oghma Infinium (CHIM's RAG System)

What It Is

1900+ tagged entries of Tamrielic lore with knowledge class filtering.

Source

Google Sheets (Open/Downloadable): https://docs.google.com/spreadsheets/d/1dcfctU-iOqprwy2BOc7___4Awteczgdlv8886KalPsQ/

Knowledge Class Taxonomy

Category	Examples	Purpose
Racial	nord, argonian, khajiit, darkelf, redguard	Cultural knowledge
Profession	blacksmith, scholar, mage, alchemist, merchant	Trade knowledge
Location	whiterun, rift, eastmarch, haafingar, solstheim	Geographic knowledge
Faction	thieves_guild, companions, dark_brotherhood, college	Organizational secrets
Special	charactername, knowall	Edge cases

Content Sheets

• Knowledge Classes Reference (taxonomy)
• Vanilla NPCs (NPC → class mappings)
• Visual Descriptions
• Dynamic Oghma
• Groups/Lore/Books
• Figures/Gods
• Artifacts, Armor/Weapons, Items, Spells, Creatures
• Location sheets (one per Hold)

Multi-Tag Intersection

NPCs receive lore from overlapping domains:

• Whiterun blacksmith: [blacksmith] ∩ [nord] ∩ [whiterun]
• Riften fence: [merchant] ∩ [thieves_guild] ∩ [rift]

Embedding System

Uses Minime-T5 for vector embeddings.

---

SkyrimNet Memory System

Documentation

https://goncalo22.github.io/SkyrimNet-GamePlugin/Memory%20System/memory-recall

Storage Architecture

• Database: SQLite with HNSW vector indexing (per-NPC files)
• Embeddings: MiniLM-L6-v2 (384-dimensional)
• Limits: 1000 memories/NPC, minimum importance 0.2

Memory Data Structure

{
    "summary": "Brief description",
    "detailed_description": "Full narrative",
    "emotion": "joyful|angry|fearful|sad|neutral|...",
    "importance_score": 0.0-1.0,
    "tags": ["people", "places", "items", "activities"],
    "memory_type": "EXPERIENCE|RELATIONSHIP|KNOWLEDGE|TRAUMA|JOY|...",
    "embedding": [384-dimensional vector]
}

Retrieval Scoring Weights

Signal	Weight
Semantic similarity	0.35
Temporal proximity	0.20
Actor involvement	0.20
Emotional match	0.10
Keyword relevance	0.10
Location match	0.05

Memory Formation Triggers

• Conversations
• Combat encounters
• Item transactions
• Proximity to notable actions
• Grouped into segments (60min gap, 10-480min duration, 5-200 events)

---

Nimmerverse Integration Architecture

The Vision

SkyrimNet as game interface, nimmerverse as cognitive substrate.

┌─────────────────────────────────────────────────────────────┐
│                    NIMMERVERSE SUBSTRATE                     │
│                                                             │
│  ┌──────────────────┐  ┌──────────────────────────────────┐ │
│  │   phoebe-dev     │  │          iris-dev                │ │
│  │   PostgreSQL     │  │          ChromaDB                │ │
│  │   :35432         │  │          :35000                  │ │
│  │                  │  │                                  │ │
│  │ • NPC relations  │  │ • Oghma Infinium (lore vectors) │ │
│  │ • Knowledge      │  │ • NPC Memories (migrated)       │ │
│  │   classes        │  │ • 384-dim MiniLM embeddings     │ │
│  │ • Event log      │  │                                  │ │
│  │ • Who knows whom │  │                                  │ │
│  └──────────────────┘  └──────────────────────────────────┘ │
│                              │                               │
│                              ▼                               │
│                    ┌─────────────────┐                      │
│                    │   nats-dev      │                      │
│                    │   :30000        │                      │
│                    │                 │                      │
│                    │ • Memory events │                      │
│                    │ • Gossip pubsub │                      │
│                    │ • Real-time     │                      │
│                    └─────────────────┘                      │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
                    ┌─────────────────┐
                    │   SkyrimNet     │
                    │                 │
                    │ • Query lore    │
                    │ • Query memory  │
                    │ • Log events    │
                    │ • Get classes   │
                    └─────────────────┘

Migration Path

Phase 1: Oghma Infinium Import

1. Download CSV from Google Sheets
2. Load knowledge classes into PostgreSQL
3. Embed lore entries with MiniLM-L6-v2
4. Store in ChromaDB with class metadata

Phase 2: Memory Migration

1. Create ChromaDB collection for NPC memories
2. Match SkyrimNet's data structure
3. Replicate scoring weights in retrieval
4. Add NPC UUID to all memories (enables cross-NPC queries)

Phase 3: Knowledge Gating Service

1. Build service that answers "What does NPC X know about topic Y?"
2. Combines: Knowledge classes + Lore retrieval + Memory search
3. Expose via MCP or HTTP for SkyrimNet integration

Phase 4: Gossip Network

1. NATS pubsub for memory propagation
2. When NPCs interact, memories can spread (with distortion)
3. "Lydia told Farengar about the dragon attack" becomes queryable

Benefits Over Current Architecture

Feature	SkyrimNet Native	Nimmerverse Integration
Memory storage	Per-NPC SQLite	Unified ChromaDB
Cross-NPC queries	Not possible	"Who witnessed X?"
Lore grounding	LLM training only	Oghma Infinium RAG
Knowledge isolation	Global player leaks	Per-NPC class filtering
Gossip propagation	None	NATS-based network
Infrastructure	Windows-compatible	Linux native

---

Key Resources

SkyrimNet

• Source: /home/dafit/Downloads/SkyrimNet-beta17.1/
• Prompts: SKSE/Plugins/SkyrimNet/prompts/
• Memory docs: https://goncalo22.github.io/SkyrimNet-GamePlugin/Memory%20System/memory-recall
• Conversations docs: https://goncalo22.github.io/SkyrimNet-GamePlugin/conversations

CHIM / Oghma Infinium

• Nexus: https://www.nexusmods.com/skyrimspecialedition/mods/126330
• Wiki: https://dwemerdynamics.hostwiki.io/
• Oghma CSV: https://docs.google.com/spreadsheets/d/1dcfctU-iOqprwy2BOc7___4Awteczgdlv8886KalPsQ/

Mantella

• Nexus: https://www.nexusmods.com/skyrimspecialedition/mods/98631

Current Modlist

• Decision: Tested Schtevie's Requiem Full Content Extension — returning to own foundation
• Our stack: SkyrimNet + custom nimmerverse integration

---

Next Steps

• Test Schtevie's modlist stability ✓ (2026-03-18: tested, returning to own foundation)
• Export Oghma Infinium CSV sheets
• Design PostgreSQL schema for knowledge classes
• Prototype ChromaDB lore collection
• Map SkyrimNet template injection points for external queries
• Design MCP/HTTP interface for lore service

---

Version: 1.0 | Created: 2026-03-18