feat: Initial NimmerSky documentation
- ARCHITECTURE-RESEARCH.md: Three-system landscape analysis (Mantella/CHIM/SkyrimNet) - Context bleed analysis and fixes - Oghma Infinium RAG documentation - Nimmerverse integration architecture - inference_architecture_plan.txt: Deployed inference architecture - Euryale-70B (dialogue) on Theia:31001 - Gemma-27B (structured JSON) on Dioscuri:31004 - Qwen3-VL-8B (vision) on Dioscuri:31005 - gameplay_stack.txt: Modlist configuration - gameplay_dependencies.txt: Mod dependencies - Mod priority/status tracking Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
dafit
261
ARCHITECTURE-RESEARCH.md
Normal file
261
ARCHITECTURE-RESEARCH.md
Normal file
@@ -0,0 +1,261 @@
|
||||
# NimmerSky Architecture Research
|
||||
|
||||
Research findings from 2026-03-18 session exploring the Skyrim AI landscape and nimmerverse integration opportunities.
|
||||
|
||||
---
|
||||
|
||||
## The Three-System Landscape
|
||||
|
||||
| System | Strength | Weakness | Architecture |
|
||||
|--------|----------|----------|--------------|
|
||||
| **Mantella** | Established, stable, wide adoption | Older architecture | External Python server |
|
||||
| **CHIM** | Oghma Infinium RAG, knowledge gating | Windows/WSL2 dependency | Dwemer Distro (Debian VM) |
|
||||
| **SkyrimNet** | Native SKSE, action system, OmniSight | No lore RAG, context bleed | C++ DLL + Inja templates |
|
||||
|
||||
**Our approach**: Harvest the best from each — SkyrimNet's game integration, CHIM's lore database, nimmerverse's memory substrate.
|
||||
|
||||
---
|
||||
|
||||
## SkyrimNet Context Bleed Analysis
|
||||
|
||||
### Source Location
|
||||
```
|
||||
/home/dafit/Downloads/SkyrimNet-beta17.1/SKSE/Plugins/SkyrimNet/prompts/
|
||||
```
|
||||
|
||||
### The Problem
|
||||
NPCs omnisciently know the player's name and events they didn't witness.
|
||||
|
||||
### Root Cause
|
||||
The `player` object is **globally available** to all NPC prompts without knowledge gating.
|
||||
|
||||
**Key files with leakage:**
|
||||
|
||||
| File | Line | Issue |
|
||||
|------|------|-------|
|
||||
| `submodules/character_bio/0600_relationships.prompt` | 7 | `Travel History with {{ player.name }}` |
|
||||
| `submodules/character_bio/0010_header.prompt` | 8, 22, 24, 28, 30 | `{{ player.name }}` / `{{ decnpc(player.UUID).name }}` |
|
||||
| `components/context/scene_context.prompt` | Various | Player references in nearby actor descriptions |
|
||||
|
||||
### Event History (Partial Isolation)
|
||||
```inja
|
||||
{% set _event_filter = append(_event_filter, npc.UUID) %}
|
||||
{% set events = get_recent_events(_event_count, _event_filter) %}
|
||||
```
|
||||
Events ARE filtered by NPC UUID — but the implementation of `get_recent_events` is in C++ (SkyrimNet.dll), unclear if it's truly per-NPC witnessed events.
|
||||
|
||||
### Potential Fixes
|
||||
|
||||
**Level 1 - Template Hack:**
|
||||
```inja
|
||||
{% if has_memory_tag(npc.UUID, "met_player") %}
|
||||
{% set known_player_name = player.name %}
|
||||
{% else %}
|
||||
{% set known_player_name = "the stranger" %}
|
||||
{% endif %}
|
||||
```
|
||||
|
||||
**Level 2 - Memory Check:**
|
||||
Add C++ decorator `npc_knows_player_name(uuid)` that queries memory system.
|
||||
|
||||
**Level 3 - Nimmerverse Substrate:**
|
||||
Route all "what does NPC X know about Y" through external memory service.
|
||||
|
||||
---
|
||||
|
||||
## Oghma Infinium (CHIM's RAG System)
|
||||
|
||||
### What It Is
|
||||
1900+ tagged entries of Tamrielic lore with knowledge class filtering.
|
||||
|
||||
### Source
|
||||
**Google Sheets (Open/Downloadable):**
|
||||
https://docs.google.com/spreadsheets/d/1dcfctU-iOqprwy2BOc7___4Awteczgdlv8886KalPsQ/
|
||||
|
||||
### Knowledge Class Taxonomy
|
||||
|
||||
| Category | Examples | Purpose |
|
||||
|----------|----------|---------|
|
||||
| **Racial** | `nord`, `argonian`, `khajiit`, `darkelf`, `redguard` | Cultural knowledge |
|
||||
| **Profession** | `blacksmith`, `scholar`, `mage`, `alchemist`, `merchant` | Trade knowledge |
|
||||
| **Location** | `whiterun`, `rift`, `eastmarch`, `haafingar`, `solstheim` | Geographic knowledge |
|
||||
| **Faction** | `thieves_guild`, `companions`, `dark_brotherhood`, `college` | Organizational secrets |
|
||||
| **Special** | `charactername`, `knowall` | Edge cases |
|
||||
|
||||
### Content Sheets
|
||||
- Knowledge Classes Reference (taxonomy)
|
||||
- Vanilla NPCs (NPC → class mappings)
|
||||
- Visual Descriptions
|
||||
- Dynamic Oghma
|
||||
- Groups/Lore/Books
|
||||
- Figures/Gods
|
||||
- Artifacts, Armor/Weapons, Items, Spells, Creatures
|
||||
- Location sheets (one per Hold)
|
||||
|
||||
### Multi-Tag Intersection
|
||||
NPCs receive lore from **overlapping domains**:
|
||||
- Whiterun blacksmith: `[blacksmith] ∩ [nord] ∩ [whiterun]`
|
||||
- Riften fence: `[merchant] ∩ [thieves_guild] ∩ [rift]`
|
||||
|
||||
### Embedding System
|
||||
Uses **Minime-T5** for vector embeddings.
|
||||
|
||||
---
|
||||
|
||||
## SkyrimNet Memory System
|
||||
|
||||
### Documentation
|
||||
https://goncalo22.github.io/SkyrimNet-GamePlugin/Memory%20System/memory-recall
|
||||
|
||||
### Storage Architecture
|
||||
- **Database**: SQLite with HNSW vector indexing (per-NPC files)
|
||||
- **Embeddings**: MiniLM-L6-v2 (384-dimensional)
|
||||
- **Limits**: 1000 memories/NPC, minimum importance 0.2
|
||||
|
||||
### Memory Data Structure
|
||||
```json
|
||||
{
|
||||
"summary": "Brief description",
|
||||
"detailed_description": "Full narrative",
|
||||
"emotion": "joyful|angry|fearful|sad|neutral|...",
|
||||
"importance_score": 0.0-1.0,
|
||||
"tags": ["people", "places", "items", "activities"],
|
||||
"memory_type": "EXPERIENCE|RELATIONSHIP|KNOWLEDGE|TRAUMA|JOY|...",
|
||||
"embedding": [384-dimensional vector]
|
||||
}
|
||||
```
|
||||
|
||||
### Retrieval Scoring Weights
|
||||
|
||||
| Signal | Weight |
|
||||
|--------|--------|
|
||||
| Semantic similarity | 0.35 |
|
||||
| Temporal proximity | 0.20 |
|
||||
| Actor involvement | 0.20 |
|
||||
| Emotional match | 0.10 |
|
||||
| Keyword relevance | 0.10 |
|
||||
| Location match | 0.05 |
|
||||
|
||||
### Memory Formation Triggers
|
||||
- Conversations
|
||||
- Combat encounters
|
||||
- Item transactions
|
||||
- Proximity to notable actions
|
||||
- Grouped into segments (60min gap, 10-480min duration, 5-200 events)
|
||||
|
||||
---
|
||||
|
||||
## Nimmerverse Integration Architecture
|
||||
|
||||
### The Vision
|
||||
SkyrimNet as **game interface**, nimmerverse as **cognitive substrate**.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ NIMMERVERSE SUBSTRATE │
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────────────────────────┐ │
|
||||
│ │ phoebe-dev │ │ iris-dev │ │
|
||||
│ │ PostgreSQL │ │ ChromaDB │ │
|
||||
│ │ :35432 │ │ :35000 │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ • NPC relations │ │ • Oghma Infinium (lore vectors) │ │
|
||||
│ │ • Knowledge │ │ • NPC Memories (migrated) │ │
|
||||
│ │ classes │ │ • 384-dim MiniLM embeddings │ │
|
||||
│ │ • Event log │ │ │ │
|
||||
│ │ • Who knows whom │ │ │ │
|
||||
│ └──────────────────┘ └──────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ nats-dev │ │
|
||||
│ │ :30000 │ │
|
||||
│ │ │ │
|
||||
│ │ • Memory events │ │
|
||||
│ │ • Gossip pubsub │ │
|
||||
│ │ • Real-time │ │
|
||||
│ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ SkyrimNet │
|
||||
│ │
|
||||
│ • Query lore │
|
||||
│ • Query memory │
|
||||
│ • Log events │
|
||||
│ • Get classes │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
### Migration Path
|
||||
|
||||
**Phase 1: Oghma Infinium Import**
|
||||
1. Download CSV from Google Sheets
|
||||
2. Load knowledge classes into PostgreSQL
|
||||
3. Embed lore entries with MiniLM-L6-v2
|
||||
4. Store in ChromaDB with class metadata
|
||||
|
||||
**Phase 2: Memory Migration**
|
||||
1. Create ChromaDB collection for NPC memories
|
||||
2. Match SkyrimNet's data structure
|
||||
3. Replicate scoring weights in retrieval
|
||||
4. Add NPC UUID to all memories (enables cross-NPC queries)
|
||||
|
||||
**Phase 3: Knowledge Gating Service**
|
||||
1. Build service that answers "What does NPC X know about topic Y?"
|
||||
2. Combines: Knowledge classes + Lore retrieval + Memory search
|
||||
3. Expose via MCP or HTTP for SkyrimNet integration
|
||||
|
||||
**Phase 4: Gossip Network**
|
||||
1. NATS pubsub for memory propagation
|
||||
2. When NPCs interact, memories can spread (with distortion)
|
||||
3. "Lydia told Farengar about the dragon attack" becomes queryable
|
||||
|
||||
### Benefits Over Current Architecture
|
||||
|
||||
| Feature | SkyrimNet Native | Nimmerverse Integration |
|
||||
|---------|-----------------|------------------------|
|
||||
| Memory storage | Per-NPC SQLite | Unified ChromaDB |
|
||||
| Cross-NPC queries | Not possible | "Who witnessed X?" |
|
||||
| Lore grounding | LLM training only | Oghma Infinium RAG |
|
||||
| Knowledge isolation | Global `player` leaks | Per-NPC class filtering |
|
||||
| Gossip propagation | None | NATS-based network |
|
||||
| Infrastructure | Windows-compatible | Linux native |
|
||||
|
||||
---
|
||||
|
||||
## Key Resources
|
||||
|
||||
### SkyrimNet
|
||||
- Source: `/home/dafit/Downloads/SkyrimNet-beta17.1/`
|
||||
- Prompts: `SKSE/Plugins/SkyrimNet/prompts/`
|
||||
- Memory docs: https://goncalo22.github.io/SkyrimNet-GamePlugin/Memory%20System/memory-recall
|
||||
- Conversations docs: https://goncalo22.github.io/SkyrimNet-GamePlugin/conversations
|
||||
|
||||
### CHIM / Oghma Infinium
|
||||
- Nexus: https://www.nexusmods.com/skyrimspecialedition/mods/126330
|
||||
- Wiki: https://dwemerdynamics.hostwiki.io/
|
||||
- Oghma CSV: https://docs.google.com/spreadsheets/d/1dcfctU-iOqprwy2BOc7___4Awteczgdlv8886KalPsQ/
|
||||
|
||||
### Mantella
|
||||
- Nexus: https://www.nexusmods.com/skyrimspecialedition/mods/98631
|
||||
|
||||
### Current Modlist
|
||||
- **Decision**: Tested Schtevie's Requiem Full Content Extension — returning to own foundation
|
||||
- Our stack: SkyrimNet + custom nimmerverse integration
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [x] Test Schtevie's modlist stability ✓ (2026-03-18: tested, returning to own foundation)
|
||||
- [ ] Export Oghma Infinium CSV sheets
|
||||
- [ ] Design PostgreSQL schema for knowledge classes
|
||||
- [ ] Prototype ChromaDB lore collection
|
||||
- [ ] Map SkyrimNet template injection points for external queries
|
||||
- [ ] Design MCP/HTTP interface for lore service
|
||||
|
||||
---
|
||||
|
||||
**Version:** 1.0 | **Created:** 2026-03-18
|
||||
Reference in New Issue
Block a user