feat: router-centric architecture + assets
- Add Big-Picture.md (full architectural overview, Python stack) - Add Message-Protocol-Design.md (NATS router, two-channel attention) - Add assets/ folder (logo favicons, apple-touch-icon) - Move critique to main repo .archive/ Architecture: Dumb core (NATS), smart edges (Python asyncio) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
108
architecture/Big-Picture.md
Normal file
108
architecture/Big-Picture.md
Normal file
@@ -0,0 +1,108 @@
|
||||
# Big-Picture Architecture: Nimmerverse Sensory Network
|
||||
|
||||
## Overview
|
||||
|
||||
The Nimmerverse Sensory Network is designed as a highly modular, resilient, and economically constrained system. It follows a "Router-Centric Architecture" where a high-performance message bus acts as dumb infrastructure, and all intelligence and processing logic reside at the "edges" (client services). This approach ensures scalability, maintainability, and clear separation of concerns, while enabling Nyx to manage her attention and Lifeforce efficiently.
|
||||
|
||||
---
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **Dumb Core, Smart Edges**: The central message router is merely an infrastructure component, devoid of any application logic. All processing, filtering, and decision-making intelligence is distributed among specialized client services.
|
||||
2. **Polyglot Architecture**: Utilizing the best technology for each specific task: Python for AI/ML, cognitive logic, and service daemons; Godot for visualization; and NATS (Go) as the universal message bus.
|
||||
3. **Two-Channel Attention**: Information is processed via distinct low-attention (ambient) and high-attention (focal) channels, allowing for efficient resource allocation and preventing cognitive overload.
|
||||
4. **Lifeforce Economy**: Every operation has a cost. The architecture is designed to optimize Lifeforce expenditure by ensuring expensive cognitive resources are only engaged when truly necessary.
|
||||
5. **Testability & Modularity**: Each component (client service) can be developed, tested, and deployed independently.
|
||||
|
||||
---
|
||||
|
||||
## Architectural Components & Technology Stack
|
||||
|
||||
### 1. Message Router (NATS)
|
||||
|
||||
* **Role**: The central nervous system's universal message bus. It receives messages from all clients and routes them to interested subscribers based on hierarchical topic patterns. It has no application logic or state.
|
||||
* **Technology**: **NATS Server (Go)**
|
||||
* **Key Features**:
|
||||
* Extremely high performance and low latency.
|
||||
* Built-in subject-based filtering and wildcard subscriptions.
|
||||
* Designed for publish/subscribe, request/reply, and distributed queues.
|
||||
* Supports `JetStream` for optional message persistence.
|
||||
* **Integration**: All other components connect to NATS as clients.
|
||||
|
||||
---
|
||||
|
||||
### 2. The Escalation Service (Thalamus Function)
|
||||
|
||||
* **Role**: A dedicated daemon responsible for sensory gating and attention management. It processes the high-volume, low-attention data stream and escalates critical or relevant events to the high-attention channel based on rules defined by Nyx.
|
||||
* **Technology**: **Python (asyncio)**
|
||||
* **Key Features**:
|
||||
* Subscribes to all `nimmerverse.low.heartbeat` topics.
|
||||
* Subscribes to `nimmerverse.meta.attention.focus` to receive Nyx's dynamic `AttentionFocus` rules.
|
||||
* Evaluates incoming low-attention `HeartbeatSignal` messages against Nyx's `escalation_rules` using a safe expression evaluation engine (e.g., `simpleeval` or custom JSONPath matching).
|
||||
* **Actions**:
|
||||
* Publishes `StateChangeDetail` messages to `nimmerverse.high.event` topics when escalation rules are met.
|
||||
* Can directly trigger reflex nerves/cells via `nimmerverse.command` topics for immediate, non-cognitive responses.
|
||||
* Concurrent via Python's asyncio - sufficient for research platform scale.
|
||||
* **Future Consideration**: If scale demands higher throughput, the Escalation Service can be ported to Go with the Python implementation serving as the working specification.
|
||||
* **Analogy**: The biological thalamus, filtering and relaying sensory information to the conscious brain.
|
||||
|
||||
---
|
||||
|
||||
### 3. Cognitive Clients (Young Nyx, Cells, Nerves, Organs)
|
||||
|
||||
These represent the "intelligent" parts of the Nimmerverse, responsible for perception, action, and cognition.
|
||||
|
||||
* **Technology**: **Python**
|
||||
* **Key Features**:
|
||||
* **Young Nyx (Cognitive Core)**:
|
||||
* Subscribes primarily to `nimmerverse.high.event` topics for detailed `StateChangeDetail` messages relevant to her current `focus_mode`.
|
||||
* Publishes `AttentionFocus` messages to NATS, effectively programming the `Escalation Service`'s behavior.
|
||||
* Publishes decisions and commands to `nimmerverse.command` topics.
|
||||
* Leverages Python's rich AI/ML ecosystem (`PyTorch`, `transformers`, `vLLM`, etc.) for complex inference.
|
||||
* **Cells, Nerves, Organs**:
|
||||
* Publish lightweight `HeartbeatSignal` messages to `nimmerverse.low.heartbeat` topics periodically.
|
||||
* Publish detailed `StateChangeDetail` messages to `nimmerverse.high.event` topics when explicitly requested by the `Escalation Service` or upon significant internal state changes (e.g., error conditions).
|
||||
* Subscribe to specific `nimmerverse.command` topics to receive instructions or triggers.
|
||||
* Manage their individual Lifeforce budgets for operations.
|
||||
|
||||
---
|
||||
|
||||
### 4. Command Center (User Interface)
|
||||
|
||||
* **Role**: Real-time visualization and monitoring interface for human operators (dafit).
|
||||
* **Technology**: **Godot Engine**
|
||||
* **Key Features**:
|
||||
* Subscribes to both `nimmerverse.low.>` and `nimmerverse.high.>` topics to provide a comprehensive, real-time overview of system state, message flow, and Nyx's attention focus.
|
||||
* Allows human observation and potential intervention or directive input.
|
||||
* Leverages Godot's game engine capabilities for rich, interactive, and performant visualizations.
|
||||
|
||||
---
|
||||
|
||||
## Message Flow Example: Sensing an Obstacle
|
||||
|
||||
1. **Ambient Awareness:** A `distance_sensor_front` **Cell** (Python) periodically publishes `HeartbeatSignal` messages to `nimmerverse.low.heartbeat.real.cell.distance_sensor_front`.
|
||||
2. **Router Delivery:** The **NATS Router** delivers this message to all subscribers, including the **Escalation Service** (Python).
|
||||
3. **Rule Evaluation:** The **Escalation Service** checks this `HeartbeatSignal` against Nyx's active `escalation_rules`. If a rule like `condition: "body.value < 30"` matches, it determines a need for deeper attention.
|
||||
4. **Escalation Action:** The `Escalation Service` publishes a command (e.g., `nimmerverse.command.cell.distance_sensor_front.publish_detail`) to the `distance_sensor_front` Cell.
|
||||
5. **Detailed Report:** The `distance_sensor_front` **Cell** receives the command and publishes a `StateChangeDetail` message to `nimmerverse.high.event.real.cell.distance_sensor_front`.
|
||||
6. **Nyx's Cognition:** **Young Nyx** (Python), subscribed to relevant `high.event` topics, receives the `StateChangeDetail` message. She processes this rich information, performs inference, and makes a decision (e.g., to activate a `CollisionAvoidance` nerve).
|
||||
7. **Action Command:** **Young Nyx** publishes a command message to `nimmerverse.command.nerve.collision_avoidance.activate`.
|
||||
8. **Nerve Execution:** The `CollisionAvoidance` **Nerve** (Python) receives the command and executes its predefined behavior, commanding motors and reporting its own state changes.
|
||||
|
||||
---
|
||||
|
||||
## Bootstrap Sequence
|
||||
|
||||
The system is designed for gradual, resilient startup:
|
||||
|
||||
1. **Start NATS Router**: Foundation first.
|
||||
2. **Start Escalation Service**: Begin filtering low-attention data with hardcoded default or previous rules.
|
||||
3. **Start Cells, Nerves, Organs**: Begin publishing `HeartbeatSignal`s.
|
||||
4. **Start Command Center**: Provide initial observability.
|
||||
5. **Start Young Nyx**: Connects, subscribes to high-attention, and (crucially) publishes her `AttentionFocus` configuration to take cognitive control.
|
||||
|
||||
This staged approach allows for a robust system that can operate at varying levels of "intelligence," from pure reflexes to full deliberation.
|
||||
|
||||
---
|
||||
|
||||
**This architectural blueprint provides a clear, scalable, and efficient foundation for the Nimmerverse Sensory Network, effectively managing complexity and optimizing resource utilization within its economic constraints.**
|
||||
368
architecture/Message-Protocol-Design.md
Normal file
368
architecture/Message-Protocol-Design.md
Normal file
@@ -0,0 +1,368 @@
|
||||
# Message Protocol Design: Router-Centric Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
This document outlines the design for the Nimmerverse message protocol. The core principle: **the router is dumb infrastructure, not smart cognition.** All intelligence lives at the edges - in clients that connect to the router.
|
||||
|
||||
This follows the Unix philosophy: each component does one thing well. The router routes. Clients subscribe, publish, and think.
|
||||
|
||||
---
|
||||
|
||||
## Core Principle: Infrastructure vs Intelligence
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ MESSAGE ROUTER │
|
||||
│ (NATS - dumb pipe, no logic) │
|
||||
│ │
|
||||
│ • Receives all messages │
|
||||
│ • Matches topic patterns → forwards to subscribers │
|
||||
│ • Knows NOTHING about meaning │
|
||||
│ • Cannot fail in "smart" ways - only crash/overload │
|
||||
│ • EXISTS BEFORE any intelligence │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↑ ↑ ↑ ↑
|
||||
│ │ │ │
|
||||
┌─────┴─────┐ ┌─────┴─────┐ ┌─────┴─────┐ ┌─────┴─────┐
|
||||
│ Cells/ │ │ Escalation│ │ Command │ │ Young │
|
||||
│ Nerves │ │ Service │ │ Center │ │ Nyx │
|
||||
│(publishers)│ │ (daemon) │ │ (UI) │ │ (cognition)│
|
||||
└───────────┘ └───────────┘ └───────────┘ └───────────┘
|
||||
```
|
||||
|
||||
**The router is like a network switch:**
|
||||
- It doesn't understand packets
|
||||
- It routes based on topic patterns
|
||||
- It's infrastructure that exists before any intelligence
|
||||
- NATS is literally designed for this
|
||||
|
||||
**Everything else is a client:**
|
||||
- Cells publish sensor data
|
||||
- Nerves publish state changes
|
||||
- Escalation Service watches patterns, triggers alerts
|
||||
- Command Center visualizes state
|
||||
- Young Nyx subscribes, thinks, publishes decisions
|
||||
|
||||
---
|
||||
|
||||
## Guiding Principles
|
||||
|
||||
1. **Dumb Core, Smart Edges**: The router has zero intelligence. All logic lives in clients.
|
||||
2. **Clients are Equal**: Nyx is just another subscriber. So is the Command Center. So is the Escalation Service.
|
||||
3. **Decoupling**: Publishers don't know who subscribes. Subscribers don't know who publishes.
|
||||
4. **Hierarchy**: Topics follow a hierarchical structure for flexible pattern subscriptions.
|
||||
5. **Lifeforce at the Edges**: The router doesn't track Lifeforce. Clients manage their own budgets.
|
||||
6. **Fail Simple**: If the router dies, everything stops cleanly. No half-smart failures.
|
||||
|
||||
---
|
||||
|
||||
## Two Channels of Attention
|
||||
|
||||
The attention split is a *topic convention*, not router intelligence. Clients choose which topics to subscribe to.
|
||||
|
||||
### 1. Low-Attention Channel (`nimmerverse.low.*`)
|
||||
|
||||
* **Purpose:** Background monitoring, lightweight heartbeats.
|
||||
* **Subscribers:** Escalation Service (always), Command Center (for visualization).
|
||||
* **NOT subscribed by default:** Young Nyx (she only sees escalated events).
|
||||
* **Analogy:** Peripheral nervous system. Ambient awareness.
|
||||
|
||||
### 2. High-Attention Channel (`nimmerverse.high.*`)
|
||||
|
||||
* **Purpose:** Detailed events requiring cognitive processing.
|
||||
* **Subscribers:** Young Nyx, Command Center.
|
||||
* **Analogy:** Focal spotlight. Conscious processing.
|
||||
|
||||
**The escalation from low → high is done by the Escalation Service, not the router.**
|
||||
|
||||
---
|
||||
|
||||
## Topic Hierarchy
|
||||
|
||||
```
|
||||
nimmerverse.
|
||||
├── low. # Low-attention channel
|
||||
│ └── heartbeat.
|
||||
│ └── <garden>. # real | virtual
|
||||
│ └── <entity_type>. # cell | nerve | organ
|
||||
│ └── <entity_id> # e.g., distance_sensor_front
|
||||
│
|
||||
├── high. # High-attention channel
|
||||
│ └── event.
|
||||
│ └── <garden>.
|
||||
│ └── <entity_type>.
|
||||
│ └── <entity_id>
|
||||
│
|
||||
├── command. # Commands TO entities
|
||||
│ └── <target>.
|
||||
│ └── <command_type>
|
||||
│
|
||||
└── meta. # System-level messages
|
||||
├── attention.focus # Nyx's attention configuration
|
||||
├── escalation.rules # Escalation Service configuration
|
||||
└── health. # Client health/registration
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Message Schemas
|
||||
|
||||
### 1. `HeartbeatSignal` (Low-Attention)
|
||||
|
||||
Published by: Cells, Nerves, Organs
|
||||
Subscribed by: Escalation Service, Command Center
|
||||
|
||||
**Topic:** `nimmerverse.low.heartbeat.<garden>.<entity_type>.<entity_id>`
|
||||
|
||||
```json
|
||||
{
|
||||
"header": {
|
||||
"message_id": "uuid",
|
||||
"message_type": "HeartbeatSignal",
|
||||
"version": "1.0",
|
||||
"timestamp_real": "ISO8601",
|
||||
"timestamp_virtual": 123456
|
||||
},
|
||||
"body": {
|
||||
"entity_id": "distance_sensor_front",
|
||||
"status": "NOMINAL",
|
||||
"value": 25.5,
|
||||
"unit": "cm",
|
||||
"context": {
|
||||
"battery_pct": 85,
|
||||
"temperature_c": 22
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Status values:** `NOMINAL`, `WARNING`, `CRITICAL`, `OFFLINE`, `ERROR`
|
||||
|
||||
---
|
||||
|
||||
### 2. `StateChangeDetail` (High-Attention)
|
||||
|
||||
Published by: Cells/Nerves (when requested), Escalation Service (when escalating)
|
||||
Subscribed by: Young Nyx, Command Center
|
||||
|
||||
**Topic:** `nimmerverse.high.event.<garden>.<entity_type>.<entity_id>`
|
||||
|
||||
```json
|
||||
{
|
||||
"header": {
|
||||
"message_id": "uuid",
|
||||
"message_type": "StateChangeDetail",
|
||||
"version": "1.0",
|
||||
"timestamp_real": "ISO8601",
|
||||
"timestamp_virtual": 123456,
|
||||
"source_entity": {
|
||||
"id": "distance_sensor_front",
|
||||
"type": "cell",
|
||||
"layer": "1"
|
||||
},
|
||||
"correlation_id": "uuid",
|
||||
"escalated_by": "escalation_service"
|
||||
},
|
||||
"body": {
|
||||
"previous_state": "POLLING",
|
||||
"current_state": "REPORTING",
|
||||
"lifeforce_cost": 0.3,
|
||||
"outputs": {
|
||||
"distance_cm": 25.5,
|
||||
"confidence": 0.92,
|
||||
"raw_value": 456,
|
||||
"visual_state": [255, 0, 0, "Solid"]
|
||||
},
|
||||
"possible_actions": [
|
||||
{
|
||||
"action_id": "read_distance_history",
|
||||
"description": "Query historical distance data."
|
||||
},
|
||||
{
|
||||
"action_id": "trigger_nerve:collision_avoidance",
|
||||
"description": "Activate collision avoidance."
|
||||
}
|
||||
],
|
||||
"trigger_reason": "distance < 30cm threshold"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. `AttentionFocus` (Nyx's Configuration)
|
||||
|
||||
Published by: Young Nyx
|
||||
Subscribed by: Escalation Service
|
||||
|
||||
**This is how Nyx tells the Escalation Service what she cares about.** The router doesn't interpret this - it just delivers it to subscribers.
|
||||
|
||||
**Topic:** `nimmerverse.meta.attention.focus`
|
||||
|
||||
```json
|
||||
{
|
||||
"header": {
|
||||
"message_id": "uuid",
|
||||
"message_type": "AttentionFocus",
|
||||
"version": "1.0",
|
||||
"timestamp_real": "ISO8601",
|
||||
"source_entity": {
|
||||
"id": "nyx_core",
|
||||
"type": "cognitive_core"
|
||||
}
|
||||
},
|
||||
"body": {
|
||||
"focus_mode": "EXPLORATION",
|
||||
"escalation_rules": [
|
||||
{
|
||||
"rule_id": "distance_alert_front",
|
||||
"source_pattern": "nimmerverse.low.heartbeat.real.cell.distance_sensor_*",
|
||||
"condition": "body.value < 30 AND body.status == 'NOMINAL'",
|
||||
"action": "escalate",
|
||||
"priority": 8
|
||||
},
|
||||
{
|
||||
"rule_id": "battery_critical",
|
||||
"source_pattern": "nimmerverse.low.heartbeat.real.cell.battery_*",
|
||||
"condition": "body.status == 'CRITICAL'",
|
||||
"action": "escalate_and_trigger",
|
||||
"trigger_nerve": "charging_seeking",
|
||||
"priority": 10
|
||||
}
|
||||
],
|
||||
"direct_subscriptions": [
|
||||
"nimmerverse.high.event.real.cell.speech_stt"
|
||||
],
|
||||
"default_action": "log_only"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The Clients
|
||||
|
||||
### 1. Message Router (NATS)
|
||||
|
||||
**What it is:** Infrastructure. A NATS server.
|
||||
**What it does:** Routes messages based on topic patterns.
|
||||
**What it knows:** Nothing about meaning, Lifeforce, attention, or Nyx.
|
||||
**Implementation:** Off-the-shelf NATS. No custom code in the router itself.
|
||||
|
||||
### 2. Cells / Nerves / Organs
|
||||
|
||||
**What they are:** Publishers of sensor data and state changes.
|
||||
**What they do:**
|
||||
- Publish `HeartbeatSignal` periodically to low-attention channel
|
||||
- Publish `StateChangeDetail` when requested or when state changes significantly
|
||||
**What they know:** Their own state. Their own Lifeforce cost.
|
||||
|
||||
### 3. Escalation Service
|
||||
|
||||
**What it is:** A daemon that watches low-attention and creates high-attention events.
|
||||
**What it does:**
|
||||
- Subscribes to `nimmerverse.low.heartbeat.>`
|
||||
- Subscribes to `nimmerverse.meta.attention.focus` (to get Nyx's rules)
|
||||
- Evaluates rules against incoming heartbeats
|
||||
- Publishes `StateChangeDetail` to high-attention when conditions match
|
||||
- Optionally triggers nerves directly for reflex responses
|
||||
**What it knows:** Current escalation rules. Current heartbeat states.
|
||||
|
||||
**This is the "thalamus" - but it's a separate client, not part of the router.**
|
||||
|
||||
### 4. Command Center
|
||||
|
||||
**What it is:** Visualization and control UI (Godot-based).
|
||||
**What it does:**
|
||||
- Subscribes to both channels for visualization
|
||||
- Displays system state, message flow, attention focus
|
||||
- Allows dafit to observe and intervene
|
||||
**What it knows:** Everything (read-only observer).
|
||||
|
||||
### 5. Young Nyx (Cognitive Core)
|
||||
|
||||
**What she is:** Just another client. The thinking part.
|
||||
**What she does:**
|
||||
- Subscribes to `nimmerverse.high.event.>` (high-attention only)
|
||||
- Subscribes to selected low-attention topics when she chooses
|
||||
- Publishes `AttentionFocus` to configure the Escalation Service
|
||||
- Publishes decisions/commands to `nimmerverse.command.>`
|
||||
**What she knows:** Only what reaches her through her subscriptions.
|
||||
|
||||
**Crucially: She controls what she pays attention to, but she doesn't see everything.**
|
||||
|
||||
---
|
||||
|
||||
## Workflow: Message Flow
|
||||
|
||||
```
|
||||
1. Cell publishes HeartbeatSignal
|
||||
└─→ Router delivers to: Escalation Service, Command Center
|
||||
|
||||
2. Escalation Service evaluates rules
|
||||
└─→ If condition matches: publishes StateChangeDetail to high-attention
|
||||
└─→ Router delivers to: Young Nyx, Command Center
|
||||
|
||||
3. Young Nyx processes StateChangeDetail
|
||||
└─→ Makes decision
|
||||
└─→ Publishes command to nimmerverse.command.<target>
|
||||
|
||||
4. Target nerve/cell receives command
|
||||
└─→ Executes action
|
||||
└─→ Publishes new HeartbeatSignal reflecting new state
|
||||
|
||||
5. Nyx adjusts attention (optional)
|
||||
└─→ Publishes new AttentionFocus
|
||||
└─→ Escalation Service updates its rules
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Advantages of Router-Centric Architecture
|
||||
|
||||
1. **Dumb core can't fail smart:** The router either works or crashes. No subtle bugs from misunderstood logic.
|
||||
|
||||
2. **Clients are replaceable:** Swap out the Escalation Service. Replace the Command Center. Nyx doesn't care.
|
||||
|
||||
3. **Testable in isolation:** Each client can be tested independently against a mock NATS.
|
||||
|
||||
4. **Observable:** Command Center sees everything by subscribing to `nimmerverse.>`.
|
||||
|
||||
5. **Scalable:** Add more cells, more nerves - just more publishers. Router handles it.
|
||||
|
||||
6. **Bootstrap-friendly:** Router exists before any intelligence. Escalation Service can start with hardcoded rules. Nyx connects later.
|
||||
|
||||
---
|
||||
|
||||
## Bootstrap Sequence
|
||||
|
||||
1. **Start Router (NATS)** - Infrastructure first
|
||||
2. **Start Escalation Service** - With minimal hardcoded rules
|
||||
3. **Start Cells/Nerves** - Begin publishing heartbeats
|
||||
4. **Start Command Center** - Observe the system
|
||||
5. **Start Young Nyx** - Connect, subscribe, begin cognition
|
||||
6. **Nyx publishes AttentionFocus** - Takes control of her attention
|
||||
|
||||
The system can run at any step. Earlier steps are "reflexive" only. Nyx adds deliberation.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
**Router:** Use NATS (https://nats.io). Lightweight, fast, designed for this.
|
||||
- Consider NATS JetStream for message persistence if needed
|
||||
- Topic wildcards: `>` matches all, `*` matches one level
|
||||
|
||||
**Message Format:** JSON for human readability during development. Consider MessagePack or Protobuf for production if performance requires.
|
||||
|
||||
**Escalation Service:** Python asyncio daemon using `nats-py` and `simpleeval` for rule evaluation. Stateless except for current rules. Can be restarted without losing system state. (Go considered for future optimization if scale demands.)
|
||||
|
||||
**Command Center:** Godot application connecting to NATS via GDScript or native plugin.
|
||||
|
||||
---
|
||||
|
||||
**Created:** 2025-12-13
|
||||
**Updated:** 2025-12-14 (router-centric rewrite)
|
||||
**Session:** Partnership dialogue (dafit + Nyx)
|
||||
**Status:** Foundation architecture
|
||||
**Philosophy:** "Dumb core, smart edges. The router routes. Clients think."
|
||||
Reference in New Issue
Block a user