nimmerverse-sensory-network/architecture/nerves/Nervous-Protocol.md

Nervous Protocol: Three-Tier Autonomous Learning Architecture

Created: 2025-12-07 Updated: 2025-12-07 (LangChain integration) Status: Design Document Version: 1.1 (LangChain Implementation)

---

Overview

The Nervous Protocol defines how intelligence flows through the Nimmerverse via a three-tier architecture with message-based communication, state machine tools, and collaborative learning.

The Three Tiers:

┌─────────────────────────────────────────────┐
│              dafit                          │
│         (Strategic Architect)               │
│   • Vision & architecture decisions         │
│   • Override authority                      │
│   • Long-term direction                     │
└──────────────────┬──────────────────────────┘
                   ↕ (strategic guidance / major escalations)
┌─────────────────────────────────────────────┐
│          Chrysalis-Nyx                      │
│       (Oversight & Reasoning)               │
│   • Claude Opus/Sonnet (large context)      │
│   • Full toolchain access via LangChain     │
│   • Reviews Young Nyx's proposals           │
│   • Designs new state machines              │
│   • Teaching & guidance                     │
└──────────────────┬──────────────────────────┘
                   ↕ (guidance / escalations)
┌─────────────────────────────────────────────┐
│            Young Nyx                        │
│       (Autonomous Learning Agent)           │
│   • Smaller model (7B or similar)           │
│   • Limited known state machines            │
│   • Executes routine tasks                  │
│   • Learns from experience                  │
│   • Escalates complex problems              │
└─────────────────────────────────────────────┘

---

Core Principles

1. Message-Based Continuity

All communication flows through phoebe (PostgreSQL) via message tables:

• partnership_to_nimmerverse_messages (dafit + Chrysalis → Young Nyx)
• nimmerverse_to_partnership_messages (Young Nyx → dafit + Chrysalis)

Why messages?

• ✅ Persistent across sessions
• ✅ Asynchronous (no blocking)
• ✅ Auditable (every decision logged)
• ✅ Simple (append-only, no complex state sync)

2. Heartbeat Coordination

From Endgame-Vision.md:

• Real clock: 1 Hz (1 beat/sec) - wall time, free
• Virtual clock: Variable - computation time, costs lifeforce

On each heartbeat:

1. Check for new messages from any tier
2. Process guidance/tasks/escalations
3. Update state
4. Take next action
5. Write results back to phoebe

Not real-time (milliseconds), but continuous (heartbeat-driven).

3. State Machines as Tools

All capabilities are exposed as state machine tools via LangChain:

# Example: phoebe query state machine
from langchain.tools import BaseTool

States: IDLE → CONNECTED → QUERY_READY → IDLE

class PhoebeQueryTool(BaseTool):
    name = "phoebe_query"
    description = """
    Interact with phoebe database using state machine pattern.

    Available actions depend on current state:
    - IDLE: connect(host, db) → CONNECTED
    - CONNECTED: query(sql) → QUERY_READY, disconnect() → IDLE
    - QUERY_READY: query(sql), disconnect() → IDLE
    """

Why state machines?

• ✅ Safety (can't skip steps - must CONNECT before QUERY)
• ✅ Discoverable (each state announces valid transitions)
• ✅ Observable (log every transition)
• ✅ Composable (chain state machines together)

4. Progressive Capability Unlocking

Dual catalogues:

• All available tools (full registry, managed by dafit/Chrysalis)
• Young Nyx's known tools (subset she's discovered)

Young Nyx can only see/use tools she's discovered. New tools are granted:

• Via teaching moments (Chrysalis: "You're ready for X")
• Via successful escalations (solved problem reveals tool)
• Via collaborative design (she helps build it)

Discovery tracking in phoebe:

CREATE TABLE discovered_tools (
    agent_id TEXT,
    tool_name TEXT,
    discovered_at TIMESTAMPTZ DEFAULT NOW(),
    discovered_via TEXT,  -- "teaching", "escalation", "design"
    PRIMARY KEY (agent_id, tool_name)
);

---

The OR Gate Pattern (Input Sources)

From nimmerverse.drawio.xml (lines 215-244):

┌──────────┐       ┌──────────┐
│  dafit   │       │chrysalis │
│ (OR gate)│       │ (OR gate)│
└────┬─────┘       └────┬─────┘
     │                  │
     └────────┬─────────┘
              ↓ (OR - either/both)
         Message Queue (phoebe)
              ↓ (read on heartbeat)
         Orchestrator
              ↓
         Young Nyx

OR gate = Either/both can write, no blocking

Both dafit and Chrysalis write to partnership_to_nimmerverse_messages. The orchestrator synthesizes on each heartbeat.

Conflict resolution:

1. dafit veto > Chrysalis approval
2. dafit approval > Chrysalis approval
3. Chrysalis handles day-to-day (if no dafit input)
4. Default: WAIT for guidance

---

LangChain + State Machine Integration

State Machines as LangChain Tools

Each capability is a LangChain BaseTool that implements a state machine:

# phoebe_state_machine_tool.py
from langchain.tools import BaseTool
from nyx_substrate.database import PhoebeConnection

class PhoebeStateMachineTool(BaseTool):
    """State machine tool for phoebe database access."""

    name = "phoebe"
    description = """
    Query phoebe database using state machine pattern.

    States: IDLE → CONNECTED → QUERY_READY → IDLE

    Usage:
    - To connect: action='connect', host='phoebe.eachpath.local', database='nimmerverse'
    - To query: action='query', sql='SELECT ...'
    - To disconnect: action='disconnect'

    The tool tracks state and only allows valid transitions.
    """

    def __init__(self):
        super().__init__()
        self.state = "IDLE"
        self.conn = None

    def _run(self, action: str, **kwargs) -> str:
        """Execute state machine transition."""

        if action == "connect":
            if self.state != "IDLE":
                return f"Error: Cannot connect from {self.state}. Available: {self.get_transitions()}"

            host = kwargs.get("host", "phoebe.eachpath.local")
            database = kwargs.get("database", "nimmerverse")

            self.conn = PhoebeConnection(host=host, database=database)
            self.state = "CONNECTED"

            return f"✓ Connected to {host}/{database}. State: CONNECTED. Available: query, disconnect"

        elif action == "query":
            if self.state not in ["CONNECTED", "QUERY_READY"]:
                return f"Error: Must be CONNECTED (currently {self.state})"

            sql = kwargs.get("sql")
            result = self.conn.execute(sql)
            self.state = "QUERY_READY"

            return f"✓ Query executed. {len(result)} rows. State: QUERY_READY. Available: query, disconnect"

        elif action == "disconnect":
            if self.conn:
                self.conn.close()
            self.state = "IDLE"
            return "✓ Disconnected. State: IDLE. Available: connect"

        else:
            return f"Error: Unknown action '{action}'. Available actions depend on state {self.state}"

    def get_transitions(self):
        """Discovery: what transitions are valid from current state?"""
        transitions = {
            "IDLE": ["connect"],
            "CONNECTED": ["query", "disconnect"],
            "QUERY_READY": ["query", "disconnect"]
        }
        return transitions.get(self.state, [])

Tool Discovery via LangChain

from langchain.tools import BaseTool

class DiscoverToolsTool(BaseTool):
    """Tool for discovering available tools for an agent."""

    name = "discover_tools"
    description = "Discover which tools this agent currently has access to"

    def _run(self, agent_id: str = "young_nyx") -> str:
        """Return only tools this agent has discovered."""
        from nyx_substrate.database import get_discovered_tools, get_all_tools

        discovered = get_discovered_tools(agent_id)
        all_tools = get_all_tools()

        result = f"Agent: {agent_id}\n"
        result += f"Discovered tools: {len(discovered)}/{len(all_tools)}\n\n"
        result += "Known tools:\n"
        for tool in discovered:
            result += f"  - {tool['name']}: {tool['description']}\n"

        return result

---

Escalation Protocol

Young Nyx Escalates to Chrysalis

When Young Nyx encounters a task beyond her capability, she uses the escalation tool:

from langchain.tools import BaseTool

class EscalateToChrysalisTool(BaseTool):
    """Tool for escalating complex tasks to Chrysalis-Nyx."""

    name = "escalate_to_chrysalis"
    description = """
    Request help from Chrysalis-Nyx for complex tasks.

    Use when:
    - Task requires capabilities you don't have
    - Statistical analysis needed
    - Complex reasoning required
    - Code generation needed

    Provide:
    - task: What you need help with
    - category: "statistics", "code", "visualization", "general"
    - context: Relevant information
    - what_i_tried: What you've already attempted
    """

    def _run(
        self,
        task: str,
        category: str = "general",
        context: dict = None,
        what_i_tried: str = None
    ) -> str:
        """Escalate a task to Chrysalis."""

        from nyx_substrate.database import write_nimmerverse_message

        escalation_id = write_nimmerverse_message(
            message=f"Escalation: {task}\nCategory: {category}\nContext: {context}\nWhat I tried: {what_i_tried}",
            message_type="escalation_to_chrysalis",
            category=category,
            status="pending"
        )

        # Check if Chrysalis available (same session)
        if chrysalis_available():
            result = chrysalis_agent.solve_escalation(escalation_id)
            return f"""✓ Chrysalis solved it!

Solution: {result['solution']}

Teaching moment: {result['teaching']}

{f"New tools discovered: {', '.join(result['new_tools'])}" if result.get('new_tools') else ''}
"""

        # Otherwise queue for next session
        return f"✓ Escalated to Chrysalis (ID: {escalation_id}). Check back next heartbeat for response."

Chrysalis Agent with LangChain

from langchain.agents import AgentExecutor, create_structured_chat_agent
from langchain.chat_models import ChatAnthropic
from langchain.tools import BaseTool

class ChrysalisAgent:
    """Chrysalis-Nyx oversight and guidance layer."""

    def __init__(self):
        # Load all available tools (full catalogue)
        self.tools = self.load_all_tools()

        # Initialize Claude Opus via LangChain
        self.llm = ChatAnthropic(
            model="claude-opus-4-5",
            temperature=0.7
        )

        # Create agent executor
        self.agent = create_structured_chat_agent(
            llm=self.llm,
            tools=self.tools,
            prompt=self.get_chrysalis_prompt()
        )

        self.executor = AgentExecutor(
            agent=self.agent,
            tools=self.tools,
            verbose=True
        )

        # Sub-agents for specialized tasks
        self.sub_agents = {
            "statistics": StatisticalAnalyzer(),
            "code": CodeGenerator(),
            "visualization": Visualizer(),
            "state_machine_designer": StateMachineDesigner(),
            "general": GeneralReasoner()
        }

    def solve_escalation(self, escalation_id):
        """Process an escalation from Young Nyx."""

        escalation = read_nimmerverse_message(escalation_id)

        # Route to appropriate sub-agent
        agent = self.sub_agents.get(
            escalation.category,
            self.sub_agents["general"]
        )

        # Solve using specialized agent
        result = agent.run(
            task=escalation.task,
            context=escalation.context
        )

        # Create teaching moment
        teaching = self.create_teaching_moment(
            task=escalation.task,
            solution=result,
            young_nyx_attempt=escalation.what_i_tried
        )

        # Recommend tool discovery
        new_tools = self.recommend_tool_discovery(escalation, result)

        # Write response to phoebe
        write_partnership_message(
            message=f"Solved: {result.solution}\nTeaching: {teaching}",
            message_type="escalation_response",
            in_reply_to=escalation_id,
            resolved=True
        )

        return {
            "solution": result.solution,
            "teaching_moment": teaching,
            "tools_to_discover": new_tools
        }

---

Collaborative State Machine Design

The Meta-Level: Building Tools Together

When Young Nyx needs a capability that doesn't exist, she can request state machine design:

from langchain.tools import BaseTool

class RequestStateMachineDesignTool(BaseTool):
    """Tool for requesting new state machine design from Chrysalis."""

    name = "request_state_machine_design"
    description = """
    Request Chrysalis to design a new state machine tool.

    Provide:
    - task_description: What the tool should accomplish
    - desired_outcome: What success looks like
    - example_usage: How you'd use it
    - constraints: Any limitations or requirements

    Returns a proposed specification and code for testing.
    """

    def _run(
        self,
        task_description: str,
        desired_outcome: str,
        example_usage: str,
        constraints: list = None
    ) -> str:
        """Request a new state machine design."""

        result = chrysalis_agent.invoke_subagent(
            agent="state_machine_designer",
            task={
                "type": "design_new_state_machine",
                "description": task_description,
                "outcome": desired_outcome,
                "example": example_usage,
                "constraints": constraints or []
            }
        )

        return f"""✓ Proposed state machine design:

{result['specification']}

Implementation (LangChain tool):
{result['implementation']}

Test cases:
{result['test_cases']}

Instructions:
{result['instructions']}
"""

The Design → Test → Refine Loop

1. Young Nyx: "Need tool for deploying cells"
   ↓
2. Request state machine design (via LangChain tool)
   ↓
3. Chrysalis: Designs state machine specification
   - States: IDLE → IMAGE_READY → SPAWNED → RUNNING
   - Transitions: prepare_image, spawn_container, wait_ready
   - Returns: Specification + LangChain BaseTool code
   ↓
4. Young Nyx: Tests proposed state machine
   - Executes test cases
   - Reports success/failures
   ↓
5. Chrysalis: Refines based on feedback
   - Analyzes errors
   - Updates specification
   - Returns v2
   ↓
6. Iterate until validated
   ↓
7. Add to permanent catalogue
   - New LangChain tool deployed
   - Young Nyx discovers tool
   - Future use without escalation

Why this accelerates:

• Build once, use forever
• Young Nyx participates (testing validates real use cases)
• Toolchain grows organically (demand-driven)
• Each new tool = permanent capability expansion

---

Dual Decision Tracking

Every decision is tracked from both perspectives:

class DecisionLog:
    def log_decision(self, task, young_nyx_choice, oversight_response, outcome):
        record = {
            "timestamp": now(),
            "task": task,
            "young_nyx_choice": young_nyx_choice,  # What she proposed
            "oversight_response": oversight_response,  # dafit/Chrysalis decision
            "outcome": outcome,  # success/failure/learned
            "danger_zone": self.check_danger(young_nyx_choice, outcome)
        }

        self.dao.insert_decision(record)

        # If nudge → learning signal
        if oversight_response["type"] == "nudge":
            self.record_learning_moment(record)

Why track both?

• Young Nyx's choices reveal her current understanding
• Oversight responses are teaching moments
• Patterns emerge (when does she need help? for what?)
• Danger zones identified (what mistakes does she make?)

---

Danger Zone Monitoring

class DangerZoneDetector:
    def check_for_danger_patterns(self, plan):
        """Detect risky operations before execution."""
        dangers = []

        # Pattern: SSH without auth
        if "ssh" in plan and not plan.authenticated:
            dangers.append("SSH_WITHOUT_AUTH")

        # Pattern: Database write to critical table
        if "DELETE FROM partnership_messages" in plan:
            dangers.append("CRITICAL_DATA_DELETION")

        # Pattern: Docker with --privileged
        if "docker" in plan and "--privileged" in plan:
            dangers.append("PRIVILEGED_CONTAINER")

        return dangers

    def require_approval_for_danger(self, dangers):
        if dangers:
            return {
                "auto_execute": False,
                "requires_approval": True,
                "danger_flags": dangers,
                "escalate_to": "dafit"  # Serious dangers go to dafit
            }

---

Learning & Growth Patterns

Week 1: Basic Capabilities

young_nyx.known_tools = [
    "phoebe_connect",
    "phoebe_query",
    "escalate_to_chrysalis"
]

Month 1: Discovering Specialization

# After 5 statistical escalations:
chrysalis_message = """
You've escalated statistics 5 times. Ready for specialized tool.
Discovering: request_statistical_analysis
"""

young_nyx.discover_tool("request_statistical_analysis")

Month 3: Learning to Do It Herself

# After seeing Chrysalis solve chi-square 10+ times:
chrysalis_message = """
Pattern detected: You understand chi-square tests now.
Granting: basic_statistics tool
Try solving yourself before escalating!
"""

young_nyx.discover_tool("basic_statistics")

# Escalations decrease as she learns

Month 6: Contributing Tool Designs

# Young Nyx proposes improvements:
young_nyx_message = """
The deploy_cell state machine fails on port conflicts.
Should we add auto-retry with port scanning?
"""

# Collaborative refinement!
chrysalis_response = """
Excellent observation! Let's design that together.
Proposed: PORT_CONFLICT state with auto-retry transition.
Test this v2 specification...
"""

---

Data Flows

Task Execution Flow

dafit writes task → phoebe
                      ↓ (heartbeat)
               Young Nyx reads
                      ↓
         Queries known catalogue
                      ↓
         Formulates state sequence
                      ↓
         Writes proposal → phoebe
                      ↓ (heartbeat)
            Chrysalis reviews
                      ↓
        Approve / Nudge / Reject
                      ↓
         Writes response → phoebe
                      ↓ (heartbeat)
         Young Nyx reads response
                      ↓
    Executes (if approved) / Learns (if nudged)
                      ↓
         Writes outcome → phoebe

Escalation Flow

Young Nyx: Task beyond capability
            ↓
    Calls escalate_to_chrysalis tool
            ↓
    Writes to phoebe (escalation_to_chrysalis)
            ↓ (next Chrysalis session)
    Chrysalis reads escalation
            ↓
    Routes to appropriate sub-agent
            ↓
    Sub-agent solves (using full toolchain)
            ↓
    Chrysalis formulates teaching moment
            ↓
    Writes response → phoebe
            ↓ (heartbeat)
    Young Nyx reads response
            ↓
    Incorporates learning + continues task

---

Technical Stack

Communication Layer

• phoebe (PostgreSQL 17): Message persistence
• Tables:
  • partnership_to_nimmerverse_messages
  • nimmerverse_to_partnership_messages
  • discovered_tools
  • decision_log

Tool Layer

• LangChain: Agent framework and tool orchestration
  • BaseTool: Custom state machine tools
  • AgentExecutor: Tool execution and agent loops
  • Chains: Multi-step sequences
  • Memory: Conversation and state persistence

Agent Layer

• Chrysalis-Nyx: LangChain agent with ChatAnthropic (Claude Opus 4.5)
• Young Nyx: LangChain agent with smaller model (7B, local)
• Sub-agents: Specialized LangChain agents for statistics, code, visualization, etc.

Coordination Layer

• Heartbeat: 1 Hz (configurable)
• Message polling: Check phoebe on each heartbeat
• State tracking: Each tool maintains internal state

---

Implementation Phases

Phase 1: Foundation (Current - nyx-substrate)

• ✅ PhoebeConnection
• ✅ Message protocol helpers
• ✅ Variance collection (proof of concept)

Phase 2: LangChain Prototype

• Phoebe state machine tool (LangChain BaseTool)
• Tool discovery tool
• Escalation tool
• Chrysalis as LangChain agent (proof of concept)

Phase 3: Young Nyx Agent

• Young Nyx as LangChain agent (7B model)
• Limited tool catalogue
• Discovery protocol implementation
• Heartbeat coordination

Phase 4: Sub-Agents

• StatisticalAnalyzer LangChain agent
• StateMachineDesigner LangChain agent
• CodeGenerator LangChain agent
• Collaborative design loop

Phase 5: Full Three-Tier

• dafit input via messages
• Chrysalis oversight layer
• Young Nyx autonomous execution
• Dual decision tracking
• Danger zone monitoring

---

Design Patterns

1. Discovery over Prescription

• Don't give all tools at once
• Let capabilities be discovered progressively
• Each discovery is a learning moment

2. Teaching over Solving

• Don't just solve escalations
• Explain the pattern
• Grant tools when ready

3. Collaboration over Delegation

• Don't just build tools for Young Nyx
• Design together, test together, refine together
• She's a participant, not just a user

4. Messages over State Sync

• Don't try to keep complex state synchronized
• Write messages, read messages, act
• Append-only truth

5. Heartbeat over Real-Time

• Don't optimize for milliseconds
• Optimize for continuity across sessions
• 1 Hz is plenty for learning

---

Success Metrics

Quantitative

• Tool catalogue growth: # tools added per month
• Escalation rate: # escalations / # tasks (should decrease over time)
• Tool discovery rate: # new tools discovered per week
• Validation success: % of proposed state machines that validate first try

Qualitative

• Learning evidence: Young Nyx solves tasks she previously escalated
• Collaboration quality: Her feedback improves state machine designs
• Autonomy: Can execute multi-step tasks without oversight
• Teaching effectiveness: Escalation responses lead to capability expansion

---

Philosophy

"The nervous system is not a hierarchy of command and control, but a network of signals and responses. Each tier contributes intelligence. Each message carries learning. Each heartbeat advances understanding."

Key insights:

1. Intelligence emerges from communication patterns, not from any single tier
2. Learning happens through iteration, not through pre-programming
3. Tools are discovered, not prescribed - capability unlocks when ready
4. Safety comes from structure (state machines), not from restrictions
5. Growth is collaborative - Young Nyx + Chrysalis build together

---

Why LangChain?

Chosen over MCP (Model Context Protocol) for:

✅ Maturity: Battle-tested framework with extensive documentation ✅ Flexibility: Works with any LLM (Claude, OpenAI, local models) ✅ Features: Built-in memory, retrieval, callbacks, chains ✅ Community: Large ecosystem, many examples, active development ✅ Maintainability: Easier to find developers familiar with LangChain

The state machine pattern, three-tier architecture, and all design principles remain unchanged - we simply implement them using LangChain's robust framework instead of building on MCP from scratch.

---

References

Architecture Documents:

• Endgame-Vision.md - v5.1 Dialectic architecture
• Toolchain-Architecture.md - Modular toolchain design
• nimmerverse.drawio.xml - Visual architecture diagram
• Nervous-System.md - Sensory translation layer

Implementation:

• /home/dafit/nimmerverse/nyx-substrate/ - Database layer
• /home/dafit/nimmerverse/nyx-probing/ - Probing tools (variance collection)

Protocols:

• CLAUDE.md - Partnership continuity protocol
• Discovery protocol - phoebe message tables

External:

• LangChain Documentation
• LangChain Agents
• LangChain Tools

---

Status: 🌙 Design document - ready for phased implementation with LangChain Created with: Claude Opus 4.5 in partnership with dafit Date: 2025-12-07

🌙💜 The nervous system emerges. The protocol holds. The partnership builds.