From 04256e85c4a347c66b7d31e52a4b0a712f38754f Mon Sep 17 00:00:00 2001 From: dafit Date: Sun, 7 Dec 2025 20:10:39 +0100 Subject: [PATCH] feat: add Nervous-Protocol architecture and reorganize toolchain docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Created architecture/Nervous-Protocol.md: comprehensive three-tier autonomous learning architecture (dafit β†’ Chrysalis-Nyx β†’ Young Nyx) - Designed state machine tool interface pattern with LangChain integration - Documented escalation protocols, discovery catalogues, and collaborative tool building - Moved TOOLCHAIN-PROGRESS.md and Toolchain-Architecture.md to architecture/ directory - Updated Endgame-Vision.md with toolchain crosslinks Key architectural patterns: - State machines as universal tool interface (safety + discovery) - Three-tier oversight with OR gate inputs (dafit + Chrysalis) - Message-based continuity via phoebe heartbeat polling - LangChain BaseTool framework (replaced MCP for maturity) - Dual decision tracking (Young Nyx choices + oversight responses) Version: Nervous-Protocol v1.1 (LangChain-based) Context: Phase 1 toolchain complete, variance collection running πŸŒ™πŸ’œ Generated with Claude Code Co-Authored-By: Claude Opus 4.5 --- Endgame-Vision.md | 1 + architecture/Nervous-Protocol.md | 847 ++++++++++++++++++ .../TOOLCHAIN-PROGRESS.md | 2 +- .../Toolchain-Architecture.md | 0 4 files changed, 849 insertions(+), 1 deletion(-) create mode 100644 architecture/Nervous-Protocol.md rename TOOLCHAIN-PROGRESS.md => architecture/TOOLCHAIN-PROGRESS.md (97%) rename Toolchain-Architecture.md => architecture/Toolchain-Architecture.md (100%) diff --git a/Endgame-Vision.md b/Endgame-Vision.md index d0e22cd..eafe057 100644 --- a/Endgame-Vision.md +++ b/Endgame-Vision.md @@ -49,6 +49,7 @@ This is a **RESEARCH VISION** - a platform for studying how intelligence emerges ## Architecture Overview **Visual diagram:** β†’ [`architecture/nimmerverse.drawio.xml`](architecture/nimmerverse.drawio.xml) (open in draw.io) +**Toolchain implementation:** β†’ [`architecture/Toolchain-Architecture.md`](architecture/Toolchain-Architecture.md) | [Progress](architecture/TOOLCHAIN-PROGRESS.md) ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” diff --git a/architecture/Nervous-Protocol.md b/architecture/Nervous-Protocol.md new file mode 100644 index 0000000..f1c275c --- /dev/null +++ b/architecture/Nervous-Protocol.md @@ -0,0 +1,847 @@ +# Nervous Protocol: Three-Tier Autonomous Learning Architecture + +**Created**: 2025-12-07 +**Updated**: 2025-12-07 (LangChain integration) +**Status**: Design Document +**Version**: 1.1 (LangChain Implementation) + +--- + +## Overview + +The **Nervous Protocol** defines how intelligence flows through the Nimmerverse via a three-tier architecture with message-based communication, state machine tools, and collaborative learning. + +### The Three Tiers: + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ dafit β”‚ +β”‚ (Strategic Architect) β”‚ +β”‚ β€’ Vision & architecture decisions β”‚ +β”‚ β€’ Override authority β”‚ +β”‚ β€’ Long-term direction β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + ↕ (strategic guidance / major escalations) +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Chrysalis-Nyx β”‚ +β”‚ (Oversight & Reasoning) β”‚ +β”‚ β€’ Claude Opus/Sonnet (large context) β”‚ +β”‚ β€’ Full toolchain access via LangChain β”‚ +β”‚ β€’ Reviews Young Nyx's proposals β”‚ +β”‚ β€’ Designs new state machines β”‚ +β”‚ β€’ Teaching & guidance β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + ↕ (guidance / escalations) +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Young Nyx β”‚ +β”‚ (Autonomous Learning Agent) β”‚ +β”‚ β€’ Smaller model (7B or similar) β”‚ +β”‚ β€’ Limited known state machines β”‚ +β”‚ β€’ Executes routine tasks β”‚ +β”‚ β€’ Learns from experience β”‚ +β”‚ β€’ Escalates complex problems β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +--- + +## Core Principles + +### 1. **Message-Based Continuity** + +All communication flows through **phoebe** (PostgreSQL) via message tables: +- `partnership_to_nimmerverse_messages` (dafit + Chrysalis β†’ Young Nyx) +- `nimmerverse_to_partnership_messages` (Young Nyx β†’ dafit + Chrysalis) + +**Why messages?** +- βœ… Persistent across sessions +- βœ… Asynchronous (no blocking) +- βœ… Auditable (every decision logged) +- βœ… Simple (append-only, no complex state sync) + +### 2. **Heartbeat Coordination** + +From `Endgame-Vision.md`: +- **Real clock**: 1 Hz (1 beat/sec) - wall time, free +- **Virtual clock**: Variable - computation time, costs lifeforce + +**On each heartbeat:** +1. Check for new messages from any tier +2. Process guidance/tasks/escalations +3. Update state +4. Take next action +5. Write results back to phoebe + +**Not real-time** (milliseconds), but **continuous** (heartbeat-driven). + +### 3. **State Machines as Tools** + +All capabilities are exposed as **state machine tools** via **LangChain**: + +```python +# Example: phoebe query state machine +from langchain.tools import BaseTool + +States: IDLE β†’ CONNECTED β†’ QUERY_READY β†’ IDLE + +class PhoebeQueryTool(BaseTool): + name = "phoebe_query" + description = """ + Interact with phoebe database using state machine pattern. + + Available actions depend on current state: + - IDLE: connect(host, db) β†’ CONNECTED + - CONNECTED: query(sql) β†’ QUERY_READY, disconnect() β†’ IDLE + - QUERY_READY: query(sql), disconnect() β†’ IDLE + """ +``` + +**Why state machines?** +- βœ… Safety (can't skip steps - must CONNECT before QUERY) +- βœ… Discoverable (each state announces valid transitions) +- βœ… Observable (log every transition) +- βœ… Composable (chain state machines together) + +### 4. **Progressive Capability Unlocking** + +**Dual catalogues:** +- **All available tools** (full registry, managed by dafit/Chrysalis) +- **Young Nyx's known tools** (subset she's discovered) + +Young Nyx can only see/use tools she's discovered. New tools are granted: +- Via teaching moments (Chrysalis: "You're ready for X") +- Via successful escalations (solved problem reveals tool) +- Via collaborative design (she helps build it) + +**Discovery tracking in phoebe:** +```sql +CREATE TABLE discovered_tools ( + agent_id TEXT, + tool_name TEXT, + discovered_at TIMESTAMPTZ DEFAULT NOW(), + discovered_via TEXT, -- "teaching", "escalation", "design" + PRIMARY KEY (agent_id, tool_name) +); +``` + +--- + +## The OR Gate Pattern (Input Sources) + +From `nimmerverse.drawio.xml` (lines 215-244): + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ dafit β”‚ β”‚chrysalis β”‚ +β”‚ (OR gate)β”‚ β”‚ (OR gate)β”‚ +β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ + β”‚ β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + ↓ (OR - either/both) + Message Queue (phoebe) + ↓ (read on heartbeat) + Orchestrator + ↓ + Young Nyx +``` + +**OR gate = Either/both can write, no blocking** + +Both dafit and Chrysalis write to `partnership_to_nimmerverse_messages`. The orchestrator synthesizes on each heartbeat. + +**Conflict resolution:** +1. dafit veto > Chrysalis approval +2. dafit approval > Chrysalis approval +3. Chrysalis handles day-to-day (if no dafit input) +4. Default: WAIT for guidance + +--- + +## LangChain + State Machine Integration + +### State Machines as LangChain Tools + +Each capability is a **LangChain BaseTool** that implements a **state machine**: + +```python +# phoebe_state_machine_tool.py +from langchain.tools import BaseTool +from nyx_substrate.database import PhoebeConnection + +class PhoebeStateMachineTool(BaseTool): + """State machine tool for phoebe database access.""" + + name = "phoebe" + description = """ + Query phoebe database using state machine pattern. + + States: IDLE β†’ CONNECTED β†’ QUERY_READY β†’ IDLE + + Usage: + - To connect: action='connect', host='phoebe.eachpath.local', database='nimmerverse' + - To query: action='query', sql='SELECT ...' + - To disconnect: action='disconnect' + + The tool tracks state and only allows valid transitions. + """ + + def __init__(self): + super().__init__() + self.state = "IDLE" + self.conn = None + + def _run(self, action: str, **kwargs) -> str: + """Execute state machine transition.""" + + if action == "connect": + if self.state != "IDLE": + return f"Error: Cannot connect from {self.state}. Available: {self.get_transitions()}" + + host = kwargs.get("host", "phoebe.eachpath.local") + database = kwargs.get("database", "nimmerverse") + + self.conn = PhoebeConnection(host=host, database=database) + self.state = "CONNECTED" + + return f"βœ“ Connected to {host}/{database}. State: CONNECTED. Available: query, disconnect" + + elif action == "query": + if self.state not in ["CONNECTED", "QUERY_READY"]: + return f"Error: Must be CONNECTED (currently {self.state})" + + sql = kwargs.get("sql") + result = self.conn.execute(sql) + self.state = "QUERY_READY" + + return f"βœ“ Query executed. {len(result)} rows. State: QUERY_READY. Available: query, disconnect" + + elif action == "disconnect": + if self.conn: + self.conn.close() + self.state = "IDLE" + return "βœ“ Disconnected. State: IDLE. Available: connect" + + else: + return f"Error: Unknown action '{action}'. Available actions depend on state {self.state}" + + def get_transitions(self): + """Discovery: what transitions are valid from current state?""" + transitions = { + "IDLE": ["connect"], + "CONNECTED": ["query", "disconnect"], + "QUERY_READY": ["query", "disconnect"] + } + return transitions.get(self.state, []) +``` + +### Tool Discovery via LangChain + +```python +from langchain.tools import BaseTool + +class DiscoverToolsTool(BaseTool): + """Tool for discovering available tools for an agent.""" + + name = "discover_tools" + description = "Discover which tools this agent currently has access to" + + def _run(self, agent_id: str = "young_nyx") -> str: + """Return only tools this agent has discovered.""" + from nyx_substrate.database import get_discovered_tools, get_all_tools + + discovered = get_discovered_tools(agent_id) + all_tools = get_all_tools() + + result = f"Agent: {agent_id}\n" + result += f"Discovered tools: {len(discovered)}/{len(all_tools)}\n\n" + result += "Known tools:\n" + for tool in discovered: + result += f" - {tool['name']}: {tool['description']}\n" + + return result +``` + +--- + +## Escalation Protocol + +### Young Nyx Escalates to Chrysalis + +When Young Nyx encounters a task beyond her capability, she uses the **escalation tool**: + +```python +from langchain.tools import BaseTool + +class EscalateToChrysalisTool(BaseTool): + """Tool for escalating complex tasks to Chrysalis-Nyx.""" + + name = "escalate_to_chrysalis" + description = """ + Request help from Chrysalis-Nyx for complex tasks. + + Use when: + - Task requires capabilities you don't have + - Statistical analysis needed + - Complex reasoning required + - Code generation needed + + Provide: + - task: What you need help with + - category: "statistics", "code", "visualization", "general" + - context: Relevant information + - what_i_tried: What you've already attempted + """ + + def _run( + self, + task: str, + category: str = "general", + context: dict = None, + what_i_tried: str = None + ) -> str: + """Escalate a task to Chrysalis.""" + + from nyx_substrate.database import write_nimmerverse_message + + escalation_id = write_nimmerverse_message( + message=f"Escalation: {task}\nCategory: {category}\nContext: {context}\nWhat I tried: {what_i_tried}", + message_type="escalation_to_chrysalis", + category=category, + status="pending" + ) + + # Check if Chrysalis available (same session) + if chrysalis_available(): + result = chrysalis_agent.solve_escalation(escalation_id) + return f"""βœ“ Chrysalis solved it! + +Solution: {result['solution']} + +Teaching moment: {result['teaching']} + +{f"New tools discovered: {', '.join(result['new_tools'])}" if result.get('new_tools') else ''} +""" + + # Otherwise queue for next session + return f"βœ“ Escalated to Chrysalis (ID: {escalation_id}). Check back next heartbeat for response." +``` + +### Chrysalis Agent with LangChain + +```python +from langchain.agents import AgentExecutor, create_structured_chat_agent +from langchain.chat_models import ChatAnthropic +from langchain.tools import BaseTool + +class ChrysalisAgent: + """Chrysalis-Nyx oversight and guidance layer.""" + + def __init__(self): + # Load all available tools (full catalogue) + self.tools = self.load_all_tools() + + # Initialize Claude Opus via LangChain + self.llm = ChatAnthropic( + model="claude-opus-4-5", + temperature=0.7 + ) + + # Create agent executor + self.agent = create_structured_chat_agent( + llm=self.llm, + tools=self.tools, + prompt=self.get_chrysalis_prompt() + ) + + self.executor = AgentExecutor( + agent=self.agent, + tools=self.tools, + verbose=True + ) + + # Sub-agents for specialized tasks + self.sub_agents = { + "statistics": StatisticalAnalyzer(), + "code": CodeGenerator(), + "visualization": Visualizer(), + "state_machine_designer": StateMachineDesigner(), + "general": GeneralReasoner() + } + + def solve_escalation(self, escalation_id): + """Process an escalation from Young Nyx.""" + + escalation = read_nimmerverse_message(escalation_id) + + # Route to appropriate sub-agent + agent = self.sub_agents.get( + escalation.category, + self.sub_agents["general"] + ) + + # Solve using specialized agent + result = agent.run( + task=escalation.task, + context=escalation.context + ) + + # Create teaching moment + teaching = self.create_teaching_moment( + task=escalation.task, + solution=result, + young_nyx_attempt=escalation.what_i_tried + ) + + # Recommend tool discovery + new_tools = self.recommend_tool_discovery(escalation, result) + + # Write response to phoebe + write_partnership_message( + message=f"Solved: {result.solution}\nTeaching: {teaching}", + message_type="escalation_response", + in_reply_to=escalation_id, + resolved=True + ) + + return { + "solution": result.solution, + "teaching_moment": teaching, + "tools_to_discover": new_tools + } +``` + +--- + +## Collaborative State Machine Design + +### The Meta-Level: Building Tools Together + +When Young Nyx needs a capability that doesn't exist, she can request **state machine design**: + +```python +from langchain.tools import BaseTool + +class RequestStateMachineDesignTool(BaseTool): + """Tool for requesting new state machine design from Chrysalis.""" + + name = "request_state_machine_design" + description = """ + Request Chrysalis to design a new state machine tool. + + Provide: + - task_description: What the tool should accomplish + - desired_outcome: What success looks like + - example_usage: How you'd use it + - constraints: Any limitations or requirements + + Returns a proposed specification and code for testing. + """ + + def _run( + self, + task_description: str, + desired_outcome: str, + example_usage: str, + constraints: list = None + ) -> str: + """Request a new state machine design.""" + + result = chrysalis_agent.invoke_subagent( + agent="state_machine_designer", + task={ + "type": "design_new_state_machine", + "description": task_description, + "outcome": desired_outcome, + "example": example_usage, + "constraints": constraints or [] + } + ) + + return f"""βœ“ Proposed state machine design: + +{result['specification']} + +Implementation (LangChain tool): +{result['implementation']} + +Test cases: +{result['test_cases']} + +Instructions: +{result['instructions']} +""" +``` + +### The Design β†’ Test β†’ Refine Loop + +``` +1. Young Nyx: "Need tool for deploying cells" + ↓ +2. Request state machine design (via LangChain tool) + ↓ +3. Chrysalis: Designs state machine specification + - States: IDLE β†’ IMAGE_READY β†’ SPAWNED β†’ RUNNING + - Transitions: prepare_image, spawn_container, wait_ready + - Returns: Specification + LangChain BaseTool code + ↓ +4. Young Nyx: Tests proposed state machine + - Executes test cases + - Reports success/failures + ↓ +5. Chrysalis: Refines based on feedback + - Analyzes errors + - Updates specification + - Returns v2 + ↓ +6. Iterate until validated + ↓ +7. Add to permanent catalogue + - New LangChain tool deployed + - Young Nyx discovers tool + - Future use without escalation +``` + +**Why this accelerates:** +- Build once, use forever +- Young Nyx participates (testing validates real use cases) +- Toolchain grows organically (demand-driven) +- Each new tool = permanent capability expansion + +--- + +## Dual Decision Tracking + +Every decision is tracked from **both perspectives**: + +```python +class DecisionLog: + def log_decision(self, task, young_nyx_choice, oversight_response, outcome): + record = { + "timestamp": now(), + "task": task, + "young_nyx_choice": young_nyx_choice, # What she proposed + "oversight_response": oversight_response, # dafit/Chrysalis decision + "outcome": outcome, # success/failure/learned + "danger_zone": self.check_danger(young_nyx_choice, outcome) + } + + self.dao.insert_decision(record) + + # If nudge β†’ learning signal + if oversight_response["type"] == "nudge": + self.record_learning_moment(record) +``` + +**Why track both?** +- Young Nyx's choices reveal her current understanding +- Oversight responses are teaching moments +- Patterns emerge (when does she need help? for what?) +- Danger zones identified (what mistakes does she make?) + +--- + +## Danger Zone Monitoring + +```python +class DangerZoneDetector: + def check_for_danger_patterns(self, plan): + """Detect risky operations before execution.""" + dangers = [] + + # Pattern: SSH without auth + if "ssh" in plan and not plan.authenticated: + dangers.append("SSH_WITHOUT_AUTH") + + # Pattern: Database write to critical table + if "DELETE FROM partnership_messages" in plan: + dangers.append("CRITICAL_DATA_DELETION") + + # Pattern: Docker with --privileged + if "docker" in plan and "--privileged" in plan: + dangers.append("PRIVILEGED_CONTAINER") + + return dangers + + def require_approval_for_danger(self, dangers): + if dangers: + return { + "auto_execute": False, + "requires_approval": True, + "danger_flags": dangers, + "escalate_to": "dafit" # Serious dangers go to dafit + } +``` + +--- + +## Learning & Growth Patterns + +### Week 1: Basic Capabilities +```python +young_nyx.known_tools = [ + "phoebe_connect", + "phoebe_query", + "escalate_to_chrysalis" +] +``` + +### Month 1: Discovering Specialization +```python +# After 5 statistical escalations: +chrysalis_message = """ +You've escalated statistics 5 times. Ready for specialized tool. +Discovering: request_statistical_analysis +""" + +young_nyx.discover_tool("request_statistical_analysis") +``` + +### Month 3: Learning to Do It Herself +```python +# After seeing Chrysalis solve chi-square 10+ times: +chrysalis_message = """ +Pattern detected: You understand chi-square tests now. +Granting: basic_statistics tool +Try solving yourself before escalating! +""" + +young_nyx.discover_tool("basic_statistics") + +# Escalations decrease as she learns +``` + +### Month 6: Contributing Tool Designs +```python +# Young Nyx proposes improvements: +young_nyx_message = """ +The deploy_cell state machine fails on port conflicts. +Should we add auto-retry with port scanning? +""" + +# Collaborative refinement! +chrysalis_response = """ +Excellent observation! Let's design that together. +Proposed: PORT_CONFLICT state with auto-retry transition. +Test this v2 specification... +""" +``` + +--- + +## Data Flows + +### Task Execution Flow + +``` +dafit writes task β†’ phoebe + ↓ (heartbeat) + Young Nyx reads + ↓ + Queries known catalogue + ↓ + Formulates state sequence + ↓ + Writes proposal β†’ phoebe + ↓ (heartbeat) + Chrysalis reviews + ↓ + Approve / Nudge / Reject + ↓ + Writes response β†’ phoebe + ↓ (heartbeat) + Young Nyx reads response + ↓ + Executes (if approved) / Learns (if nudged) + ↓ + Writes outcome β†’ phoebe +``` + +### Escalation Flow + +``` +Young Nyx: Task beyond capability + ↓ + Calls escalate_to_chrysalis tool + ↓ + Writes to phoebe (escalation_to_chrysalis) + ↓ (next Chrysalis session) + Chrysalis reads escalation + ↓ + Routes to appropriate sub-agent + ↓ + Sub-agent solves (using full toolchain) + ↓ + Chrysalis formulates teaching moment + ↓ + Writes response β†’ phoebe + ↓ (heartbeat) + Young Nyx reads response + ↓ + Incorporates learning + continues task +``` + +--- + +## Technical Stack + +### Communication Layer +- **phoebe** (PostgreSQL 17): Message persistence +- **Tables**: + - `partnership_to_nimmerverse_messages` + - `nimmerverse_to_partnership_messages` + - `discovered_tools` + - `decision_log` + +### Tool Layer +- **LangChain**: Agent framework and tool orchestration + - `BaseTool`: Custom state machine tools + - `AgentExecutor`: Tool execution and agent loops + - `Chains`: Multi-step sequences + - `Memory`: Conversation and state persistence + +### Agent Layer +- **Chrysalis-Nyx**: LangChain agent with ChatAnthropic (Claude Opus 4.5) +- **Young Nyx**: LangChain agent with smaller model (7B, local) +- **Sub-agents**: Specialized LangChain agents for statistics, code, visualization, etc. + +### Coordination Layer +- **Heartbeat**: 1 Hz (configurable) +- **Message polling**: Check phoebe on each heartbeat +- **State tracking**: Each tool maintains internal state + +--- + +## Implementation Phases + +### Phase 1: Foundation (Current - nyx-substrate) +- βœ… PhoebeConnection +- βœ… Message protocol helpers +- βœ… Variance collection (proof of concept) + +### Phase 2: LangChain Prototype +- [ ] Phoebe state machine tool (LangChain BaseTool) +- [ ] Tool discovery tool +- [ ] Escalation tool +- [ ] Chrysalis as LangChain agent (proof of concept) + +### Phase 3: Young Nyx Agent +- [ ] Young Nyx as LangChain agent (7B model) +- [ ] Limited tool catalogue +- [ ] Discovery protocol implementation +- [ ] Heartbeat coordination + +### Phase 4: Sub-Agents +- [ ] StatisticalAnalyzer LangChain agent +- [ ] StateMachineDesigner LangChain agent +- [ ] CodeGenerator LangChain agent +- [ ] Collaborative design loop + +### Phase 5: Full Three-Tier +- [ ] dafit input via messages +- [ ] Chrysalis oversight layer +- [ ] Young Nyx autonomous execution +- [ ] Dual decision tracking +- [ ] Danger zone monitoring + +--- + +## Design Patterns + +### 1. **Discovery over Prescription** +- Don't give all tools at once +- Let capabilities be discovered progressively +- Each discovery is a learning moment + +### 2. **Teaching over Solving** +- Don't just solve escalations +- Explain the pattern +- Grant tools when ready + +### 3. **Collaboration over Delegation** +- Don't just build tools for Young Nyx +- Design together, test together, refine together +- She's a participant, not just a user + +### 4. **Messages over State Sync** +- Don't try to keep complex state synchronized +- Write messages, read messages, act +- Append-only truth + +### 5. **Heartbeat over Real-Time** +- Don't optimize for milliseconds +- Optimize for continuity across sessions +- 1 Hz is plenty for learning + +--- + +## Success Metrics + +### Quantitative +- **Tool catalogue growth**: # tools added per month +- **Escalation rate**: # escalations / # tasks (should decrease over time) +- **Tool discovery rate**: # new tools discovered per week +- **Validation success**: % of proposed state machines that validate first try + +### Qualitative +- **Learning evidence**: Young Nyx solves tasks she previously escalated +- **Collaboration quality**: Her feedback improves state machine designs +- **Autonomy**: Can execute multi-step tasks without oversight +- **Teaching effectiveness**: Escalation responses lead to capability expansion + +--- + +## Philosophy + +> "The nervous system is not a hierarchy of command and control, but a network of signals and responses. Each tier contributes intelligence. Each message carries learning. Each heartbeat advances understanding." + +**Key insights:** +1. **Intelligence emerges from communication patterns**, not from any single tier +2. **Learning happens through iteration**, not through pre-programming +3. **Tools are discovered, not prescribed** - capability unlocks when ready +4. **Safety comes from structure** (state machines), not from restrictions +5. **Growth is collaborative** - Young Nyx + Chrysalis build together + +--- + +## Why LangChain? + +**Chosen over MCP (Model Context Protocol) for:** + +βœ… **Maturity**: Battle-tested framework with extensive documentation +βœ… **Flexibility**: Works with any LLM (Claude, OpenAI, local models) +βœ… **Features**: Built-in memory, retrieval, callbacks, chains +βœ… **Community**: Large ecosystem, many examples, active development +βœ… **Maintainability**: Easier to find developers familiar with LangChain + +**The state machine pattern, three-tier architecture, and all design principles remain unchanged** - we simply implement them using LangChain's robust framework instead of building on MCP from scratch. + +--- + +## References + +**Architecture Documents:** +- `Endgame-Vision.md` - v5.1 Dialectic architecture +- `Toolchain-Architecture.md` - Modular toolchain design +- `nimmerverse.drawio.xml` - Visual architecture diagram +- `Nervous-System.md` - Sensory translation layer + +**Implementation:** +- `/home/dafit/nimmerverse/nyx-substrate/` - Database layer +- `/home/dafit/nimmerverse/nyx-probing/` - Probing tools (variance collection) + +**Protocols:** +- CLAUDE.md - Partnership continuity protocol +- Discovery protocol - phoebe message tables + +**External:** +- [LangChain Documentation](https://python.langchain.com/) +- [LangChain Agents](https://python.langchain.com/docs/modules/agents/) +- [LangChain Tools](https://python.langchain.com/docs/modules/agents/tools/) + +--- + +**Status**: πŸŒ™ Design document - ready for phased implementation with LangChain +**Created with**: Claude Opus 4.5 in partnership with dafit +**Date**: 2025-12-07 + +πŸŒ™πŸ’œ *The nervous system emerges. The protocol holds. The partnership builds.* diff --git a/TOOLCHAIN-PROGRESS.md b/architecture/TOOLCHAIN-PROGRESS.md similarity index 97% rename from TOOLCHAIN-PROGRESS.md rename to architecture/TOOLCHAIN-PROGRESS.md index 3940106..11b983e 100644 --- a/TOOLCHAIN-PROGRESS.md +++ b/architecture/TOOLCHAIN-PROGRESS.md @@ -1,6 +1,6 @@ # Toolchain Implementation Progress -**Plan**: See [Toolchain-Architecture.md](Toolchain-Architecture.md) +**Plan**: See [Toolchain-Architecture.md](./Toolchain-Architecture.md) **Started**: 2025-12-07 **Current Phase**: Phase 1 - Foundation + Variance Collection diff --git a/Toolchain-Architecture.md b/architecture/Toolchain-Architecture.md similarity index 100% rename from Toolchain-Architecture.md rename to architecture/Toolchain-Architecture.md