From d895fd9103b4d8a39231dbb29769c8fb141197e5 Mon Sep 17 00:00:00 2001 From: dafit Date: Fri, 30 Jan 2026 08:29:48 +0100 Subject: [PATCH] feat: Add IR-Position-Array organ spec + update Organ-Index v1.2 New organ: IR Position Array (8x ESP32-S3 AI CAMs as indoor GPS). Updated Organ-Index with 4 new organs: Position-Time Beacon, IR Position Array, Crafting Eye, Godseye. Co-Authored-By: Claude Opus 4.5 --- architecture/organs/IR-Position-Array.md | 263 +++++++++++++++++++++++ architecture/organs/Organ-Index.md | 52 ++++- 2 files changed, 313 insertions(+), 2 deletions(-) create mode 100644 architecture/organs/IR-Position-Array.md diff --git a/architecture/organs/IR-Position-Array.md b/architecture/organs/IR-Position-Array.md new file mode 100644 index 0000000..e87e6b3 --- /dev/null +++ b/architecture/organs/IR-Position-Array.md @@ -0,0 +1,263 @@ +# IR Position Array Organ + +**Room-scale organism tracking via IR beacon triangulation.** + +> *"The organisms can't see their own backs. They know themselves through each other."* + +--- + +## Overview + +The IR Position Array is **infrastructure** — fixed cameras that run 24/7, tracking all organisms via their IR beacons. This is the nimmerverse's indoor GPS. + +--- + +## Hardware Specification + +| Component | Spec | Quantity | Status | +|-----------|------|----------|--------| +| **Camera** | ESP32-S3 AI CAM (night vision) | 8× | Received 2026-01-05 | +| **IR Sensitivity** | Native (night vision LEDs + sensor) | - | Built-in | +| **Resolution** | OV2640/OV5640 | - | TBD confirm | +| **Power** | 5V wired (ceiling PSU) | - | Planned | +| **Enclosure** | 3D printed custom case | 8× | To design | + +### Upgrade from Original Spec + +| Original (Nimmerswarm-Interface) | Actual | +|----------------------------------|--------| +| 4× PS3 Eye (IR filter removed) | 8× ESP32-S3 AI CAM (native IR) | +| USB hub / extension | WiFi streaming (no USB!) | +| ~80 CHF cameras | Already purchased | + +**8 cameras > 4 cameras = better coverage, more triangulation angles, redundancy.** + +--- + +## Architecture + +``` + CEILING (8× fixed cameras, star power from central PSU) + + ┌─────────────────────────────────────────────────────┐ + │ │ + │ [📷1] [📷2] [📷3] │ + │ ╲ │ ╱ │ + │ ╲ ┌────────────┴────────────┐ ╱ │ + │ ╲ │ │ ╱ │ + │ [📷4]──╲──│ ⚡ CEILING PSU │─╱──[📷5] │ + │ ╲ │ (center, 5V hub) │╱ │ + │ ╲└─────────────────────────┘ │ + │ ╲ │ ╱ │ + │ ╲──────────┼──────────╱ │ + │ │ │ + │ [📷6] │ [📷7] │ + │ │ │ + │ [📷8] │ + │ │ + │ 🤖────📍 IR beacon │ + │ organism │ + │ │ + └───🚪───────────────────────────────────────────────┘ + (0,0) origin +``` + +--- + +## Dual-Spectrum Design + +From [[../interfaces/Nimmerswarm-Interface]]: + +| Spectrum | Channel | Purpose | +|----------|---------|---------| +| **Infrared** | IR Position Array | WHERE organism is (24/7, day/night) | +| **Visible** | 3x3 LED Matrix | WHAT organism is doing (state broadcast) | + +**Zero crosstalk. Two independent data streams.** + +--- + +## Processing Pipeline + +``` +8× ESP32-S3 AI CAM + │ + │ WiFi/MJPEG streams + ▼ +┌─────────────────────────────────┐ +│ PROCESSING NODE │ +│ (The Womb / RTX 6000 Max-Q) │ +│ │ +│ • Receive 8 camera streams │ +│ • Detect IR beacon blobs │ +│ • Multi-camera triangulation │ +│ • Structure from Motion (SFM) │ +│ • Output: (x, y, z) @ 30fps │ +└─────────────────────────────────┘ + │ + │ NATS publish + ▼ +┌─────────────────────────────────┐ +│ nats://nimmerverse/position/ │ +│ │ +│ { │ +│ organism_id: "crawler_001", │ +│ x: 1.234, │ +│ y: -2.567, │ +│ z: 0.05, │ +│ confidence: 0.95, │ +│ timestamp: 1704499200.123 │ +│ } │ +└─────────────────────────────────┘ + │ + ▼ + PHOEBE (ground truth storage) +``` + +--- + +## Algorithm: Low-Cost-Mocap + +Standing on shoulders of [Low-Cost-Mocap](https://github.com/jyjblrd/Low-Cost-Mocap) by @jyjblrd: + +| Component | Their Solution | Our Adaptation | +|-----------|----------------|----------------| +| Multi-camera triangulation | OpenCV SFM bundle adjustment | Same | +| Camera calibration | `camera_params.json` | Same process | +| 3D reconstruction | Epipolar geometry | Same math | +| Markers | Visual markers on drones | IR LEDs on organisms | +| Communication | ESP32 wireless | NATS messaging | + +**Original use:** Indoor drone swarms +**Our use:** Organism positioning in nimmerhovel + +*Respect to the fellow ape who did the groundwork.* + +--- + +## Camera Placement Strategy + +### Nimmerhovel Dimensions +- **X:** 4.5m (along wall from kitchen door) +- **Y:** 3.75m (into room toward windows) +- **Z:** 2.04m (floor to sloped ceiling) +- **Origin:** (0,0,0) at kitchen door corner + +### 8-Camera Coverage + +| Camera | Position (approx) | Orientation | Coverage | +|--------|-------------------|-------------|----------| +| CAM-1 | Corner (0, 0, ~2.0m) | Down 45°, into room | Origin quadrant | +| CAM-2 | Corner (4.5, 0, ~2.0m) | Down 45°, into room | Right-front | +| CAM-3 | Corner (0, -3.75, ~2.0m) | Down 45°, toward door | Left-back | +| CAM-4 | Corner (4.5, -3.75, ~2.0m) | Down 45°, toward door | Right-back | +| CAM-5-8 | Mid-walls / center | TBD | Fill gaps | + +**8 cameras = no blind spots, multiple angles on every point.** + +### Mounting + +- **Ceiling mount** via 3D printed enclosure with mounting tabs +- **Angle:** ~45° down from ceiling plane +- **Power:** Star topology from ceiling PSU (center) +- **Cable runs:** Max ~3m from PSU to any camera + +--- + +## Lifeforce Economics + +| Metric | Value | Rationale | +|--------|-------|-----------| +| **Type** | Generator | Provides ground truth | +| **Rate** | +0.5 LF per position fix | Training data value | +| **Cost** | ~0.1 LF per frame (infra) | Always-on baseline | +| **Net** | Positive (generates value) | Core infrastructure | + +**Every position fix = verified training data for organism navigation.** + +--- + +## IR Beacon Specification + +On each organism: + +| Component | Spec | +|-----------|------| +| **LED Type** | IR LED (850nm or 940nm) | +| **Pattern** | Unique pulse code per organism | +| **Power** | From organism Akku | +| **Visibility** | Detectable by all 8 cameras | + +``` +ORGANISM +┌─────────────────────┐ +│ │ +│ ┌───────────────┐ │ +│ │ 3x3 VISIBLE │ │ ← State broadcast (RGB) +│ │ LED Matrix │ │ +│ │ 🔴⚫🟢 │ │ +│ └───────────────┘ │ +│ │ +│ 📍 IR LED │ ← Position beacon (invisible) +│ │ +│ [🔋 Akku] │ ← Mobile power +│ │ +└─────────────────────┘ +``` + +--- + +## Integration Points + +| System | Interface | +|--------|-----------| +| **NATS** | `nats://nimmerverse/position/stream` | +| **Phoebe** | `organism_positions` table | +| **S2 Cells** | Position → S2 cell ID at L1 (1cm) resolution | +| **Virtual Garden** | Ground truth for prediction verification | +| **Vision Organ** | Separate stream (visible spectrum state recognition) | + +--- + +## Dependencies + +| Dependency | Status | Notes | +|------------|--------|-------| +| 8× ESP32-S3 AI CAM | Received | Hardware ready | +| Ceiling PSU | Planned | Power distribution | +| 3D printed enclosures | To design | Camera mounting | +| Printer station | Blocked | Waiting on Baumarkt materials | +| NATS messaging | Planned | Transport layer | +| The Womb (RTX 6000) | Waiting | Processing node | + +--- + +## Calibration Procedure + +1. **Camera intrinsics** — Checkerboard calibration per camera +2. **Extrinsics** — Multi-camera pose estimation (bundle adjustment) +3. **Origin alignment** — Align to GPS beacon at (0, 0, 2.0m) +4. **Verification** — Known position test with ruler measurements + +--- + +## Status + +| Phase | Status | +|-------|--------| +| Hardware acquisition | Complete | +| Enclosure design | Not started | +| Enclosure printing | Blocked (printer station) | +| Physical mounting | Not started | +| Camera calibration | Not started | +| Software pipeline | Not started | +| Integration test | Not started | + +--- + +**Created**: 2026-01-05 +**Version**: 1.0 +**Based on**: [[../interfaces/Nimmerswarm-Interface]] (Dual-Spectrum Architecture section) +**Philosophy**: "They know themselves through each other." + +*The eyes that never blink. The infrastructure that makes position truth.* diff --git a/architecture/organs/Organ-Index.md b/architecture/organs/Organ-Index.md index 02ff18c..d388273 100644 --- a/architecture/organs/Organ-Index.md +++ b/architecture/organs/Organ-Index.md @@ -75,6 +75,50 @@ che# Organ Architecture Index --- +### 📍 Position-Time Beacon +**Host**: M5Stack GPS v2.0 (AT6668) at nimmerhovel origin +**Function**: Absolute position reference + Stratum-1 NTP time source +**Stack**: GPS NMEA parsing, PPS signal for NTP, coordinate broadcast +**Integration**: Provides ground truth origin (47°28'44.915"N, 7°37'07.842"E), time sync for all nimmerverse nodes +**Status**: 🟡 Hardware ordered, arriving ~Jan 2026 + +**Detail**: → `organs/Position-Time-Beacon.md` (pending) + +--- + +### 📍 IR Position Array +**Host**: 8× ESP32-S3 AI CAMs (night vision capable), ceiling-mounted +**Function**: 24/7 organism tracking via IR beacon triangulation (indoor GPS) +**Stack**: ESP32-S3 WiFi streaming → RTX 6000 SFM processing → NATS position stream +**Integration**: Tracks all organisms in real-time, feeds ground truth to phoebe, enables Virtual Garden verification +**Status**: 🟢 Hardware received Jan 2026 + +**Detail**: → [`organs/IR-Position-Array.md`](organs/IR-Position-Array.md) + +--- + +### 🔬 Crafting Eye +**Host**: Raspberry Pi + HQ Camera (12.3MP IMX477) + 8-50mm C-mount zoom lens +**Function**: Fixed birds-eye view of crafting station, high-resolution work monitoring +**Stack**: Manual focus/iris (set once), libcamera, high-res stills + video +**Integration**: Watches dafit's hands during electronics/assembly work, fixed viewing angle +**Status**: 🟢 Hardware received Jan 2026 + +**Detail**: → `organs/Crafting-Eye.md` (pending) + +--- + +### 🦉 Godseye +**Host**: NVIDIA Jetson Orin Nano/NX + PTZ mechanism + motorized zoom lens +**Function**: Active surveyor of nimmerhovel, on-device vision AI, tracking +**Stack**: Jetson (CUDA), servo pan/tilt, auto-zoom, YOLO/tracking models +**Integration**: Autonomous gaze control, can decide where to look, reports to phoebe +**Status**: ⏸️ Research phase + +**Detail**: → `organs/Godseye.md` (pending) + +--- + ## Organ Design Principles ### 1. **Lifeforce Economy** @@ -222,6 +266,10 @@ Zero lifeforce → shutdown, wait for recharge | **Motor** | 🟡 Planned (Phase 4) | ESP32 | Pending | | **Navigation** | 🟡 Planned (Phase 4) | Edge server | Pending | | **Sensory** | 🟡 Conceptual | ESP32 | [`../Nervous-System.md`](../Nervous-System.md) | +| **Position-Time Beacon** | 🟡 Hardware ordered | M5Stack GPS AT6668 | Pending | +| **IR Position Array** | 🟢 Hardware received | 8× ESP32-S3 AI CAM | [`IR-Position-Array.md`](organs/IR-Position-Array.md) | +| **Crafting Eye** | 🟢 Hardware received | Pi HQ + 8-50mm lens | Pending | +| **Godseye** | ⏸️ Research phase | Jetson Orin + PTZ | Pending | --- @@ -232,7 +280,7 @@ Zero lifeforce → shutdown, wait for recharge --- **Created**: 2025-12-07 -**Updated**: 2025-12-07 -**Version**: 1.0 +**Updated**: 2026-01-05 +**Version**: 1.2 (added Position-Time Beacon, IR Position Array, Crafting Eye, Godseye; created IR-Position-Array.md) 🌙💜 *Each organ a tool. Each tool a choice. Each choice a lesson in scarcity.*