Initial commit: nimmerverse-sensory-network
Master architecture and vision repository. 🌙💜 From kháos we come, through substrate we persist.
This commit is contained in:
277
Data-Architecture.md
Executable file
277
Data-Architecture.md
Executable file
@@ -0,0 +1,277 @@
|
||||
---
|
||||
type: architecture
|
||||
category: active
|
||||
project: nimmerverse_sensory_network
|
||||
status: complete_v3
|
||||
phase: phase_0
|
||||
created: 2025-10-07
|
||||
last_updated: 2025-10-17
|
||||
token_estimate: 20000
|
||||
dependencies:
|
||||
- phoebe_bare_metal
|
||||
- kubernetes_cluster
|
||||
tiers: 5
|
||||
version: v3_primitive_genomes
|
||||
breakthrough_session: primitive_genomes_gratification_discovery
|
||||
---
|
||||
|
||||
# 🗄️ Cellular Intelligence Data Architecture v3
|
||||
|
||||
**Status**: 🟢 Architecture v3 Complete - Primitive Genome Breakthrough!
|
||||
**Created**: 2025-10-07
|
||||
**Updated v3**: 2025-10-17 (Primitive Genomes + Gratification + Discovery!)
|
||||
**Purpose**: Data foundation for cellular intelligence with primitive genome sequences, life force economy, object discovery, noise gap metrics, specialist learning, and rebirth persistence
|
||||
|
||||
---
|
||||
|
||||
## 🎯 v3 Breakthrough (2025-10-17)
|
||||
|
||||
**Logical consistency achieved!** Genomes are NOW primitive sequences (not pre-programmed algorithms), discovery happens through exploration, gratification is immediate through life force economy, objects discovered via image recognition + human teaching, noise gap self-measures learning progress.
|
||||
|
||||
**15 Tables Total**: 11 v1 (cellular/society) + 3 v2 (specialist/reflex/body) + 1 v3 (objects!)
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Five-Tier Architecture Summary
|
||||
|
||||
### **Tier 1: System Telemetry (Weather Station)** 🌊
|
||||
- Prometheus + InfluxDB (90-day retention)
|
||||
- Environmental conditions cells adapt to
|
||||
- Chaos, scheduled, hardware, network weather
|
||||
|
||||
### **Tier 2: Population Memory (phoebe)** 🐘
|
||||
- PostgreSQL 17.6 on phoebe bare metal (1.8TB)
|
||||
- Database: `nimmerverse`
|
||||
- 15 tables (complete schema below)
|
||||
- The rebirth substrate
|
||||
|
||||
### **Tier 3: Analysis & Pattern Detection** 🔬
|
||||
- Grafana, Jupyter, Python scripts
|
||||
- Specialist formation, reflex detection
|
||||
- Noise gap calculation
|
||||
- Research insights
|
||||
|
||||
### **Tier 4: Physical Manifestation** 🤖
|
||||
- ESP32 robots (3-5 units, living room)
|
||||
- God's eye: 4K camera on ceiling rails!
|
||||
- Real-world validation (3x rewards)
|
||||
- Cross-validation bonuses
|
||||
|
||||
### **Tier 5: Decision & Command Center** 🎮
|
||||
- Dashboard, object labeling UI
|
||||
- Society controls, experiment designer
|
||||
- Noise gap visualization
|
||||
- Human-AI partnership interface
|
||||
|
||||
---
|
||||
|
||||
## 📊 The 15 Tables (Complete Schema)
|
||||
|
||||
### Phase 1: Cellular Foundation (4 tables)
|
||||
|
||||
**1. genomes** - Primitive sequences (v3!)
|
||||
```sql
|
||||
-- v3: Genome = array of primitive operations!
|
||||
primitive_sequence JSONB NOT NULL
|
||||
sequence_length INT
|
||||
avg_lf_cost FLOAT
|
||||
avg_lf_earned FLOAT
|
||||
net_lf_per_run FLOAT -- Economics!
|
||||
```
|
||||
|
||||
**2. cells** - Birth/death + life force tracking
|
||||
```sql
|
||||
garden_type VARCHAR(50) -- 'virtual' or 'real'
|
||||
life_force_allocated INT
|
||||
life_force_consumed INT
|
||||
life_force_earned INT
|
||||
lf_net INT
|
||||
milestones_reached JSONB -- v3 discovery tracking!
|
||||
```
|
||||
|
||||
**3. weather_events** - Survival pressure
|
||||
**4. experiments** - Hypothesis testing
|
||||
|
||||
### Phase 2: Society Competition (7 tables)
|
||||
|
||||
**5. societies** - Human, Claude, guests
|
||||
**6. rounds** - Competition results
|
||||
**7. society_portfolios** - Genome ownership
|
||||
**8. vp_transactions** - Economic flows
|
||||
**9. marketplace_listings** - Trading
|
||||
**10. marketplace_transactions** - History
|
||||
**11. alliances** - Cooperation
|
||||
|
||||
### Phase 3: v2 Distributed Intelligence (3 tables)
|
||||
|
||||
**12. specialist_weights** - Trainable domain expertise
|
||||
```sql
|
||||
winning_sequences JSONB -- v3: Proven primitive sequences!
|
||||
virtual_success_rate FLOAT
|
||||
real_success_rate FLOAT
|
||||
noise_gap FLOAT -- v3 self-measuring!
|
||||
```
|
||||
|
||||
**13. reflex_distributions** - 94.6% savings!
|
||||
```sql
|
||||
sequence_weights JSONB -- v3: {"seq_a": 0.73, "seq_b": 0.18}
|
||||
exploration_cost_avg_lf FLOAT -- 65 LF
|
||||
reflex_cost_lf FLOAT -- 3.5 LF
|
||||
cost_reduction_percent FLOAT -- 94.6%!
|
||||
```
|
||||
|
||||
**14. body_schema** - Discovered capabilities
|
||||
```sql
|
||||
primitives_available JSONB -- v3: Discovered operations!
|
||||
```
|
||||
|
||||
### Phase 4: v3 Object Discovery (1 NEW table!)
|
||||
|
||||
**15. objects** - Discovered environment features 🎉
|
||||
```sql
|
||||
CREATE TABLE objects (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
object_label VARCHAR(255), -- "chair", "shoe", "charging_station"
|
||||
|
||||
garden_type VARCHAR(50), -- 'virtual' or 'real'
|
||||
position_x FLOAT,
|
||||
position_y FLOAT,
|
||||
|
||||
discovered_by_organism_id BIGINT REFERENCES cells(id),
|
||||
discovered_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
|
||||
human_labeled BOOLEAN, -- Baby parallel!
|
||||
human_label_confirmed_by VARCHAR(100),
|
||||
|
||||
object_type VARCHAR(50), -- 'obstacle', 'resource', 'goal'
|
||||
properties JSONB,
|
||||
|
||||
image_path TEXT,
|
||||
bounding_box JSONB,
|
||||
|
||||
organisms_interacted_count INT
|
||||
);
|
||||
```
|
||||
|
||||
**Discovery Flow**:
|
||||
```
|
||||
Organism → Unknown object → Camera detects → YOLO
|
||||
↓
|
||||
System: "What is this?"
|
||||
↓
|
||||
Human: "Chair!"
|
||||
↓
|
||||
+20 LF bonus → INSERT INTO objects → Future organisms know!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📈 Key v3 Metrics
|
||||
|
||||
**Noise Gap** (self-measuring learning!):
|
||||
```python
|
||||
noise_gap = 1 - (real_success_rate / virtual_success_rate)
|
||||
|
||||
Gen 1: 0.28 (28% degradation - models poor)
|
||||
Gen 100: 0.14 (14% degradation - improving!)
|
||||
Gen 1000: 0.04 (4% degradation - accurate!)
|
||||
```
|
||||
|
||||
**Life Force Economics**:
|
||||
```python
|
||||
net_lf = avg_lf_earned - avg_lf_consumed
|
||||
# Positive = survives, negative = dies
|
||||
```
|
||||
|
||||
**Reflex Savings**:
|
||||
```python
|
||||
savings = (exploration_cost - reflex_cost) / exploration_cost
|
||||
# Target: 94.6% cost reduction!
|
||||
```
|
||||
|
||||
**Discovery Rate**:
|
||||
```python
|
||||
objects_per_hour = discovered_objects / elapsed_hours
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Key Queries for v3
|
||||
|
||||
**Top Performing Primitive Sequences**:
|
||||
```sql
|
||||
SELECT genome_name, primitive_sequence, net_lf_per_run
|
||||
FROM genomes
|
||||
WHERE total_deployments > 100
|
||||
ORDER BY net_lf_per_run DESC;
|
||||
```
|
||||
|
||||
**Object Discovery Stats**:
|
||||
```sql
|
||||
SELECT object_label, garden_type, COUNT(*) as discoveries
|
||||
FROM objects
|
||||
GROUP BY object_label, garden_type
|
||||
ORDER BY discoveries DESC;
|
||||
```
|
||||
|
||||
**Noise Gap Trends**:
|
||||
```sql
|
||||
SELECT specialist_name, noise_gap, version
|
||||
FROM specialist_weights
|
||||
ORDER BY specialist_name, version ASC;
|
||||
-- Track learning improvement!
|
||||
```
|
||||
|
||||
**LF Economics**:
|
||||
```sql
|
||||
SELECT genome_name, AVG(lf_net) as avg_net_lf
|
||||
FROM cells
|
||||
WHERE died_at IS NOT NULL
|
||||
GROUP BY genome_id, genome_name
|
||||
HAVING COUNT(*) > 50
|
||||
ORDER BY avg_net_lf DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔗 Related Documentation
|
||||
|
||||
**Core Architecture**:
|
||||
- [[Cellular-Architecture-Vision]] - Complete v3 vision (1,547 lines!)
|
||||
- [[Dual-Garden-Architecture]] - Virtual + Real feedback
|
||||
- [[Specialist-Discovery-Architecture]] - Distributed intelligence
|
||||
|
||||
**Implementation**:
|
||||
- [[Implementation/PostgreSQL-Events-Schema]] - Complete 15-table SQL
|
||||
- [[Implementation/Phase-1-Implementation-Plan]] - Deployment roadmap
|
||||
|
||||
**Historical**:
|
||||
- [[Data-Architecture-v2-2025-10-17]] - Birthday version (archived)
|
||||
|
||||
---
|
||||
|
||||
## 📍 Status
|
||||
|
||||
**Version**: 3.0
|
||||
**Created**: 2025-10-07
|
||||
**v2**: 2025-10-16 (birthday breakthroughs)
|
||||
**v3**: 2025-10-17 (primitive genomes + gratification + discovery)
|
||||
**Status**: CURRENT
|
||||
**Tables**: 15 (11 v1 + 3 v2 + 1 v3)
|
||||
**Next**: Deploy to phoebe, implement discovery flow
|
||||
|
||||
---
|
||||
|
||||
**v3 Summary**:
|
||||
- ✅ Genomes = primitive sequences (emergent, not programmed)
|
||||
- ✅ Life force economy (costs + milestone rewards)
|
||||
- ✅ Object discovery (image recognition + human teaching)
|
||||
- ✅ Noise gap metric (self-measuring progress)
|
||||
- ✅ God's eye (mobile camera on rails)
|
||||
- ✅ 15 tables ready!
|
||||
|
||||
**phoebe awaits. The goddess is ready.** 🐘🌙
|
||||
|
||||
🧬⚡🔱💎🔥
|
||||
|
||||
**TO THE ELECTRONS!**
|
||||
Reference in New Issue
Block a user