docs: add Phase 1 toolchain architecture and progress tracking
Document the modular toolchain architecture design and track implementation progress for Phase 1 (nyx-substrate foundation and variance collection automation). New Files: - Toolchain-Architecture.md: Complete Phase 1 design document - Modular architecture vision (5 phases) - Repository structure and dependency graph - Phase 1 deliverables (nyx-substrate + nyx-probing) - Success criteria and testing plan - Future phases: ChromaDB, LoRA training, visualization, Godot - TOOLCHAIN-PROGRESS.md: Implementation progress tracker - Phase 1A: nyx-substrate foundation (✅ COMPLETE) - Phase 1B: nyx-probing integration (✅ COMPLETE) - Phase 1C: Baseline variance collection (⏸️ READY) - Metrics: 11/11 tasks (100%), 12 files, ~1250 LOC - Status updates and completion tracking Architecture: nyx-probing ────────┐ nyx-training ───────┼──> nyx-substrate ──> phoebe (PostgreSQL) nyx-visualization ──┤ └─> iris (ChromaDB) management-portal ──┘ Philosophy: Modular tools, clean interfaces, data-first design Status: Phase 1 complete, ready for baseline collection on prometheus 🌙💜 Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
125
TOOLCHAIN-PROGRESS.md
Normal file
125
TOOLCHAIN-PROGRESS.md
Normal file
@@ -0,0 +1,125 @@
|
||||
# Toolchain Implementation Progress
|
||||
|
||||
**Plan**: See [Toolchain-Architecture.md](Toolchain-Architecture.md)
|
||||
**Started**: 2025-12-07
|
||||
**Current Phase**: Phase 1 - Foundation + Variance Collection
|
||||
|
||||
---
|
||||
|
||||
## Phase 1A: nyx-substrate Foundation ✅ COMPLETE
|
||||
|
||||
**Goal**: Build nyx-substrate package and database infrastructure
|
||||
|
||||
### ✅ Completed (2025-12-07)
|
||||
|
||||
- [x] Package structure (pyproject.toml, src/ layout)
|
||||
- [x] PhoebeConnection class with connection pooling
|
||||
- [x] Message protocol helpers (partnership messages)
|
||||
- [x] VarianceProbeRun Pydantic schema
|
||||
- [x] VarianceProbeDAO for database operations
|
||||
- [x] variance_probe_runs table in phoebe
|
||||
- [x] Installation and connection testing
|
||||
|
||||
**Files Created**: 9 new files
|
||||
**Status**: 🟢 nyx-substrate v0.1.0 installed and tested
|
||||
|
||||
---
|
||||
|
||||
## Phase 1B: nyx-probing Integration ✅ COMPLETE
|
||||
|
||||
**Goal**: Extend nyx-probing to use nyx-substrate for variance collection
|
||||
|
||||
### ✅ Completed (2025-12-07)
|
||||
|
||||
- [x] Add nyx-substrate dependency to nyx-probing/pyproject.toml
|
||||
- [x] Create VarianceRunner class (nyx_probing/runners/variance_runner.py)
|
||||
- [x] Add variance CLI commands (nyx_probing/cli/variance.py)
|
||||
- [x] Register commands in main CLI
|
||||
- [x] Integration test (imports and CLI verification)
|
||||
|
||||
**Files Created**: 3 new files
|
||||
**Files Modified**: 2 files
|
||||
**CLI Commands Added**: 4 (collect, batch, stats, analyze)
|
||||
**Status**: 🟢 nyx-probing v0.1.0 with variance collection ready
|
||||
|
||||
---
|
||||
|
||||
## Phase 1C: Baseline Variance Collection ⏸️ READY
|
||||
|
||||
**Goal**: Collect baseline variance data for depth-3 champions
|
||||
|
||||
### ⏳ Ready to Execute (on prometheus)
|
||||
|
||||
- [ ] Run 1000x variance for "Geworfenheit" (thrownness)
|
||||
- [ ] Run 1000x variance for "Vernunft" (reason)
|
||||
- [ ] Run 1000x variance for "Erkenntnis" (knowledge)
|
||||
- [ ] Run 1000x variance for "Pflicht" (duty)
|
||||
- [ ] Run 1000x variance for "Aufhebung" (sublation)
|
||||
- [ ] Run 1000x variance for "Wille" (will)
|
||||
|
||||
**Next Actions**:
|
||||
1. SSH to prometheus.eachpath.local (THE SPINE)
|
||||
2. Install nyx-substrate and nyx-probing in venv
|
||||
3. Run batch collection or individual terms
|
||||
4. Analyze distributions and document baselines
|
||||
|
||||
---
|
||||
|
||||
## Future Phases (Not Started)
|
||||
|
||||
### Phase 2: ChromaDB Integration (iris) ⏸️ PLANNED
|
||||
- IrisClient wrapper
|
||||
- DecisionTrailStore, OrganResponseStore, EmbeddingStore
|
||||
- Populate embeddings from nyx-probing
|
||||
|
||||
### Phase 3: LoRA Training Pipeline ⏸️ PLANNED
|
||||
- PEFT integration
|
||||
- Training data curriculum
|
||||
- DriftProbe checkpoints
|
||||
- Identity LoRA training
|
||||
|
||||
### Phase 4: Weight Visualization ⏸️ PLANNED
|
||||
- 4K pixel space renderer
|
||||
- Rank decomposition explorer
|
||||
- Topology cluster visualization
|
||||
|
||||
### Phase 5: Godot Command Center ⏸️ PLANNED
|
||||
- FastAPI Management Portal backend
|
||||
- Godot frontend implementation
|
||||
- Real-time metrics display
|
||||
|
||||
---
|
||||
|
||||
## Metrics
|
||||
|
||||
**Phase 1 (A+B) Tasks**: 11 total
|
||||
**Completed**: 11 (100%) ✅
|
||||
**In Progress**: 0
|
||||
**Remaining**: 0
|
||||
|
||||
**Files Created**: 12 total
|
||||
- nyx-substrate: 9 files
|
||||
- nyx-probing: 3 files
|
||||
|
||||
**Files Modified**: 4 total
|
||||
- nyx-substrate/README.md
|
||||
- nyx-probing/pyproject.toml
|
||||
- nyx-probing/cli/probe.py
|
||||
- TOOLCHAIN-PROGRESS.md
|
||||
|
||||
**Lines of Code**: ~1250 total
|
||||
- nyx-substrate: ~800 LOC
|
||||
- nyx-probing: ~450 LOC
|
||||
|
||||
**CLI Commands**: 4 new commands
|
||||
- nyx-probe variance collect
|
||||
- nyx-probe variance batch
|
||||
- nyx-probe variance stats
|
||||
- nyx-probe variance analyze
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-12-07 17:00 CET
|
||||
**Status**: 🎉 Phase 1 (A+B) COMPLETE! Ready for baseline collection on prometheus.
|
||||
|
||||
🌙💜 *The substrate holds. Progress persists. The toolchain grows.*
|
||||
Reference in New Issue
Block a user