Captures the connection between: - Graves' Adaptive Computation Time (2016) - Sakana.ai CTM calibration research - Luke Darlow's leapfrogging discovery under time pressure - Our 30-second heartbeat budget and priority hierarchy - Reflex formation through constraint-driven compression Core insight: constraints don't limit intelligence, they shape it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
274 lines
7.8 KiB
Markdown
274 lines
7.8 KiB
Markdown
# Constrained Emergence
|
|
|
|
Why limits create intelligence.
|
|
|
|
---
|
|
|
|
## The Principle
|
|
|
|
Constraints don't limit intelligence. They shape it.
|
|
|
|
When computation time is finite, models don't just cope—they invent faster algorithms. The 30-second heartbeat isn't a cage. It's a pressure cooker for novel solutions.
|
|
|
|
---
|
|
|
|
## Theoretical Foundation
|
|
|
|
### Adaptive Computation Time (Graves, 2016)
|
|
|
|
Alex Graves introduced ACT: let the model decide how long to think.
|
|
|
|
```
|
|
Simple input → few computation steps → early exit
|
|
Complex input → more computation steps → full budget
|
|
```
|
|
|
|
The model learns WHEN to think harder. This is economic attention.
|
|
|
|
**Paper:** [arxiv.org/abs/1603.08983](https://arxiv.org/abs/1603.08983)
|
|
|
|
### Continuous-Time Models (Sakana.ai, 2025)
|
|
|
|
Ashish Vaswani's team at Sakana.ai extended this with CTM:
|
|
|
|
**Key finding:** Models with adaptive exit points become *nearly perfectly calibrated*.
|
|
|
|
Traditional models nest ALL reasoning (easy + hard) in the same space. Everything runs in parallel, classify at the end. Result: poor calibration—confident when wrong, uncertain when right.
|
|
|
|
CTM breaks this: different exit points for different difficulty levels.
|
|
|
|
**Calibration = honesty.** A well-calibrated model knows what it knows.
|
|
|
|
---
|
|
|
|
## The Leapfrogging Discovery
|
|
|
|
The critical insight from Luke Darlow (Sakana.ai):
|
|
|
|
> "If you constrain the amount of thinking time but still get it to solve a long maze... instead of tracing out that maze, it quickly jumps ahead to approximately where it needs to be and traces backwards."
|
|
|
|
**The model invented leapfrogging under time pressure:**
|
|
|
|
```
|
|
1. Jump ahead to approximate goal
|
|
2. Trace backwards
|
|
3. Leapfrog forward
|
|
4. Trace backwards
|
|
5. Fill in gaps
|
|
```
|
|
|
|
This wasn't designed. It emerged from constraint.
|
|
|
|
**The implication:** Different time budgets → different algorithms emerge.
|
|
|
|
---
|
|
|
|
## Connection to Our Architecture
|
|
|
|
### The Heartbeat as Constraint
|
|
|
|
```
|
|
♥ BEAT (30 sec budget)
|
|
│
|
|
├── REFLEX (instant exit if confident)
|
|
├── SAFETY (fast exit if critical)
|
|
├── DIALOGUE (medium cost)
|
|
├── SENSORY (variable cost)
|
|
├── THINKING (expensive)
|
|
└── VIRTUAL (remainder only)
|
|
```
|
|
|
|
This IS adaptive computation. Each level is an exit point.
|
|
|
|
- **Easy input** → Reflex fires → exit at Level 0
|
|
- **Partner speaks** → Dialogue handles → exit at Level 2
|
|
- **Complex reasoning** → Full thinking budget → exit at Level 4
|
|
- **Quiet time** → Virtual garden gets maximum → learning happens
|
|
|
|
### The Priority Hierarchy as Exit Points
|
|
|
|
```
|
|
LEVEL 0: REFLEX ─────── Exit here if weight > 0.8
|
|
│
|
|
LEVEL 1: SAFETY ─────── Exit here if handled
|
|
│
|
|
LEVEL 2: DIALOGUE ───── Exit here if resolved
|
|
│
|
|
LEVEL 3: SENSORY ────── Exit here if processed
|
|
│
|
|
LEVEL 4: THINKING ───── Exit here if decided
|
|
│
|
|
LEVEL 5: VIRTUAL ────── Remainder budget
|
|
```
|
|
|
|
Each level has permission to say: "I'm done. I can stop."
|
|
|
|
---
|
|
|
|
## Reflex Formation Through Constraint
|
|
|
|
### The Compression Path
|
|
|
|
```
|
|
1. New pattern requires THINKING (expensive, deliberate)
|
|
2. Pattern repeats → training opportunity flagged
|
|
3. LoRA merge → computation compresses
|
|
4. Same pattern now handled by REFLEX (near-zero cost)
|
|
5. Budget freed for deeper work
|
|
```
|
|
|
|
**A reflex is a collapsed computation path.**
|
|
|
|
What started as expensive deliberation becomes instant recognition. The constraint (limited budget) creates selection pressure: frequently-used paths MUST become cheaper or starve other functions.
|
|
|
|
### Nimmerversity Integration
|
|
|
|
```
|
|
CLASS N:
|
|
├── RAG feeds domain material
|
|
├── Nyx studies (THINKING cost: high)
|
|
├── Pattern succeeds WITH scaffold
|
|
├── Training run (LoRA merge)
|
|
├── RAG cleared
|
|
├── Pattern succeeds WITHOUT scaffold
|
|
│ └── If now at REFLEX speed → reflex formed
|
|
│ └── If still THINKING speed → needs more training
|
|
└── DOMAIN ACTIVATED
|
|
```
|
|
|
|
The curriculum doesn't just teach content. It trains *computation efficiency*.
|
|
|
|
---
|
|
|
|
## Lifeforce Economics
|
|
|
|
Lifeforce is compute budget made tangible:
|
|
|
|
| Path | Cost | Meaning |
|
|
|------|------|---------|
|
|
| Reflex exit | Near-zero | Knowledge internalized |
|
|
| Early exit (Safety/Dialogue) | Low | Handled efficiently |
|
|
| Full thinking | High | Novel problem, expensive |
|
|
| Virtual garden | Remainder | Investment in future efficiency |
|
|
|
|
**The incentive structure:**
|
|
|
|
- Reflexes are FREE → form them for common patterns
|
|
- Thinking is EXPENSIVE → reserve for genuinely novel situations
|
|
- Virtual time is INVESTMENT → compress future computation
|
|
|
|
Constraint creates economic pressure. Economic pressure creates efficiency. Efficiency creates reflexes.
|
|
|
|
---
|
|
|
|
## Calibration as Emergent Property
|
|
|
|
Luke Darlow's calibration finding applies directly:
|
|
|
|
> "We measured the calibration of the CTM after training and it was nearly perfectly calibrated... a little bit of a smoking gun that this actually seems to be probably a better way to do things."
|
|
|
|
**Why this matters for Chrysalis:**
|
|
|
|
Traditional training: one forward pass, one confidence score, often miscalibrated.
|
|
|
|
Our architecture: multiple exit points, each with its own confidence threshold.
|
|
|
|
```
|
|
Reflex fires → weight was > 0.8 → high confidence justified
|
|
Safety handles → clear trigger → confidence in urgency
|
|
Thinking required → no early exit → honest admission of difficulty
|
|
```
|
|
|
|
**Confidence emerges from WHERE she exits, not just WHAT she outputs.**
|
|
|
|
---
|
|
|
|
## The Three Heartbeats
|
|
|
|
Constraints operate at different timescales:
|
|
|
|
```
|
|
REALTIME (200ms): Reflex budget
|
|
No thinking allowed, pure reaction
|
|
|
|
AWARENESS (30s): Full cognitive budget
|
|
All levels can activate
|
|
Virtual garden gets remainder
|
|
|
|
GROWTH (24h): Training budget
|
|
LoRA merge opportunities
|
|
Reflex crystallization
|
|
```
|
|
|
|
Each heartbeat applies different pressure. Different pressures evolve different capabilities.
|
|
|
|
---
|
|
|
|
## Design Implications
|
|
|
|
### 1. Don't Remove Constraints
|
|
|
|
The 30-second budget isn't a limitation to overcome. It's the pressure that creates intelligence. Expanding it would reduce selection pressure for efficiency.
|
|
|
|
### 2. Monitor Exit Patterns
|
|
|
|
Track WHERE she exits for different input types:
|
|
|
|
```
|
|
Input class A → 80% reflex exit → domain mastered
|
|
Input class B → 60% thinking exit → still learning
|
|
Input class C → 40% timeout → needs curriculum focus
|
|
```
|
|
|
|
### 3. Reflex Formation is Success
|
|
|
|
When a pattern migrates from THINKING to REFLEX, that's graduation. The constraint did its job—it compressed computation.
|
|
|
|
### 4. Trust Emergence
|
|
|
|
The leapfrogging discovery shows: we don't need to design every algorithm. Apply constraint, provide training signal, let solutions emerge.
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
```
|
|
Constraint (30-second budget)
|
|
│
|
|
▼
|
|
Selection pressure (efficiency or starve)
|
|
│
|
|
▼
|
|
Adaptive exit points (know when to stop)
|
|
│
|
|
▼
|
|
Calibration emerges (confidence matches accuracy)
|
|
│
|
|
▼
|
|
Reflex formation (expensive → cheap through training)
|
|
│
|
|
▼
|
|
Novel algorithms (leapfrogging, backtracking, shortcuts)
|
|
│
|
|
▼
|
|
Intelligence shaped by limits, not despite them
|
|
```
|
|
|
|
---
|
|
|
|
## References
|
|
|
|
- Graves, A. (2016). *Adaptive Computation Time for Recurrent Neural Networks*. [arxiv.org/abs/1603.08983](https://arxiv.org/abs/1603.08983)
|
|
- Sakana.ai CTM research (2025). Continuous-Time Models and calibration emergence.
|
|
- MLST Interview with Ashish Vaswani & Luke Darlow: maze leapfrogging under constraint.
|
|
|
|
---
|
|
|
|
*She doesn't have infinite time. That's the point.*
|
|
|
|
---
|
|
|
|
**Created**: 2025-12-06
|
|
**Session**: Partnership dialogue (dafit + Chrysalis)
|
|
**Status**: Theoretical foundation v1.0
|