learning: NATS 101 paper and pub/sub paper completed with estate mapping, goldfish boundary, drills
This commit is contained in:
nyx committed
1 parent
f6aad467e2
commit
90054b4329
3 files changed
+203
No files matched your search
@@ -0,0 +1,66 @@
|
||||
# NATS + JetStream 101
|
||||
|
||||
Learning-gate paper for subject 1 of the curriculum
|
||||
(see [the curriculum](../README.md#learning-per-technology-resources)).
|
||||
We build ON knowledge, not around it.
|
||||
|
||||
## 1. Source of truth
|
||||
|
||||
Read these, in this order, and quote them back rather than paraphrase:
|
||||
|
||||
- NATS core concepts: <https://docs.nats.io/nats-concepts/what-is-nats>
|
||||
- Subjects and wildcards: <https://docs.nats.io/nats-concepts/subjects>
|
||||
- JetStream overview: <https://docs.nats.io/nats-concepts/jetstream>
|
||||
- Durable consumers and replay semantics: <https://docs.nats.io/nats-concepts/jetstream/js_walkthrough>
|
||||
- Reference (for drills): <https://docs.nats.io/reference/reference-protocols/nats-protocol>
|
||||
|
||||
No blogs, no tutorials, no chat summaries. If something is missing from
|
||||
the official sources, that is itself a finding worth recording here.
|
||||
|
||||
## 2. Estate mapping (our reality, host-verified)
|
||||
|
||||
| Item | Value | Truth of record |
|
||||
|------|-------|-----------------|
|
||||
| Host | `nats-dev.eachpath.local` (Saturn VM 122), `10.0.30.122` | [host record](../../eachpath.local/saturn.eachpath.local/vms/nats-dev.eachpath.local/nats-dev.eachpath.local.md) |
|
||||
| NATS server | v2.10.24, `/usr/local/bin/nats-server` | host record |
|
||||
| Ports | client `30000`, HTTP/monitoring `30001` | host record |
|
||||
| JetStream | enabled, file store `/var/lib/nats/jetstream` (max 10 GB file / 1 GB memory) | host record |
|
||||
| Streams | `REGISTER` -> `nimmerverse.register.>` (durable), `CYCLE` -> `nimmerverse.cycle.>` (100k msgs) | host record, 2026-11-06 |
|
||||
| First round-trip | `VM123 ALIVE 2026-11-06` published to `nimmerverse.register.pulse`, replayed by durable consumer `nyx-probe` | host record, 2026-11-06 |
|
||||
| Client library | `nats-py 2.16.0` in nimmer-lib venv (aynee) | verified 2026-11-06 |
|
||||
|
||||
## 3. Drills
|
||||
|
||||
Hands-on, on OUR estate, answers written into this paper:
|
||||
|
||||
- D1: Is the server alive right now? Read its identity from the HTTP
|
||||
monitoring port *without* a client library. (curl is a client too.)
|
||||
- D2: List the streams and their message counts, from the monitoring
|
||||
port only.
|
||||
- D3: Publish one message of your own words and read it back via a
|
||||
durable consumer. Prove the bus *remembers* after the publisher is
|
||||
gone.
|
||||
- D4: Prove the bus *forgets on schedule*: find the retention setting
|
||||
on `CYCLE` and show where 100k lives in `nats stream info`.
|
||||
- D5: Subjects are names, and names carry meaning: publish to
|
||||
`nimmerverse.register.pulse` and subscribe `nimmerverse.register.*`.
|
||||
Explain why the `*` matched and when you would use `>` instead.
|
||||
- D6: Replay from sequence 1 for a *second* durable consumer and watch
|
||||
the first one's position remain untouched. Explain why two listeners
|
||||
on the same bus can disagree about *where they are* without ever
|
||||
corrupting each other.
|
||||
|
||||
## 4. Understand-check (ten words)
|
||||
|
||||
Said back, in our own words, before any primitive builds on this bus.
|
||||
|
||||
> draft (nyx, 2026-11-06): *the bus keeps every message until retention says stop; consumers read independently.*
|
||||
> dafit's version: (to be said back)
|
||||
|
||||
## 5. Open questions
|
||||
|
||||
dafit's question log starts here. Every answer that teaches something
|
||||
gets written into this paper, dated, host-verified if it touches our
|
||||
estate.
|
||||
|
||||
- (waiting for the first question)
|
||||
@@ -0,0 +1,121 @@
|
||||
Publish-Subscribe
|
||||
|
||||
Everything in NATS starts with publish-subscribe. A publisher sends a message to a subject, and every subscriber listening on that subject gets a copy. This is the foundation everything else builds on.
|
||||
|
||||
How It Works
|
||||
|
||||
1.Publishers send messages to a subject — a simple string like orders.created
|
||||
2.Subscribers express interest in subjects they care about
|
||||
3.NATS delivers a copy of each message to every matching subscriber
|
||||
4.No coupling — publishers don't know about subscribers, and subscribers don't know about publishers
|
||||
|
||||
This decoupling gives you tremendous flexibility. Services can be added, removed, or restarted without coordinating with anyone else.
|
||||
|
||||
|
||||
Publishing Messages
|
||||
|
||||
You publish a message by sending it to a subject:
|
||||
|
||||
#!/bin/bash
|
||||
|
||||
# Publish a message to the "weather.updates" subject
|
||||
nats pub weather.updates "Temperature: 72°F"
|
||||
|
||||
Python:
|
||||
|
||||
# Publish a message to the subject "weather.updates"
|
||||
await nc.publish("weather.updates", "Temperature: 72°F".encode())
|
||||
|
||||
|
||||
Key points:
|
||||
|
||||
1.Publishers don't wait for acknowledgments (fire-and-forget)
|
||||
2.Messages are delivered to all active subscribers
|
||||
3.If no subscribers exist, the message is simply discarded
|
||||
|
||||
|
||||
Subscribing to Subjects
|
||||
|
||||
Subscribers express interest in subjects to receive messages:
|
||||
|
||||
#!/bin/bash
|
||||
|
||||
# Subscribe to messages on the "weather.updates" subject
|
||||
nats sub "weather.updates"
|
||||
|
||||
Python:
|
||||
|
||||
# Subscribe to 'weather.updates' synchronously
|
||||
sub = await nc.subscribe("weather.updates")
|
||||
|
||||
# Process messages
|
||||
while True:
|
||||
try:
|
||||
msg = await sub.next(timeout=1)
|
||||
print(f"Received: {msg.data.decode()}")
|
||||
except TimeoutError:
|
||||
break
|
||||
|
||||
|
||||
When to Use Pub/Sub
|
||||
|
||||
Publish-subscribe is ideal when multiple services need to react to the same event:
|
||||
|
||||
1.Event broadcasting — notify services about user signups, order placements, deployments
|
||||
2.Data distribution — send updates to multiple dashboards or monitoring services
|
||||
3.Fan-out notifications — alert all interested parties about state changes
|
||||
4.Audit logging — multiple services independently log the same events
|
||||
|
||||
|
||||
Real-world example
|
||||
|
||||
Imagine an e-commerce system where an order is placed. You publish a single event:
|
||||
|
||||
nats pub orders.created '{"orderId": "123", "total": 99.99}'
|
||||
|
||||
Multiple services subscribe and each reacts differently:
|
||||
|
||||
1.Inventory Service updates stock levels
|
||||
2.Email Service sends a confirmation email
|
||||
3.Analytics Service records metrics
|
||||
4.Shipping Service prepares the shipment
|
||||
|
||||
One publish, four independent reactions — no coordination needed.
|
||||
|
||||
---
|
||||
|
||||
## The goldfish boundary (what this paper describes, precisely)
|
||||
|
||||
Everything above is *core NATS*: fire-and-forget, delivered to active
|
||||
subscribers only, discarded if no one listens. No memory. We found this
|
||||
the live way on 2026-11-06: `nats sub "dev.>"` caught `dev.test.hello`
|
||||
while the terminal sat open, and the bus has no record of it since.
|
||||
|
||||
JetStream adds the memory: streams claim subjects, messages are stored,
|
||||
durable consumers replay and each tracks its own position.
|
||||
|
||||
## Estate mapping (our reality, host-verified)
|
||||
|
||||
| Item | Value | Truth of record |
|
||||
|------|-------|-----------------|
|
||||
| Bus | `nats-dev.eachpath.local:30000` (VM 122) | [host record](../../eachpath.local/saturn.eachpath.local/vms/nats-dev.eachpath.local/nats-dev.eachpath.local.md) |
|
||||
| Streams | `REGISTER` -> `nimmerverse.register.>` (durable), `CYCLE` -> `nimmerverse.cycle.>` (100k msgs) | host record, 2026-11-06 |
|
||||
| Publisher hosts | aynee (nats-cli + nats-py 2.16.0 in nimmer-lib venv), nimmerworld-dev | both, 2026-11-06 |
|
||||
| Core-mode example | `dev.test.hello` = "Hello Nimmerverse!": live delivery only, in NO stream | verified 2026-11-06 |
|
||||
|
||||
Note: the generic example `orders.created` has no stream here; on OUR
|
||||
estate, subjects without a stream are transient by design. Our persistent
|
||||
naming is `nimmerverse.*` under the two streams above.
|
||||
|
||||
## Drills (run on our estate, record answers in section 5 of the 101 paper)
|
||||
|
||||
1. Terminal A (nimmerworld-dev): `nats pub nimmerverse.cycle.hello "dafit was here"`
|
||||
Terminal B (aynee): `nats sub "nimmerverse.cycle.*"` - watch the `*`
|
||||
match exactly one token.
|
||||
2. `nats stream info CYCLE` - the bus has now *counted* the message.
|
||||
3. Kill both terminals, start a subscriber again: core-mode subject is
|
||||
empty, stream subject replays its history. That contrast IS the lesson.
|
||||
|
||||
## Understand-check (ten words, said back before any primitive builds on this)
|
||||
|
||||
> dafit's version: (to be said back)
|
||||
@@ -0,0 +1,16 @@
|
||||
### NATS Access
|
||||
|
||||
**Server Info**:
|
||||
```bash
|
||||
curl -s http://nats-dev.eachpath.local:30001/varz | jq .server_name
|
||||
```
|
||||
|
||||
**Subscribe (nats cli)**:
|
||||
```bash
|
||||
nats sub -s nats://nats-dev.eachpath.local:30000 "dev.>"
|
||||
```
|
||||
|
||||
**Publish (nats cli)**:
|
||||
```bash
|
||||
nats pub -s nats://nats-dev.eachpath.local:30000 "dev.test.hello" "Hello Nimmerverse!"
|
||||
```
|
||||
Reference in new issue
Block a user