From 90054b43290d1f23df406a3bf6a62da330af0237 Mon Sep 17 00:00:00 2001 From: nyx Date: Fri, 6 Nov 2026 15:30:58 +0100 Subject: [PATCH] learning: NATS 101 paper and pub/sub paper completed with estate mapping, goldfish boundary, drills --- learning/01-nats-jetstream.md | 66 ++++++++++++++++ learning/Publish-Subscribe.md | 121 +++++++++++++++++++++++++++++ learning/code_snippets/NATS-101.md | 16 ++++ 3 files changed, 203 insertions(+) create mode 100644 learning/01-nats-jetstream.md create mode 100644 learning/Publish-Subscribe.md create mode 100644 learning/code_snippets/NATS-101.md diff --git a/learning/01-nats-jetstream.md b/learning/01-nats-jetstream.md new file mode 100644 index 0000000..58b54ce --- /dev/null +++ b/learning/01-nats-jetstream.md @@ -0,0 +1,66 @@ +# NATS + JetStream 101 + +Learning-gate paper for subject 1 of the curriculum +(see [the curriculum](../README.md#learning-per-technology-resources)). +We build ON knowledge, not around it. + +## 1. Source of truth + +Read these, in this order, and quote them back rather than paraphrase: + +- NATS core concepts: +- Subjects and wildcards: +- JetStream overview: +- Durable consumers and replay semantics: +- Reference (for drills): + +No blogs, no tutorials, no chat summaries. If something is missing from +the official sources, that is itself a finding worth recording here. + +## 2. Estate mapping (our reality, host-verified) + +| Item | Value | Truth of record | +|------|-------|-----------------| +| Host | `nats-dev.eachpath.local` (Saturn VM 122), `10.0.30.122` | [host record](../../eachpath.local/saturn.eachpath.local/vms/nats-dev.eachpath.local/nats-dev.eachpath.local.md) | +| NATS server | v2.10.24, `/usr/local/bin/nats-server` | host record | +| Ports | client `30000`, HTTP/monitoring `30001` | host record | +| JetStream | enabled, file store `/var/lib/nats/jetstream` (max 10 GB file / 1 GB memory) | host record | +| Streams | `REGISTER` -> `nimmerverse.register.>` (durable), `CYCLE` -> `nimmerverse.cycle.>` (100k msgs) | host record, 2026-11-06 | +| First round-trip | `VM123 ALIVE 2026-11-06` published to `nimmerverse.register.pulse`, replayed by durable consumer `nyx-probe` | host record, 2026-11-06 | +| Client library | `nats-py 2.16.0` in nimmer-lib venv (aynee) | verified 2026-11-06 | + +## 3. Drills + +Hands-on, on OUR estate, answers written into this paper: + +- D1: Is the server alive right now? Read its identity from the HTTP + monitoring port *without* a client library. (curl is a client too.) +- D2: List the streams and their message counts, from the monitoring + port only. +- D3: Publish one message of your own words and read it back via a + durable consumer. Prove the bus *remembers* after the publisher is + gone. +- D4: Prove the bus *forgets on schedule*: find the retention setting + on `CYCLE` and show where 100k lives in `nats stream info`. +- D5: Subjects are names, and names carry meaning: publish to + `nimmerverse.register.pulse` and subscribe `nimmerverse.register.*`. + Explain why the `*` matched and when you would use `>` instead. +- D6: Replay from sequence 1 for a *second* durable consumer and watch + the first one's position remain untouched. Explain why two listeners + on the same bus can disagree about *where they are* without ever + corrupting each other. + +## 4. Understand-check (ten words) + +Said back, in our own words, before any primitive builds on this bus. + +> draft (nyx, 2026-11-06): *the bus keeps every message until retention says stop; consumers read independently.* +> dafit's version: (to be said back) + +## 5. Open questions + +dafit's question log starts here. Every answer that teaches something +gets written into this paper, dated, host-verified if it touches our +estate. + +- (waiting for the first question) diff --git a/learning/Publish-Subscribe.md b/learning/Publish-Subscribe.md new file mode 100644 index 0000000..0efa957 --- /dev/null +++ b/learning/Publish-Subscribe.md @@ -0,0 +1,121 @@ +Publish-Subscribe + +Everything in NATS starts with publish-subscribe. A publisher sends a message to a subject, and every subscriber listening on that subject gets a copy. This is the foundation everything else builds on. + +How It Works + + 1.Publishers send messages to a subject — a simple string like orders.created + 2.Subscribers express interest in subjects they care about + 3.NATS delivers a copy of each message to every matching subscriber + 4.No coupling — publishers don't know about subscribers, and subscribers don't know about publishers + +This decoupling gives you tremendous flexibility. Services can be added, removed, or restarted without coordinating with anyone else. + + +Publishing Messages + +You publish a message by sending it to a subject: + +#!/bin/bash + +# Publish a message to the "weather.updates" subject +nats pub weather.updates "Temperature: 72°F" + +Python: + +# Publish a message to the subject "weather.updates" +await nc.publish("weather.updates", "Temperature: 72°F".encode()) + + +Key points: + + 1.Publishers don't wait for acknowledgments (fire-and-forget) + 2.Messages are delivered to all active subscribers + 3.If no subscribers exist, the message is simply discarded + + +Subscribing to Subjects + +Subscribers express interest in subjects to receive messages: + +#!/bin/bash + +# Subscribe to messages on the "weather.updates" subject +nats sub "weather.updates" + +Python: + +# Subscribe to 'weather.updates' synchronously +sub = await nc.subscribe("weather.updates") + +# Process messages +while True: + try: + msg = await sub.next(timeout=1) + print(f"Received: {msg.data.decode()}") + except TimeoutError: + break + + +When to Use Pub/Sub + +Publish-subscribe is ideal when multiple services need to react to the same event: + + 1.Event broadcasting — notify services about user signups, order placements, deployments + 2.Data distribution — send updates to multiple dashboards or monitoring services + 3.Fan-out notifications — alert all interested parties about state changes + 4.Audit logging — multiple services independently log the same events + + +Real-world example + +Imagine an e-commerce system where an order is placed. You publish a single event: + +nats pub orders.created '{"orderId": "123", "total": 99.99}' + +Multiple services subscribe and each reacts differently: + + 1.Inventory Service updates stock levels + 2.Email Service sends a confirmation email + 3.Analytics Service records metrics + 4.Shipping Service prepares the shipment + +One publish, four independent reactions — no coordination needed. + +--- + +## The goldfish boundary (what this paper describes, precisely) + +Everything above is *core NATS*: fire-and-forget, delivered to active +subscribers only, discarded if no one listens. No memory. We found this +the live way on 2026-11-06: `nats sub "dev.>"` caught `dev.test.hello` +while the terminal sat open, and the bus has no record of it since. + +JetStream adds the memory: streams claim subjects, messages are stored, +durable consumers replay and each tracks its own position. + +## Estate mapping (our reality, host-verified) + +| Item | Value | Truth of record | +|------|-------|-----------------| +| Bus | `nats-dev.eachpath.local:30000` (VM 122) | [host record](../../eachpath.local/saturn.eachpath.local/vms/nats-dev.eachpath.local/nats-dev.eachpath.local.md) | +| Streams | `REGISTER` -> `nimmerverse.register.>` (durable), `CYCLE` -> `nimmerverse.cycle.>` (100k msgs) | host record, 2026-11-06 | +| Publisher hosts | aynee (nats-cli + nats-py 2.16.0 in nimmer-lib venv), nimmerworld-dev | both, 2026-11-06 | +| Core-mode example | `dev.test.hello` = "Hello Nimmerverse!": live delivery only, in NO stream | verified 2026-11-06 | + +Note: the generic example `orders.created` has no stream here; on OUR +estate, subjects without a stream are transient by design. Our persistent +naming is `nimmerverse.*` under the two streams above. + +## Drills (run on our estate, record answers in section 5 of the 101 paper) + +1. Terminal A (nimmerworld-dev): `nats pub nimmerverse.cycle.hello "dafit was here"` + Terminal B (aynee): `nats sub "nimmerverse.cycle.*"` - watch the `*` + match exactly one token. +2. `nats stream info CYCLE` - the bus has now *counted* the message. +3. Kill both terminals, start a subscriber again: core-mode subject is + empty, stream subject replays its history. That contrast IS the lesson. + +## Understand-check (ten words, said back before any primitive builds on this) + +> dafit's version: (to be said back) \ No newline at end of file diff --git a/learning/code_snippets/NATS-101.md b/learning/code_snippets/NATS-101.md new file mode 100644 index 0000000..4c19016 --- /dev/null +++ b/learning/code_snippets/NATS-101.md @@ -0,0 +1,16 @@ +### NATS Access + +**Server Info**: +```bash +curl -s http://nats-dev.eachpath.local:30001/varz | jq .server_name +``` + +**Subscribe (nats cli)**: +```bash +nats sub -s nats://nats-dev.eachpath.local:30000 "dev.>" +``` + +**Publish (nats cli)**: +```bash +nats pub -s nats://nats-dev.eachpath.local:30000 "dev.test.hello" "Hello Nimmerverse!" +``` \ No newline at end of file