docs: add discovery flow diagrams, document restart detection limitation

- Four Mermaid sequence diagrams: startup, steady-state keepalive, node
  loss/timeout, and node restart
- Explicitly document that the site_id-change restart heuristic does not work
  in practice (site_id is static config, not a runtime value)
- Describe what needs to change: a boot nonce (random u32 at startup)
- Add boot nonce as a deferred item in planning.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-30 04:39:52 +00:00
parent f3a6be0701
commit 92ba1adf29
2 changed files with 72 additions and 7 deletions

View File

@@ -43,17 +43,81 @@ A node may set multiple bits — a relay that also archives sets both `RELAY` an
### Behaviour ### Behaviour
- Nodes send announcements periodically (e.g. every 5 s) and immediately on startup via multicast - Nodes send announcements periodically (default every 5 s) and immediately on startup via multicast
- No daemon — the node process itself sends and listens; no background service required - No daemon — the node process itself sends and listens; no background service required
- On receiving an announcement the control plane records the peer (address, port, name, function) and can initiate a transport connection if needed - On receiving an announcement the node records the peer (address, port, name, capabilities) and can initiate a transport connection if needed
- A node going silent for a configured number of announcement intervals is considered offline - A peer that goes silent for `timeout_intervals × interval_ms` is considered offline and removed from the peer table
- Announcements are informational only — the hub validates identity at connection time - Announcements are informational only — identity is validated at TCP connection time
#### Targeted replies #### Startup — new node joins the network
Multicast is only used for the periodic keep-alive broadcast. When a node receives an announcement from a peer it does not yet know, or detects that a known peer has restarted (its `site_id` changed for the same address and port), it sends an **immediate unicast reply** directly back to that peer's IP address. This ensures the new or restarted peer learns about this node quickly without waiting up to `interval_ms`, while avoiding a multicast blast that would unnecessarily wake every other node on the subnet. ```mermaid
sequenceDiagram
participant N as New Node
participant MC as Multicast group
participant A as Node A
participant B as Node B
Steady-state keepalive packets from already-known peers do not trigger any reply. N->>MC: announce (multicast)
MC-->>A: receives announce
MC-->>B: receives announce
A->>N: announce (unicast reply)
B->>N: announce (unicast reply)
Note over N,B: All parties now know each other.<br/>Subsequent keepalives are multicast only.
```
Each node that hears a new peer sends a **unicast reply** directly to that peer. This allows the new node to populate its peer table within one round-trip rather than waiting up to `interval_ms` for other nodes' next scheduled broadcast.
#### Steady-state keepalive
```mermaid
sequenceDiagram
participant A as Node A
participant MC as Multicast group
participant B as Node B
participant C as Node C
loop every interval_ms
A->>MC: announce (multicast)
MC-->>B: receives — updates last_seen_ms, no reply
MC-->>C: receives — updates last_seen_ms, no reply
end
```
Known peers update their `last_seen_ms` timestamp and do nothing else. No reply is sent, so there is no amplification.
#### Node loss — timeout
```mermaid
sequenceDiagram
participant A as Node A
participant B as Node B (offline)
Note over B: Node B stops sending
loop timeout_intervals × interval_ms elapses
A->>A: check_timeouts() — not yet expired
end
A->>A: check_timeouts() — expired, remove B
A->>A: on_peer_lost(B) callback
```
#### Node restart — known limitation
The current implementation attempts to detect a restart by checking whether `site_id` changed for a known `(addr, port)` entry. In practice this **does not work**: `site_id` is a static configuration value and will be the same before and after a restart. A restarted node will therefore simply be treated as a continuing keepalive and will not receive an immediate unicast reply — it will have to wait up to `interval_ms` for the next scheduled multicast broadcast from its peers.
```mermaid
sequenceDiagram
participant R as Restarted Node
participant MC as Multicast group
participant A as Node A
Note over R: Node restarts — same addr, port, site_id
R->>MC: announce (multicast)
MC-->>A: receives — site_id unchanged, treated as keepalive
Note over A: No unicast reply sent. R waits up to interval_ms<br/>to learn about A via A's next scheduled multicast.
```
**What needs to change:** a **boot nonce** (random `u32` generated at startup, not configured) should be added to the announcement payload. A change in boot nonce for a known peer unambiguously signals a restart and triggers an immediate unicast reply. This requires a wire format version bump and updates to the peer table struct, announcement builder, and receive logic.
### No Avahi/Bonjour Dependency ### No Avahi/Bonjour Dependency

View File

@@ -136,4 +136,5 @@ These are open questions tracked in `architecture.md` that do not need to be res
- controller_cli is a temporary dev tool; the long-term replacement is a dedicated `controller` binary outside `dev/cli/` that maintains simultaneous connections to all discovered nodes (not switching between them). Commands address a specific node by peer index. This mirrors the web UI's model of administering the whole network rather than one node at a time. The `connect` / active-connection model in the current controller_cli is an interim design choice that should not be carried forward. - controller_cli is a temporary dev tool; the long-term replacement is a dedicated `controller` binary outside `dev/cli/` that maintains simultaneous connections to all discovered nodes (not switching between them). Commands address a specific node by peer index. This mirrors the web UI's model of administering the whole network rather than one node at a time. The `connect` / active-connection model in the current controller_cli is an interim design choice that should not be carried forward.
- start-ingest peer addressing: the `dest_host` + `dest_port` in START_INGEST is awkward to type manually and requires the caller to know the target's TCP port. Should accept a peer ID (index from the discovered peer table on the node) so the node can resolve the address itself. Requires the node to run discovery and expose its peer table. - start-ingest peer addressing: the `dest_host` + `dest_port` in START_INGEST is awkward to type manually and requires the caller to know the target's TCP port. Should accept a peer ID (index from the discovered peer table on the node) so the node can resolve the address itself. Requires the node to run discovery and expose its peer table.
- Connection multiplexing: currently each ingest stream opens its own outbound TCP connection to the destination. Multiple streams between the same two peers should share one connection, with stream_id used to demultiplex frames. This is the priority/encapsulation scheme described in the architecture — high-priority and low-latency frames from different streams travel over the same socket rather than competing across separate sockets. - Connection multiplexing: currently each ingest stream opens its own outbound TCP connection to the destination. Multiple streams between the same two peers should share one connection, with stream_id used to demultiplex frames. This is the priority/encapsulation scheme described in the architecture — high-priority and low-latency frames from different streams travel over the same socket rather than competing across separate sockets.
- Discovery boot nonce: the announcement payload needs a `boot_nonce` field (random u32 generated at startup, not configured). The current restart detection uses `site_id` change as a proxy, but `site_id` is static config and does not change on restart, so restarts are not detected and the restarted node waits up to `interval_ms` for peers to reply. Adding a boot nonce gives a reliable restart signal: a nonce change for a known (addr, port) entry triggers an immediate unicast reply. Requires a wire format version bump, peer table struct update, and changes to the announcement builder and receive logic.
- Control grouping: controls should be organizable into named groups for both display organisation (collapsible sections in a UI) and protocol semantics (enumerate controls within a group, set a group of related controls atomically). Relevant for display devices where scale_mode, anchor, position, and size are logically related, and for cameras where white balance, exposure, and gain belong together. The current flat list of (control_id, name, type, value) tuples does not capture this. - Control grouping: controls should be organizable into named groups for both display organisation (collapsible sections in a UI) and protocol semantics (enumerate controls within a group, set a group of related controls atomically). Relevant for display devices where scale_mode, anchor, position, and size are logically related, and for cameras where white balance, exposure, and gain belong together. The current flat list of (control_id, name, type, value) tuples does not capture this.