- protocol.md: add START_DISPLAY (0x000A) and STOP_DISPLAY (0x000B) wire
schemas and field descriptions; add both to command table
- xorg.md: add 'Multiple windows' section covering glfwPollEvents global
behaviour, per-context glfwMakeContextCurrent requirement, and
glfwInit/glfwTerminate ref-counting; includes the gotcha that
short-circuiting the event loop can starve non-polled windows
- planning.md: add cooperative capture release deferred decision;
add xorg viewer remote controls (zoom, pan, scale, future shader
post-processing) to deferred decisions; note xorg viewer controls
not yet exposed remotely in module table
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Protocol:
- Add PROTO_CMD_START_DISPLAY (0x000A) and PROTO_CMD_STOP_DISPLAY (0x000B)
with write/read functions; Proto_Start_Display carries stream_id, window
position/size, scale and anchor; PROTO_DISPLAY_SCALE_*/ANCHOR_* constants
Node display sink:
- Display_Slot struct with wanted_state/current_state (DISP_CLOSED/DISP_OPEN);
handlers set wanted state, display_loop_tick on main thread reconciles
- Up to MAX_DISPLAYS (4) simultaneous viewer windows
- on_frame routes incoming VIDEO_FRAME messages to matching display slot;
transport thread deposits payload, main thread consumes without holding lock
during JPEG decode/upload
- Main thread runs GL event loop when xorg is available; headless fallback
joins reconciler timer thread as before
Xorg multi-window:
- Ref-count glfwInit/glfwTerminate via glfw_acquire/glfw_release so closing
one viewer does not terminate GLFW for remaining windows
- Add glfwMakeContextCurrent before GL calls in push_yuv420, push_bgra,
push_mjpeg and poll so each viewer uses its own GL context correctly
Transport random port:
- Bind port 0 lets the OS assign a free port; getsockname reads it back
into server->bound_port after bind
- Add transport_server_get_port() accessor
- Default tcp_port changed from 8000 to 0 (random); node prints actual
port after server start so it is always visible in output
- Add --port PORT CLI override (before config-file argument)
controller_cli:
- Add start-display and stop-display commands
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add 'force' phony prerequisite to all sub-make delegation rules in
dev/cli/Makefile and src/node/Makefile so the sub-make is always
invoked and can check source timestamps itself; previously a stale
.o would never be rebuilt by a dependent Makefile
- Move stream_stats_record_frame inside the successful send branch in
on_ingest_frame so stats reflect actual delivered frames rather than
capture throughput; avoids misleading Mbps readings when the
transport is disconnected
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Connects to a running video node by host:port. Supports:
enum-devices, enum-controls, get-control, set-control,
start-ingest, stop-ingest
Uses semaphore-based request/response synchronisation (same pattern as
query_cli). start-ingest maps directly to the new START_INGEST protocol
command with optional format/size/fps args; defaults to auto-select.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each ingest stream gets two reconciler resources (device, transport) with
dependencies: transport waits for device OPEN (needs format for STREAM_OPEN),
device waits for transport CONNECTED before starting capture.
START_INGEST sets wanted state and triggers a tick; the reconciler drives
device CLOSED→OPEN→STREAMING and transport DISCONNECTED→CONNECTED over
subsequent ticks. STOP_INGEST reverses both.
External events (transport drop, ingest thread error) use
reconciler_force_current to push state backward; the periodic 500ms timer
thread re-drives toward wanted state automatically.
All 8 stream slots are pre-allocated at startup. on_ingest_frame sends
VIDEO_FRAME messages over the outbound transport connection, protected by
a per-stream conn_mutex.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
START_INGEST carries stream_id, format/width/height/fps, dest_host:port,
transport_mode (encapsulated or opaque), and device_path. All format fields
default to 0 (auto-select). STOP_INGEST carries stream_id only.
Both commands set wanted state on the node; reconciliation is asynchronous.
Protocol doc updated with wire schemas for both commands.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
reconciler: generic resource state machine — BFS pathfinding from current
to wanted state, dependency constraints, event/periodic tick model.
reconciler_cli exercises it with simulated device/transport/stream resources.
ingest: V4L2 capture module — open device, negotiate MJPEG format, MMAP
buffer pool, capture thread with on_frame callback. start/stop lifecycle
designed for reconciler management. Transport-agnostic: caller wires
on_frame to proto_write_video_frame.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
architecture.md is now a concise overview (~155 lines) with a
Documentation section linking to all sub-docs.
New sub-docs in docs/:
transport.md — wire modes, frame header, serialization, web peer
relay.md — delivery modes, memory model, congestion, scheduler
codec.md — stream metadata, format negotiation, codec backends
xorg.md — screen grab, viewer sink, render loop, overlays
discovery.md — multicast announcements, multi-site, site gateways
node-state.md — wanted/current state, reconciler, stats, queries
device-resilience.md — device loss handling, stream events, audio (future)
All cross-references updated to file links. Every sub-doc links back
to architecture.md. docs/transport.md links to docs/protocol.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Graph representation is plain ESM objects in the web interface.
No special format needed. Graph reconstruction, topology diffing,
and layout logic belong in ESM rather than C. Future TUI/CLI tools
reuse the same ESM libraries via Node.js.
No open questions remain.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Drop policy (per-output configurable), stream ID passthrough at relay,
TCP-only transport for now, soft byte budget limits with hysteresis,
and relay scheduler (strict priority first, pluggable interface) were
all already decided — move them out of Open Questions.
Only genuinely open question remaining: graph representation format.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add 'Declarative, Not Imperative' section near top explaining why the
control model is wanted-state-based rather than imperative commands
- Update Control Plane section: remove 'connection instructions' language,
replace with wanted state; note CLI controller comes before web UI
- Fix node naming example: xorg:preview instead of mpv:preview
- Update ingestion diagram: 'wanted state' instead of 'connection config'
- Add Per-Stream Stats note (stream_stats.h) to Node State Model
- Mark GET_CONFIG_STATE / GET_RUNTIME_STATE as planned, not yet implemented
- Split Open Questions: add Decided section for resolved questions
(connection direction, stream ID assignment, single port, first delivery mode)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Document the wanted/current state separation, generic resource state
machine reconciler (BFS pathfinding, event + periodic tick), node state
queries (GET_CONFIG_STATE / GET_RUNTIME_STATE), stream ID assignment
by controller, and connection direction model.
Add reconciler module to module order and reconciler_cli experiment
to CLI tools table in planning.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add stream_send_cli (V4L2 capture → TCP → VIDEO_FRAME) and
stream_recv_cli (TCP → threaded frame slot → GLFW display) to
exercise end-to-end streaming between two nodes on the same machine
or across the network.
Add include/stream_stats.h (header-only rolling-window fps/Mbps tracker)
and include/v4l2_fmt.h (header-only V4L2 format enumeration shared between
v4l2_view_cli and stream_send_cli). Refactor v4l2_view_cli to use the
shared header.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- tools/gen_font_atlas: Python/Pillow build tool — skyline packs DejaVu
Sans glyphs 32-255 into a grayscale atlas, emits build/gen/font_atlas.h
with pixel data and Font_Glyph[256] metrics table
- xorg: bitmap font atlas text overlay rendering (GL_R8 atlas texture,
alpha-blended glyph quads, dark background rect per overlay)
- xorg: add xorg_viewer_set_overlay_text / clear_overlays API
- xorg: add xorg_viewer_handle_events for streaming use (events only,
no redundant render)
- xorg_cli: show today's date as white text overlay
- v4l2_view_cli: new tool — V4L2 capture with format auto-selection
(highest FPS then largest resolution), MJPEG/YUYV, measured FPS overlay
- docs: update README, planning, architecture to reflect current status
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
make — builds with glfw, vulkan, turbojpeg, xorg, vaapi
make FEATURES= — headless build with no optional dependencies
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
libjpeg-turbo can decompress directly to planar YUV, bypassing CPU-side
color conversion entirely. Document the precise pipeline: separate Y/Cb/Cr
GL_RED textures, BT.601 matrix in fragment shader, SIMD Huffman+DCT on CPU only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace XShmPutImage approach with GLFW+OpenGL as the initial renderer.
Documents the two-renderer plan: GLFW handles window/input for both;
only the rendering backend differs. Notes that both renderers should
conform to the same internal interface for swappability.
Adds input event forwarding (keyboard/mouse → INPUT_EVENT upstream)
as a first-class capability of the viewer sink.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents multi-input scheduling as a distinct concern from delivery
mode. Covers strict priority, round-robin, weighted round-robin, deficit
round-robin, and source suppression policies. Notes that the relay module
should expose a pluggable scheduler interface. Adds scheduler policy
selection to Open Questions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
README status table was showing transport/discovery/protocol/node as
"not started" when all are done. Added summary sentence, notes column,
config and dev/web rows. Fixed dev/web structure description.
planning.md: removed stale "prerequisite" note about web UI — it is
already implemented and working.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
src/node/main.c: ctrl_enum_cb was discarding menu_count and menu_items,
causing empty dropdowns for all MENU/INTEGER_MENU controls. Added a
menu item pool (MAX_MENU_POOL=128 items) to Ctrl_Build; the callback now
copies items into the pool and sets menu_count/menu_items on the control.
docs/protocol.md: add missing sections — str8 primitive, ENUM_DEVICES,
ENUM_CONTROLS (with control type/flag tables and menu item notes),
GET_CONTROL, and SET_CONTROL schemas.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Switching a menu control (e.g. exposure auto → manual) changes the
flags on related controls (e.g. exposure_absolute loses FLAG_GRABBED).
Re-fetch and re-render controls silently after any non-slider change so
the updated enabled/read-only states are reflected immediately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- index.html: minimal shell with <template> elements for all repeated
DOM structures (node-item, device-group, device-item, ctrl-group,
ctrl-row, capture-badge); links external style.css
- style.css: all styles extracted from index.html
- lib/dom.mjs: by_id, qs, clone, show, hide helpers
- app.mjs: persistent SSE node list replaces Discover button; clicking
a node connects to it; uses clone()/replaceChildren() throughout;
no innerHTML for structure; event wiring at bottom
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- protocol.mjs: all reads/writes switched to LE to match serial.h
- node_client.mjs: persistent error handler prevents ECONNRESET crash
- discovery.mjs: remove unnecessary SO_REUSEPORT
- server.mjs: discovery runs at startup (not per SSE open); uses
EventEmitter + known_peers Map so SSE replays existing peers on connect
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Protocol module: framed binary encoding for control requests/responses
(ENUM_DEVICES, ENUM_CONTROLS, GET/SET_CONTROL, STREAM_OPEN/CLOSE)
- video-node: scans /dev/media* and /dev/video*, serves V4L2 device
topology and controls over TCP; uses UDP discovery for peer announce
- query_cli: auto-discovers a node, queries devices and controls
- protocol_cli: low-level protocol frame decoder for debugging
- dev/web: Express 5 ESM web inspector — live SSE discovery picker,
REST bridge to video-node, controls UI with sliders/selects/checkboxes
- Makefile: sequential module builds before cli/node to fix make -j races
- common.mk: add DEPFLAGS (-MMD -MP) for automatic header dependencies
- All module Makefiles: split compile/link, generate .d dependency files
- discovery: replace 100ms poll loop with pthread_cond_timedwait;
respond to all announcements (not just new peers) for instant re-discovery
- ENUM_DEVICES response: carry device_caps (V4L2_CAP_*) per video node
so clients can distinguish capture nodes from metadata nodes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Config_Entry now holds a union {s, u16, u32, flags} typed at parse time.
Getters read directly from the union — no string conversion at access time.
config_dump reconstructs flag display as 'token | token' from the bitmask.
Separators in flag values: comma, pipe, and whitespace all accepted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Config_Def schema tables declare section/key/type/default per module.
Typed getters: config_get_str, _u16, _u32, _flags.
FLAGS type parses space/comma-separated tokens via a Config_Flag_Def table.
config_defaults() gives schema defaults without a file.
config_dump() prints effective values for diagnostics.
config_cli: load a file or --defaults and dump effective config.
dev/example.cfg: sample config covering node, discovery, transport.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the receive thread detects a genuinely new peer it sets
early_announce, causing the announce thread to break out of its
sleep and send immediately. The new node receives our announcement
within ~100ms rather than waiting up to interval_ms.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sends 6-byte framed announcements to 224.0.0.251:5353 on startup and
every interval_ms (default 5s). Receive thread maintains a peer table
(max 64 entries); fires on_peer_found for new peers, on_peer_lost when
a peer misses timeout_intervals (default 3) consecutive intervals.
Own announcements are filtered by name+site_id. SO_REUSEADDR+REUSEPORT
allows multiple processes on the same host for testing.
discovery_cli: announce <name> <tcp_port> [flags] — prints found/lost events.
Also notes future config module in planning.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both sites default to site_id=0; gateway must rewrite site_id in
announcements and protocol messages crossing the boundary. site_id=0
on a cross-site link is a gateway protocol error.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers all layers: TCP → transport frame → message type dispatcher →
payload schemas. Documents frame format (6-byte header), all message
types, STREAM_OPEN/CLOSE lifecycle, codec/pixel_format/origin tables,
stream events, and discovery announcement wire format.
Also fixes stale channel_id reference in audio section of architecture.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes channel_id from the header. All message-specific identifiers
(stream_id, request_id, etc.) now live at the start of the payload,
interpreted by each message type handler. A relay seeing an unknown
type can skip or forward it using only payload_length, with no
knowledge of the payload structure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
transport_server_create/start: binds TCP, spawns accept thread, closes
excess inbound connections when max_connections is reached.
transport_connect: outbound TCP, spawns read thread before returning.
transport_send_frame: packs 8-byte header with serial put_*, then writes
header + payload under a per-connection mutex (thread-safe).
Read thread: reads header, validates payload_length <= max_payload, mallocs
payload, calls on_frame (callback owns and must free payload). On error or
disconnect calls on_disconnect then frees conn.
transport_cli: server mode echoes received frames; client mode sends 3
test frames and prints echoes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
No central message hub or broker. Controller is a function_flags bit — any
node with a user interface (e.g. web UI) holds it. Multiple nodes can hold
the role simultaneously. Controller communicates directly with peers over
the binary protocol; no intermediary.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ingest module: dequeue V4L2 buffers, emit one encapsulated frame per buffer.
Driver guarantees per-buffer framing for V4L2_PIX_FMT_MJPEG; no scanning needed.
mjpeg_scan: future optional module for non-compliant hardware only.
Explicitly a workaround, not part of the primary pipeline.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
V4L2 with proper node: driver guarantees per-buffer framing, no scan needed.
Opaque pipe (dd|nc): buffer boundaries lost, EOI scanner is the correct tool.
Weird containers (RIFF-wrapped USB cams, IP cameras, RTSP): route via ffmpeg,
not a custom parser. Scanner is an option only for constrained raw-stream cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
format (u16): what the bytes are — drives decode, stable across encoder changes
pixel_format (u16): layout for raw formats, ignored otherwise
origin (u16): how it was produced — informational only, no effect on decode
Eliminates numerical range assumptions (0x01xx ffmpeg range). A camera
outputting MJPEG natively and libjpeg-turbo encoding MJPEG are the same
format with different origins; receiver handles both identically.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
0x01xx range reserved for ffmpeg-backed formats (H.265, AV1, FFV1,
ProRes). Documents libavcodec vs subprocess trade-offs: subprocess suits
archival completeness paths, libavcodec suits low-latency encode. Receiver
only cares about wire format, not which encoder produced it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
xorg module: XRandR geometry queries, screen grab source (XShmGetImage),
frame viewer sink (XShmPutImage, fullscreen per monitor). All exposed as
standard source/sink node roles on the existing transport.
Audio: deferred but transport is already compatible — channel_id mux,
audio_frame message type slot reserved, relay/allocator are payload-agnostic.
Also marks serial as done in planning.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
