libjpeg-turbo can decompress directly to planar YUV, bypassing CPU-side
color conversion entirely. Document the precise pipeline: separate Y/Cb/Cr
GL_RED textures, BT.601 matrix in fragment shader, SIMD Huffman+DCT on CPU only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace XShmPutImage approach with GLFW+OpenGL as the initial renderer.
Documents the two-renderer plan: GLFW handles window/input for both;
only the rendering backend differs. Notes that both renderers should
conform to the same internal interface for swappability.
Adds input event forwarding (keyboard/mouse → INPUT_EVENT upstream)
as a first-class capability of the viewer sink.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents multi-input scheduling as a distinct concern from delivery
mode. Covers strict priority, round-robin, weighted round-robin, deficit
round-robin, and source suppression policies. Notes that the relay module
should expose a pluggable scheduler interface. Adds scheduler policy
selection to Open Questions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both sites default to site_id=0; gateway must rewrite site_id in
announcements and protocol messages crossing the boundary. site_id=0
on a cross-site link is a gateway protocol error.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers all layers: TCP → transport frame → message type dispatcher →
payload schemas. Documents frame format (6-byte header), all message
types, STREAM_OPEN/CLOSE lifecycle, codec/pixel_format/origin tables,
stream events, and discovery announcement wire format.
Also fixes stale channel_id reference in audio section of architecture.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes channel_id from the header. All message-specific identifiers
(stream_id, request_id, etc.) now live at the start of the payload,
interpreted by each message type handler. A relay seeing an unknown
type can skip or forward it using only payload_length, with no
knowledge of the payload structure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
No central message hub or broker. Controller is a function_flags bit — any
node with a user interface (e.g. web UI) holds it. Multiple nodes can hold
the role simultaneously. Controller communicates directly with peers over
the binary protocol; no intermediary.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ingest module: dequeue V4L2 buffers, emit one encapsulated frame per buffer.
Driver guarantees per-buffer framing for V4L2_PIX_FMT_MJPEG; no scanning needed.
mjpeg_scan: future optional module for non-compliant hardware only.
Explicitly a workaround, not part of the primary pipeline.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
V4L2 with proper node: driver guarantees per-buffer framing, no scan needed.
Opaque pipe (dd|nc): buffer boundaries lost, EOI scanner is the correct tool.
Weird containers (RIFF-wrapped USB cams, IP cameras, RTSP): route via ffmpeg,
not a custom parser. Scanner is an option only for constrained raw-stream cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
format (u16): what the bytes are — drives decode, stable across encoder changes
pixel_format (u16): layout for raw formats, ignored otherwise
origin (u16): how it was produced — informational only, no effect on decode
Eliminates numerical range assumptions (0x01xx ffmpeg range). A camera
outputting MJPEG natively and libjpeg-turbo encoding MJPEG are the same
format with different origins; receiver handles both identically.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
0x01xx range reserved for ffmpeg-backed formats (H.265, AV1, FFV1,
ProRes). Documents libavcodec vs subprocess trade-offs: subprocess suits
archival completeness paths, libavcodec suits low-latency encode. Receiver
only cares about wire format, not which encoder produced it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
xorg module: XRandR geometry queries, screen grab source (XShmGetImage),
frame viewer sink (XShmPutImage, fullscreen per monitor). All exposed as
standard source/sink node roles on the existing transport.
Audio: deferred but transport is already compatible — channel_id mux,
audio_frame message type slot reserved, relay/allocator are payload-agnostic.
Also marks serial as done in planning.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
site_id (u16) is reserved in the announcement payload from day one,
always 0 in single-site deployments. Documents the site gateway node
concept and fully-qualified addressing (site_id:namespace:instance) so
multi-site can be added later without wire format changes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A node can declare multiple roles simultaneously (e.g. relay + sink).
Replaces the function string with a fixed-size flags field; keeps the
payload layout simple and fixed-width up to the name.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reuse UDP multicast transport (224.0.0.251:5353) with our own binary
wire format — no Avahi, no Bonjour, no daemon dependency.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
architecture.md:
- Web interface connects as a first-class binary protocol peer; no JSON
bridge in C; DataView in JS maps directly to get_u32 etc.
- Future preprocessor section: protocol schema defined once, emits both
C (put/get, write_*/read_*) and ESM JS (DataView encode/decode);
same tool as planned for error location codes
planning.md:
- Add web node (entry #11) to module order
- Add Future: Protocol Preprocessor section above Deferred Decisions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
architecture.md: replace JSON control payloads with binary serialization;
add Protocol Serialization section describing little-endian wire format,
put/get buffer layer, and write_*/read_* protocol layer.
planning.md: mark common/media_ctrl/v4l2_ctrl done; insert serial (#4)
and protocol (#6) modules with descriptions.
conventions.md: document -flto and its implication (no manual static for
inlining — compiler handles it at link time).
Makefiles: add -flto to CFLAGS in all four Makefiles.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Initial documentation for the multi-peer video routing system:
- architecture.md covers graph model, transport protocol, relay design,
and the Pi get-it-on-the-wire-first rationale
- planning.md defines module build order and directory structure
- conventions.md captures C11 code style, naming, error handling approach,
and directory layout rules
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
