mikael-lovqvists-claude-agent/video-setup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add protocol reference document

Browse Source 
Covers all layers: TCP → transport frame → message type dispatcher →
payload schemas. Documents frame format (6-byte header), all message
types, STREAM_OPEN/CLOSE lifecycle, codec/pixel_format/origin tables,
stream events, and discovery announcement wire format.

Also fixes stale channel_id reference in audio section of architecture.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
mikael-lovqvists-claude-agent
2026-03-26 22:03:02 +00:00
parent 197ab7d5db
commit 57e46af57b
3 changed files with 265 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
architecture.md
View File

@@ -386,11 +386,11 @@ Scale and crop are applied at render time — the incoming frame is stretched or
Audio streams are not in scope for the initial implementation but the transport is designed to accommodate them without structural changes.
The `channel_id` field already provides stream multiplexing on a single connection. A future audio channel is just another channel on an existing transport connection — no new connection type is needed. The message type table has room for an `audio_frame` type alongside `video_frame`.
A future audio stream is just another message type on an existing transport connection — no new connection type or header field is needed. `stream_id` in the payload already handles multiplexing. The message type table has room for an `audio_frame` type alongside `video_frame`.
The main open question is codec and container: raw PCM is trivial to handle but large; compressed formats (Opus, AAC) need framing conventions. This is deferred until video is solid.
The frame allocator, relay, and archive modules should not make assumptions that `channel_id` implies video — they operate on opaque byte payloads with a message type and length, so audio frames will pass through the same infrastructure unchanged.
The frame allocator, relay, and archive modules should not assume that a frame implies video — they operate on opaque byte payloads with a message type and length, so audio frames will pass through the same infrastructure unchanged.
---