feat: xorg viewer scale modes, resize fix, arch notes
Scale modes (STRETCH/FIT/FILL/1:1) with CENTER/TOP_LEFT anchor: - UV crop via u_uv_scale/u_uv_offset uniforms in vertex shader - glViewport sub-rect + glClear for FIT and 1:1 modes - xorg_viewer_set_scale() / xorg_viewer_set_anchor() setters - Stub implementations for both Resize fix: glfwSetWindowUserPointer + framebuffer_size_callback calls render() synchronously during resize so image tracks window edge immediately. Forward declaration added to fix implicit decl error. Q/Escape close the window via key_callback. xorg_cli: --scale and --anchor arguments added. architecture.md: - Scale mode table and anchor docs in Frame Viewer Sink section - Render loop design note: frame-driven not timer-driven, resize callback rationale, threading note (GL context ownership, frame queue) - Text overlay section: tier 1 bitmap atlas (Pillow build tool, skyline packing, quad rendering), tier 2 HarfBuzz+FreeType, migration path Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -396,7 +396,18 @@ The module can act as a video sink by creating a window and rendering the latest
|
||||
- Displays the most recently received frame — driven by the low-latency output mode of the relay; never buffers for completeness
|
||||
- Forwards keyboard and mouse events back upstream as `INPUT_EVENT` protocol messages, enabling remote control use cases
|
||||
|
||||
Scale and crop are applied in the renderer — the incoming frame is stretched or letterboxed to fill the window. This allows a high-resolution source (Pi camera, screen grab) to be displayed scaled-down on a different machine.
|
||||
Scale and crop are applied in the renderer. Four display modes are supported (selected per viewer):
|
||||
|
||||
| Mode | Behaviour |
|
||||
|---|---|
|
||||
| `STRETCH` | Fill window, ignore aspect ratio |
|
||||
| `FIT` | Largest rect that fits, preserve aspect, black bars |
|
||||
| `FILL` | Scale to cover, preserve aspect, crop edges |
|
||||
| `1:1` | Native pixel size, no scaling; excess cropped |
|
||||
|
||||
Each mode combines with an anchor (`CENTER` or `TOP_LEFT`) that controls placement when the frame does not fill the window exactly.
|
||||
|
||||
This allows a high-resolution source (Pi camera, screen grab) to be displayed scaled-down on a different machine, or viewed at native resolution with panning.
|
||||
|
||||
This makes it the display-side counterpart of the V4L2 capture source: a frame grabbed from a camera on a Pi can be viewed on any machine in the network running a viewer sink node, with the relay handling the path and delivery mode.
|
||||
|
||||
@@ -417,6 +428,41 @@ For **raw pixel formats** (BGRA, YUV planar from the wire): uploaded directly wi
|
||||
|
||||
This keeps CPU load minimal — the only CPU work for MJPEG is Huffman decode and DCT, which libjpeg-turbo runs with SIMD. All color conversion and scaling is on the GPU.
|
||||
|
||||
#### Text overlays (future)
|
||||
|
||||
Two tiers are planned, implemented in order:
|
||||
|
||||
**Tier 1 — bitmap font atlas (initial)**
|
||||
|
||||
A build-time script (Python Pillow) renders glyphs from a TTF font into a packed PNG atlas and emits a metadata file (JSON or generated C header) with per-glyph UV rects and advance widths. At runtime the atlas is uploaded as a `GL_RGBA` texture and each character is rendered as a small quad, alpha-blended over the frame. Simple skyline packing keeps the atlas compact.
|
||||
|
||||
The generator lives in `tools/gen_font_atlas/` and runs as part of `make build`. Sufficient for ASCII overlays: timestamps, stream labels, debug info.
|
||||
|
||||
**Tier 2 — HarfBuzz + FreeType (later)**
|
||||
|
||||
A proper runtime font stack for full typography: correct shaping, kerning, ligatures, bidirectional text, non-Latin scripts. Added as a feature flag with its own runtime deps alongside the blit path.
|
||||
|
||||
When Tier 2 is implemented, the Pillow build dependency may be replaced by a purpose-built atlas generator (removing the Python dep entirely), if the blit path is still useful alongside the full shaping path.
|
||||
|
||||
#### Render loop
|
||||
|
||||
The viewer is driven by incoming frames rather than a fixed-rate loop. The intended pattern for callers:
|
||||
|
||||
```c
|
||||
while (xorg_viewer_poll(v)) {
|
||||
if (new_frame_available()) {
|
||||
xorg_viewer_push_yuv420(v, ...); /* upload + render */
|
||||
}
|
||||
/* no new frame → no redundant GPU work */
|
||||
}
|
||||
```
|
||||
|
||||
`xorg_viewer_poll` calls `glfwPollEvents` which dispatches input and resize events. A `framebuffer_size_callback` registered on the window calls `render()` synchronously during the resize, so the image tracks the window edge without a one-frame lag. This avoids both a busy render loop and the latency of waiting for the next poll iteration.
|
||||
|
||||
For a static image (test tool, paused stream), `glfwWaitEventsTimeout(interval)` is a better substitute for `glfwPollEvents` — it sleeps until an event arrives or the timeout expires, eliminating idle CPU usage.
|
||||
|
||||
Threading note: the GL context must be used from the thread that created it. In the video node, incoming frames arrive on a network receive thread. A frame queue between the receive thread and the render thread (which owns the GL context) is the correct model — the render thread drains the queue each poll iteration rather than having the network thread call push functions directly.
|
||||
|
||||
#### Renderer: Vulkan (future alternative)
|
||||
|
||||
A Vulkan renderer is planned as an alternative to the OpenGL one. GLFW's surface creation API is renderer-agnostic, so the window management and input handling code is shared. Only the renderer backend changes.
|
||||
|
||||
Reference in New Issue
Block a user