mikael-lovqvists-claude-agent/video-setup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: xorg viewer scale modes, resize fix, arch notes

Browse Source 
Scale modes (STRETCH/FIT/FILL/1:1) with CENTER/TOP_LEFT anchor:
- UV crop via u_uv_scale/u_uv_offset uniforms in vertex shader
- glViewport sub-rect + glClear for FIT and 1:1 modes
- xorg_viewer_set_scale() / xorg_viewer_set_anchor() setters
- Stub implementations for both

Resize fix: glfwSetWindowUserPointer + framebuffer_size_callback calls
render() synchronously during resize so image tracks window edge
immediately. Forward declaration added to fix implicit decl error.

Q/Escape close the window via key_callback.

xorg_cli: --scale and --anchor arguments added.

architecture.md:
- Scale mode table and anchor docs in Frame Viewer Sink section
- Render loop design note: frame-driven not timer-driven, resize callback
  rationale, threading note (GL context ownership, frame queue)
- Text overlay section: tier 1 bitmap atlas (Pillow build tool, skyline
  packing, quad rendering), tier 2 HarfBuzz+FreeType, migration path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
mikael-lovqvists-claude-agent
2026-03-28 21:30:28 +00:00
parent ef0319b45b
commit 7fd79e6120
6 changed files with 413 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

48
architecture.md
View File

@@ -396,7 +396,18 @@ The module can act as a video sink by creating a window and rendering the latest
- Displays the most recently received frame — driven by the low-latency output mode of the relay; never buffers for completeness - Displays the most recently received frame — driven by the low-latency output mode of the relay; never buffers for completeness
- Forwards keyboard and mouse events back upstream as `INPUT_EVENT` protocol messages, enabling remote control use cases - Forwards keyboard and mouse events back upstream as `INPUT_EVENT` protocol messages, enabling remote control use cases
Scale and crop are applied in the renderer — the incoming frame is stretched or letterboxed to fill the window. This allows a high-resolution source (Pi camera, screen grab) to be displayed scaled-down on a different machine. Scale and crop are applied in the renderer. Four display modes are supported (selected per viewer):
| Mode | Behaviour |
|---|---|
| `STRETCH` | Fill window, ignore aspect ratio |
| `FIT` | Largest rect that fits, preserve aspect, black bars |
| `FILL` | Scale to cover, preserve aspect, crop edges |
| `1:1` | Native pixel size, no scaling; excess cropped |
Each mode combines with an anchor (`CENTER` or `TOP_LEFT`) that controls placement when the frame does not fill the window exactly.
This allows a high-resolution source (Pi camera, screen grab) to be displayed scaled-down on a different machine, or viewed at native resolution with panning.
This makes it the display-side counterpart of the V4L2 capture source: a frame grabbed from a camera on a Pi can be viewed on any machine in the network running a viewer sink node, with the relay handling the path and delivery mode. This makes it the display-side counterpart of the V4L2 capture source: a frame grabbed from a camera on a Pi can be viewed on any machine in the network running a viewer sink node, with the relay handling the path and delivery mode.
@@ -417,6 +428,41 @@ For **raw pixel formats** (BGRA, YUV planar from the wire): uploaded directly wi
This keeps CPU load minimal — the only CPU work for MJPEG is Huffman decode and DCT, which libjpeg-turbo runs with SIMD. All color conversion and scaling is on the GPU. This keeps CPU load minimal — the only CPU work for MJPEG is Huffman decode and DCT, which libjpeg-turbo runs with SIMD. All color conversion and scaling is on the GPU.
#### Text overlays (future)
Two tiers are planned, implemented in order:
**Tier 1 — bitmap font atlas (initial)**
A build-time script (Python Pillow) renders glyphs from a TTF font into a packed PNG atlas and emits a metadata file (JSON or generated C header) with per-glyph UV rects and advance widths. At runtime the atlas is uploaded as a `GL_RGBA` texture and each character is rendered as a small quad, alpha-blended over the frame. Simple skyline packing keeps the atlas compact.
The generator lives in `tools/gen_font_atlas/` and runs as part of `make build`. Sufficient for ASCII overlays: timestamps, stream labels, debug info.
**Tier 2 — HarfBuzz + FreeType (later)**
A proper runtime font stack for full typography: correct shaping, kerning, ligatures, bidirectional text, non-Latin scripts. Added as a feature flag with its own runtime deps alongside the blit path.
When Tier 2 is implemented, the Pillow build dependency may be replaced by a purpose-built atlas generator (removing the Python dep entirely), if the blit path is still useful alongside the full shaping path.
#### Render loop
The viewer is driven by incoming frames rather than a fixed-rate loop. The intended pattern for callers:
```c
while (xorg_viewer_poll(v)) {
if (new_frame_available()) {
xorg_viewer_push_yuv420(v, ...); /* upload + render */
}
/* no new frame → no redundant GPU work */
}
```
`xorg_viewer_poll` calls `glfwPollEvents` which dispatches input and resize events. A `framebuffer_size_callback` registered on the window calls `render()` synchronously during the resize, so the image tracks the window edge without a one-frame lag. This avoids both a busy render loop and the latency of waiting for the next poll iteration.
For a static image (test tool, paused stream), `glfwWaitEventsTimeout(interval)` is a better substitute for `glfwPollEvents` — it sleeps until an event arrives or the timeout expires, eliminating idle CPU usage.
Threading note: the GL context must be used from the thread that created it. In the video node, incoming frames arrive on a network receive thread. A frame queue between the receive thread and the render thread (which owns the GL context) is the correct model — the render thread drains the queue each poll iteration rather than having the network thread call push functions directly.
#### Renderer: Vulkan (future alternative) #### Renderer: Vulkan (future alternative)
A Vulkan renderer is planned as an alternative to the OpenGL one. GLFW's surface creation API is renderer-agnostic, so the window management and input handling code is shared. Only the renderer backend changes. A Vulkan renderer is planned as an alternative to the OpenGL one. GLFW's surface creation API is renderer-agnostic, so the window management and input handling code is shared. Only the renderer backend changes.

14
dev/cli/Makefile
View File

@@ -11,6 +11,7 @@ DISCOVERY_OBJ = $(BUILD)/discovery/discovery.o
CONFIG_OBJ = $(BUILD)/config/config.o CONFIG_OBJ = $(BUILD)/config/config.o
PROTOCOL_OBJ = $(BUILD)/protocol/protocol.o PROTOCOL_OBJ = $(BUILD)/protocol/protocol.o
TEST_IMAGE_OBJ = $(BUILD)/test_image/test_image.o TEST_IMAGE_OBJ = $(BUILD)/test_image/test_image.o
XORG_OBJ = $(BUILD)/xorg/xorg.o
CLI_SRCS = \ CLI_SRCS = \
media_ctrl_cli.c \ media_ctrl_cli.c \
@@ -20,7 +21,8 @@ CLI_SRCS = \
config_cli.c \ config_cli.c \
protocol_cli.c \ protocol_cli.c \
query_cli.c \ query_cli.c \
test_image_cli.c test_image_cli.c \
xorg_cli.c
CLI_OBJS = $(CLI_SRCS:%.c=$(CLI_BUILD)/%.o) CLI_OBJS = $(CLI_SRCS:%.c=$(CLI_BUILD)/%.o)
@@ -34,7 +36,8 @@ all: \
$(CLI_BUILD)/config_cli \ $(CLI_BUILD)/config_cli \
$(CLI_BUILD)/protocol_cli \ $(CLI_BUILD)/protocol_cli \
$(CLI_BUILD)/query_cli \ $(CLI_BUILD)/query_cli \
$(CLI_BUILD)/test_image_cli $(CLI_BUILD)/test_image_cli \
$(CLI_BUILD)/xorg_cli
# Module objects delegate to their sub-makes. # Module objects delegate to their sub-makes.
$(COMMON_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/common $(COMMON_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/common
@@ -46,6 +49,7 @@ $(DISCOVERY_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/discovery
$(CONFIG_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/config $(CONFIG_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/config
$(PROTOCOL_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/protocol $(PROTOCOL_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/protocol
$(TEST_IMAGE_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/test_image $(TEST_IMAGE_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/test_image
$(XORG_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/xorg
# Compile each CLI source to its own .o (generates .d alongside). # Compile each CLI source to its own .o (generates .d alongside).
$(CLI_BUILD)/%.o: %.c | $(CLI_BUILD) $(CLI_BUILD)/%.o: %.c | $(CLI_BUILD)
@@ -76,6 +80,9 @@ $(CLI_BUILD)/query_cli: $(CLI_BUILD)/query_cli.o $(COMMON_OBJ) $(SERIAL_OBJ) $(T
$(CLI_BUILD)/test_image_cli: $(CLI_BUILD)/test_image_cli.o $(TEST_IMAGE_OBJ) $(CLI_BUILD)/test_image_cli: $(CLI_BUILD)/test_image_cli.o $(TEST_IMAGE_OBJ)
$(CC) $(CFLAGS) -o $@ $^ $(CC) $(CFLAGS) -o $@ $^
$(CLI_BUILD)/xorg_cli: $(CLI_BUILD)/xorg_cli.o $(TEST_IMAGE_OBJ) $(XORG_OBJ)
$(CC) $(CFLAGS) -o $@ $^ $(PKG_LDFLAGS)
$(CLI_BUILD): $(CLI_BUILD):
mkdir -p $@ mkdir -p $@
@@ -90,6 +97,7 @@ clean:
$(CLI_BUILD)/config_cli \ $(CLI_BUILD)/config_cli \
$(CLI_BUILD)/protocol_cli \ $(CLI_BUILD)/protocol_cli \
$(CLI_BUILD)/query_cli \ $(CLI_BUILD)/query_cli \
$(CLI_BUILD)/test_image_cli $(CLI_BUILD)/test_image_cli \
$(CLI_BUILD)/xorg_cli
-include $(CLI_OBJS:%.o=%.d) -include $(CLI_OBJS:%.o=%.d)

125
dev/cli/xorg_cli.c Normal file
View File

@@ -0,0 +1,125 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "test_image.h"
#include "xorg.h"
static void usage(void)
{
fprintf(stderr,
"usage: xorg_cli [--pattern bars|ramp|grid]\n"
" [--width N] [--height N]\n"
" [--format yuv420|bgra]\n"
" [--scale stretch|fit|fill|1:1]\n"
" [--anchor center|topleft]\n"
" [--x N] [--y N]\n"
"\n"
"Opens a window and renders a test image using the xorg viewer sink.\n"
"Q or Escape closes the window.\n"
"\n"
"defaults: bars 1280x720 yuv420 stretch center at 0,0\n");
}
int main(int argc, char **argv)
{
Test_Pattern pattern = TEST_PATTERN_BARS;
Test_Fmt fmt = TEST_FMT_YUV420;
Xorg_Scale scale = XORG_SCALE_STRETCH;
Xorg_Anchor anchor = XORG_ANCHOR_CENTER;
int width = 1280;
int height = 720;
int win_x = 0;
int win_y = 0;
for (int i = 1; i < argc; i++) {
if (strcmp(argv[i], "--pattern") == 0 && i + 1 < argc) {
i++;
if (strcmp(argv[i], "bars") == 0) { pattern = TEST_PATTERN_BARS; }
else if (strcmp(argv[i], "ramp") == 0) { pattern = TEST_PATTERN_RAMP; }
else if (strcmp(argv[i], "grid") == 0) { pattern = TEST_PATTERN_GRID; }
else { fprintf(stderr, "unknown pattern: %s\n", argv[i]); usage(); return 1; }
} else if (strcmp(argv[i], "--width") == 0 && i + 1 < argc) {
width = atoi(argv[++i]);
} else if (strcmp(argv[i], "--height") == 0 && i + 1 < argc) {
height = atoi(argv[++i]);
} else if (strcmp(argv[i], "--x") == 0 && i + 1 < argc) {
win_x = atoi(argv[++i]);
} else if (strcmp(argv[i], "--y") == 0 && i + 1 < argc) {
win_y = atoi(argv[++i]);
} else if (strcmp(argv[i], "--scale") == 0 && i + 1 < argc) {
i++;
if (strcmp(argv[i], "stretch") == 0) { scale = XORG_SCALE_STRETCH; }
else if (strcmp(argv[i], "fit") == 0) { scale = XORG_SCALE_FIT; }
else if (strcmp(argv[i], "fill") == 0) { scale = XORG_SCALE_FILL; }
else if (strcmp(argv[i], "1:1") == 0) { scale = XORG_SCALE_1_1; }
else { fprintf(stderr, "unknown scale: %s\n", argv[i]); usage(); return 1; }
} else if (strcmp(argv[i], "--anchor") == 0 && i + 1 < argc) {
i++;
if (strcmp(argv[i], "center") == 0) { anchor = XORG_ANCHOR_CENTER; }
else if (strcmp(argv[i], "topleft") == 0) { anchor = XORG_ANCHOR_TOP_LEFT; }
else { fprintf(stderr, "unknown anchor: %s\n", argv[i]); usage(); return 1; }
} else if (strcmp(argv[i], "--format") == 0 && i + 1 < argc) {
i++;
if (strcmp(argv[i], "yuv420") == 0) { fmt = TEST_FMT_YUV420; }
else if (strcmp(argv[i], "bgra") == 0) { fmt = TEST_FMT_BGRA; }
else { fprintf(stderr, "unknown format: %s\n", argv[i]); usage(); return 1; }
} else {
usage(); return 1;
}
}
if (!xorg_available()) {
fprintf(stderr, "xorg_cli: built without HAVE_GLFW — viewer not available\n");
return 1;
}
if (width < 2 || height < 2) {
fprintf(stderr, "xorg_cli: width and height must be >= 2\n");
return 1;
}
Test_Frame *f = test_image_alloc(width, height, fmt);
if (!f) {
fprintf(stderr, "xorg_cli: allocation failed\n");
return 1;
}
test_image_generate(f, pattern);
const char *pat_name = pattern == TEST_PATTERN_BARS ? "bars"
: pattern == TEST_PATTERN_RAMP ? "ramp"
: "grid";
const char *fmt_name = fmt == TEST_FMT_YUV420 ? "yuv420" : "bgra";
const char *scale_name = scale == XORG_SCALE_STRETCH ? "stretch"
: scale == XORG_SCALE_FIT ? "fit"
: scale == XORG_SCALE_FILL ? "fill"
: "1:1";
const char *anchor_name = anchor == XORG_ANCHOR_CENTER ? "center" : "topleft";
printf("opening %dx%d %s %s scale=%s anchor=%s at (%d,%d)\n",
width, height, fmt_name, pat_name, scale_name, anchor_name, win_x, win_y);
Xorg_Viewer *v = xorg_viewer_open(win_x, win_y, width, height, "xorg_cli");
xorg_viewer_set_scale(v, scale);
xorg_viewer_set_anchor(v, anchor);
if (!v) {
fprintf(stderr, "xorg_cli: failed to open viewer window\n");
test_image_free(f);
return 1;
}
if (fmt == TEST_FMT_YUV420) {
xorg_viewer_push_yuv420(v,
f->plane[0], f->plane[1], f->plane[2],
f->width, f->height);
} else {
xorg_viewer_push_bgra(v, f->plane[0], f->width, f->height);
}
test_image_free(f);
while (xorg_viewer_poll(v)) { /* wait for window close */ }
xorg_viewer_close(v);
return 0;
}

24
include/xorg.h
View File

@@ -6,6 +6,26 @@
typedef struct Xorg_Viewer Xorg_Viewer; typedef struct Xorg_Viewer Xorg_Viewer;
/*
* How the frame is scaled to fit the window.
* SCALE_STRETCH is the only implemented mode; others are reserved.
*/
typedef enum Xorg_Scale {
XORG_SCALE_STRETCH, /* fill window, ignore aspect ratio (default) */
XORG_SCALE_FIT, /* largest rect that fits, preserve aspect, black bars */
XORG_SCALE_FILL, /* smallest rect that covers, preserve aspect, crop edges */
XORG_SCALE_1_1, /* native pixel size, no scaling */
} Xorg_Scale;
/*
* Where the frame is positioned within the window.
* Used with XORG_SCALE_FIT, XORG_SCALE_FILL, and XORG_SCALE_1_1.
*/
typedef enum Xorg_Anchor {
XORG_ANCHOR_CENTER, /* center frame in window (default) */
XORG_ANCHOR_TOP_LEFT, /* align frame to top-left corner */
} Xorg_Anchor;
/* Returns false when compiled without HAVE_GLFW. */ /* Returns false when compiled without HAVE_GLFW. */
bool xorg_available(void); bool xorg_available(void);
@@ -32,6 +52,10 @@ bool xorg_viewer_push_bgra(Xorg_Viewer *v,
bool xorg_viewer_push_mjpeg(Xorg_Viewer *v, bool xorg_viewer_push_mjpeg(Xorg_Viewer *v,
const uint8_t *data, size_t size); const uint8_t *data, size_t size);
/* Change scale/anchor at any time; takes effect on the next render. */
void xorg_viewer_set_scale(Xorg_Viewer *v, Xorg_Scale scale);
void xorg_viewer_set_anchor(Xorg_Viewer *v, Xorg_Anchor anchor);
/* /*
* Process pending window events. * Process pending window events.
* Returns false when the user has closed the window. * Returns false when the user has closed the window.

256
src/modules/xorg/xorg.c
View File

@@ -16,17 +16,20 @@
/* ------------------------------------------------------------------ */ /* ------------------------------------------------------------------ */
/* /*
* Full-screen quad via gl_VertexID — no VBO needed. * Full-screen quad via gl_VertexID.
* UV is flipped on Y so that image row 0 appears at the top of the window * u_uv_scale / u_uv_offset let render() apply UV crop for FILL and 1:1 modes.
* (OpenGL texture origin is bottom-left; image data is stored top-row-first). * UV Y is flipped: image row 0 appears at the top of the window.
*/ */
static const char *VERT_SRC = static const char *VERT_SRC =
"#version 330 core\n" "#version 330 core\n"
"out vec2 v_uv;\n" "out vec2 v_uv;\n"
"uniform vec2 u_uv_scale;\n"
"uniform vec2 u_uv_offset;\n"
"void main() {\n" "void main() {\n"
" float x = float(gl_VertexID & 1) * 2.0 - 1.0;\n" " float x = float(gl_VertexID & 1) * 2.0 - 1.0;\n"
" float y = float((gl_VertexID >> 1) & 1) * 2.0 - 1.0;\n" " float y = float((gl_VertexID >> 1) & 1) * 2.0 - 1.0;\n"
" v_uv = vec2((x + 1.0) * 0.5, (1.0 - y) * 0.5);\n" " vec2 uv = vec2((x + 1.0) * 0.5, (1.0 - y) * 0.5);\n"
" v_uv = uv * u_uv_scale + u_uv_offset;\n"
" gl_Position = vec4(x, y, 0.0, 1.0);\n" " gl_Position = vec4(x, y, 0.0, 1.0);\n"
"}\n"; "}\n";
@@ -62,17 +65,26 @@ static const char *FRAG_RGB_SRC =
/* Viewer state */ /* Viewer state */
/* ------------------------------------------------------------------ */ /* ------------------------------------------------------------------ */
typedef enum { MODE_NONE, MODE_YUV, MODE_RGB } Render_Mode;
struct Xorg_Viewer { struct Xorg_Viewer {
GLFWwindow *window; GLFWwindow *window;
GLuint prog_yuv; GLuint prog_yuv;
GLint u_tex_y, u_tex_cb, u_tex_cr; GLint u_tex_y, u_tex_cb, u_tex_cr;
GLint u_uv_scale_yuv, u_uv_offset_yuv;
GLuint prog_rgb; GLuint prog_rgb;
GLint u_tex_rgb; GLint u_tex_rgb;
GLint u_uv_scale_rgb, u_uv_offset_rgb;
GLuint vao; GLuint vao;
GLuint tex[4]; /* 0=Y 1=Cb 2=Cr 3=BGRA/RGB */ GLuint tex[4]; /* 0=Y 1=Cb 2=Cr 3=BGRA/RGB */
Render_Mode mode;
Xorg_Scale scale;
Xorg_Anchor anchor;
int frame_w, frame_h;
#ifdef HAVE_TURBOJPEG #ifdef HAVE_TURBOJPEG
tjhandle tj; tjhandle tj;
@@ -133,8 +145,27 @@ static GLuint link_program(const char *frag_src)
/* Public API */ /* Public API */
/* ------------------------------------------------------------------ */ /* ------------------------------------------------------------------ */
static void render(Xorg_Viewer *v); /* forward declaration */
bool xorg_available(void) { return true; } bool xorg_available(void) { return true; }
static void key_callback(GLFWwindow *window, int key, int scancode,
int action, int mods)
{
(void)scancode; (void)mods;
if (action == GLFW_PRESS &&
(key == GLFW_KEY_ESCAPE || key == GLFW_KEY_Q)) {
glfwSetWindowShouldClose(window, GLFW_TRUE);
}
}
static void framebuffer_size_callback(GLFWwindow *window, int width, int height)
{
(void)width; (void)height;
Xorg_Viewer *v = glfwGetWindowUserPointer(window);
if (v) { render(v); }
}
Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height, Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height,
const char *title) const char *title)
{ {
@@ -154,8 +185,9 @@ Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height,
return NULL; return NULL;
} }
glfwSetWindowPos(win, x, y); glfwSetWindowPos(win, x, y);
glfwSetKeyCallback(win, key_callback);
glfwMakeContextCurrent(win); glfwMakeContextCurrent(win);
glfwSwapInterval(1); /* vsync */ glfwSwapInterval(1);
glewExperimental = GL_TRUE; glewExperimental = GL_TRUE;
if (glewInit() != GLEW_OK) { if (glewInit() != GLEW_OK) {
@@ -172,6 +204,10 @@ Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height,
return NULL; return NULL;
} }
v->window = win; v->window = win;
v->scale = XORG_SCALE_STRETCH;
v->anchor = XORG_ANCHOR_CENTER;
glfwSetWindowUserPointer(win, v);
glfwSetFramebufferSizeCallback(win, framebuffer_size_callback);
v->prog_yuv = link_program(FRAG_YUV_SRC); v->prog_yuv = link_program(FRAG_YUV_SRC);
v->prog_rgb = link_program(FRAG_RGB_SRC); v->prog_rgb = link_program(FRAG_RGB_SRC);
@@ -180,10 +216,15 @@ Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height,
return NULL; return NULL;
} }
v->u_tex_y = glGetUniformLocation(v->prog_yuv, "u_tex_y"); v->u_tex_y = glGetUniformLocation(v->prog_yuv, "u_tex_y");
v->u_tex_cb = glGetUniformLocation(v->prog_yuv, "u_tex_cb"); v->u_tex_cb = glGetUniformLocation(v->prog_yuv, "u_tex_cb");
v->u_tex_cr = glGetUniformLocation(v->prog_yuv, "u_tex_cr"); v->u_tex_cr = glGetUniformLocation(v->prog_yuv, "u_tex_cr");
v->u_tex_rgb = glGetUniformLocation(v->prog_rgb, "u_tex_rgb"); v->u_uv_scale_yuv = glGetUniformLocation(v->prog_yuv, "u_uv_scale");
v->u_uv_offset_yuv= glGetUniformLocation(v->prog_yuv, "u_uv_offset");
v->u_tex_rgb = glGetUniformLocation(v->prog_rgb, "u_tex_rgb");
v->u_uv_scale_rgb = glGetUniformLocation(v->prog_rgb, "u_uv_scale");
v->u_uv_offset_rgb= glGetUniformLocation(v->prog_rgb, "u_uv_offset");
glGenVertexArrays(1, &v->vao); glGenVertexArrays(1, &v->vao);
glGenTextures(4, v->tex); glGenTextures(4, v->tex);
@@ -209,8 +250,144 @@ Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height,
return v; return v;
} }
void xorg_viewer_set_scale(Xorg_Viewer *v, Xorg_Scale scale)
{
if (v) { v->scale = scale; }
}
void xorg_viewer_set_anchor(Xorg_Viewer *v, Xorg_Anchor anchor)
{
if (v) { v->anchor = anchor; }
}
/* ------------------------------------------------------------------ */ /* ------------------------------------------------------------------ */
/* Internal: upload YUV planes and render */ /* Internal: compute layout and render from existing textures */
/* ------------------------------------------------------------------ */
static void render(Xorg_Viewer *v)
{
if (v->mode == MODE_NONE) { return; }
int fb_w, fb_h;
glfwGetFramebufferSize(v->window, &fb_w, &fb_h);
int vp_x = 0, vp_y = 0, vp_w = fb_w, vp_h = fb_h;
float uv_sx = 1.0f, uv_sy = 1.0f;
float uv_ox = 0.0f, uv_oy = 0.0f;
int fw = v->frame_w, fh = v->frame_h;
if (fw > 0 && fh > 0 && v->scale != XORG_SCALE_STRETCH) {
float fa = (float)fw / fh;
float wa = (float)fb_w / fb_h;
switch (v->scale) {
case XORG_SCALE_FIT:
/* Largest rect that fits; black bars fill the rest. */
if (fa > wa) {
vp_w = fb_w;
vp_h = (int)(fb_w / fa);
} else {
vp_h = fb_h;
vp_w = (int)(fb_h * fa);
}
vp_x = (v->anchor == XORG_ANCHOR_CENTER) ? (fb_w - vp_w) / 2 : 0;
vp_y = (v->anchor == XORG_ANCHOR_CENTER) ? (fb_h - vp_h) / 2
: fb_h - vp_h;
break;
case XORG_SCALE_FILL:
/*
* Scale to cover the window; crop the overflowing axis.
* UV range is narrowed on the cropped axis.
*/
if (fa > wa) {
/* Frame wider than window: fit height, crop width. */
uv_sx = wa / fa;
uv_ox = (v->anchor == XORG_ANCHOR_CENTER)
? (1.0f - uv_sx) * 0.5f : 0.0f;
} else {
/* Frame taller than window: fit width, crop height. */
uv_sy = fa / wa;
uv_oy = (v->anchor == XORG_ANCHOR_CENTER)
? (1.0f - uv_sy) * 0.5f : 0.0f;
}
break;
case XORG_SCALE_1_1:
/*
* One frame pixel = one screen pixel.
* If frame is larger than the window, the excess is cropped.
*/
vp_w = fw < fb_w ? fw : fb_w;
vp_h = fh < fb_h ? fh : fb_h;
if (v->anchor == XORG_ANCHOR_CENTER) {
vp_x = (fb_w - fw) / 2;
vp_y = (fb_h - fh) / 2;
/* If frame overflows, crop from centre via UV offset. */
if (vp_x < 0) {
uv_ox = (float)(-vp_x) / fw;
uv_sx = (float)vp_w / fw;
vp_x = 0;
}
if (vp_y < 0) {
uv_oy = (float)(-vp_y) / fh;
uv_sy = (float)vp_h / fh;
vp_y = 0;
}
} else {
/* Top-left anchor: show top-left portion of frame. */
vp_x = 0;
vp_y = fb_h - vp_h;
uv_sx = (float)vp_w / fw;
uv_sy = (float)vp_h / fh;
}
break;
case XORG_SCALE_STRETCH:
break;
}
}
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
glViewport(vp_x, vp_y, vp_w, vp_h);
GLuint prog = (v->mode == MODE_YUV) ? v->prog_yuv : v->prog_rgb;
GLint u_uv_scale = (v->mode == MODE_YUV) ? v->u_uv_scale_yuv : v->u_uv_scale_rgb;
GLint u_uv_off = (v->mode == MODE_YUV) ? v->u_uv_offset_yuv : v->u_uv_offset_rgb;
glUseProgram(prog);
glUniform2f(u_uv_scale, uv_sx, uv_sy);
glUniform2f(u_uv_off, uv_ox, uv_oy);
if (v->mode == MODE_YUV) {
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, v->tex[0]);
glUniform1i(v->u_tex_y, 0);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, v->tex[1]);
glUniform1i(v->u_tex_cb, 1);
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, v->tex[2]);
glUniform1i(v->u_tex_cr, 2);
} else {
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, v->tex[3]);
glUniform1i(v->u_tex_rgb, 0);
}
glBindVertexArray(v->vao);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glBindVertexArray(0);
glfwSwapBuffers(v->window);
}
/* ------------------------------------------------------------------ */
/* Internal: upload YUV planes */
/* ------------------------------------------------------------------ */ /* ------------------------------------------------------------------ */
static void upload_yuv(Xorg_Viewer *v, static void upload_yuv(Xorg_Viewer *v,
@@ -234,29 +411,8 @@ static void upload_yuv(Xorg_Viewer *v,
glBindTexture(GL_TEXTURE_2D, 0); glBindTexture(GL_TEXTURE_2D, 0);
int fb_w, fb_h; v->mode = MODE_YUV;
glfwGetFramebufferSize(v->window, &fb_w, &fb_h); render(v);
glViewport(0, 0, fb_w, fb_h);
glUseProgram(v->prog_yuv);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, v->tex[0]);
glUniform1i(v->u_tex_y, 0);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, v->tex[1]);
glUniform1i(v->u_tex_cb, 1);
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, v->tex[2]);
glUniform1i(v->u_tex_cr, 2);
glBindVertexArray(v->vao);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glBindVertexArray(0);
glfwSwapBuffers(v->window);
} }
/* ------------------------------------------------------------------ */ /* ------------------------------------------------------------------ */
@@ -268,6 +424,8 @@ bool xorg_viewer_push_yuv420(Xorg_Viewer *v,
int width, int height) int width, int height)
{ {
if (!v) { return false; } if (!v) { return false; }
v->frame_w = width;
v->frame_h = height;
upload_yuv(v, y, width, height, cb, width / 2, height / 2, cr); upload_yuv(v, y, width, height, cb, width / 2, height / 2, cr);
return true; return true;
} }
@@ -277,25 +435,16 @@ bool xorg_viewer_push_bgra(Xorg_Viewer *v,
{ {
if (!v) { return false; } if (!v) { return false; }
v->frame_w = width;
v->frame_h = height;
glBindTexture(GL_TEXTURE_2D, v->tex[3]); glBindTexture(GL_TEXTURE_2D, v->tex[3]);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, width, height, 0, glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, width, height, 0,
GL_BGRA, GL_UNSIGNED_BYTE, data); GL_BGRA, GL_UNSIGNED_BYTE, data);
glBindTexture(GL_TEXTURE_2D, 0); glBindTexture(GL_TEXTURE_2D, 0);
int fb_w, fb_h; v->mode = MODE_RGB;
glfwGetFramebufferSize(v->window, &fb_w, &fb_h); render(v);
glViewport(0, 0, fb_w, fb_h);
glUseProgram(v->prog_rgb);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, v->tex[3]);
glUniform1i(v->u_tex_rgb, 0);
glBindVertexArray(v->vao);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glBindVertexArray(0);
glfwSwapBuffers(v->window);
return true; return true;
} }
@@ -315,7 +464,6 @@ bool xorg_viewer_push_mjpeg(Xorg_Viewer *v,
return false; return false;
} }
/* Reallocate if dimensions or subsampling changed. */
if (w != v->tj_width || h != v->tj_height || subsamp != v->tj_subsamp) { if (w != v->tj_width || h != v->tj_height || subsamp != v->tj_subsamp) {
free(v->yuv_buf); free(v->yuv_buf);
v->yuv_buf = malloc(tjBufSizeYUV2(w, 1, h, subsamp)); v->yuv_buf = malloc(tjBufSizeYUV2(w, 1, h, subsamp));
@@ -342,6 +490,8 @@ bool xorg_viewer_push_mjpeg(Xorg_Viewer *v,
return false; return false;
} }
v->frame_w = w;
v->frame_h = h;
upload_yuv(v, planes[0], y_w, y_h, planes[1], c_w, c_h, planes[2]); upload_yuv(v, planes[0], y_w, y_h, planes[1], c_w, c_h, planes[2]);
return true; return true;
#endif #endif
@@ -355,7 +505,9 @@ bool xorg_viewer_poll(Xorg_Viewer *v)
{ {
if (!v || glfwWindowShouldClose(v->window)) { return false; } if (!v || glfwWindowShouldClose(v->window)) { return false; }
glfwPollEvents(); glfwPollEvents();
return !glfwWindowShouldClose(v->window); if (glfwWindowShouldClose(v->window)) { return false; }
render(v);
return true;
} }
void xorg_viewer_close(Xorg_Viewer *v) void xorg_viewer_close(Xorg_Viewer *v)

2
src/modules/xorg/xorg_stub.c
View File

@@ -32,5 +32,7 @@ bool xorg_viewer_push_mjpeg(Xorg_Viewer *v,
return false; return false;
} }
void xorg_viewer_set_scale(Xorg_Viewer *v, Xorg_Scale scale) { (void)v; (void)scale; }
void xorg_viewer_set_anchor(Xorg_Viewer *v, Xorg_Anchor anchor) { (void)v; (void)anchor; }
bool xorg_viewer_poll(Xorg_Viewer *v) { (void)v; return false; } bool xorg_viewer_poll(Xorg_Viewer *v) { (void)v; return false; }
void xorg_viewer_close(Xorg_Viewer *v) { (void)v; } void xorg_viewer_close(Xorg_Viewer *v) { (void)v; }