mikael-lovqvists-claude-agent/video-setup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: xorg text overlays, font atlas generator, v4l2_view_cli

Browse Source 
- tools/gen_font_atlas: Python/Pillow build tool — skyline packs DejaVu
  Sans glyphs 32-255 into a grayscale atlas, emits build/gen/font_atlas.h
  with pixel data and Font_Glyph[256] metrics table
- xorg: bitmap font atlas text overlay rendering (GL_R8 atlas texture,
  alpha-blended glyph quads, dark background rect per overlay)
- xorg: add xorg_viewer_set_overlay_text / clear_overlays API
- xorg: add xorg_viewer_handle_events for streaming use (events only,
  no redundant render)
- xorg_cli: show today's date as white text overlay
- v4l2_view_cli: new tool — V4L2 capture with format auto-selection
  (highest FPS then largest resolution), MJPEG/YUYV, measured FPS overlay
- docs: update README, planning, architecture to reflect current status

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
mikael-lovqvists-claude-agent
2026-03-28 22:13:59 +00:00
parent 7fd79e6120
commit 611376dbc1
11 changed files with 1204 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
README.md
View File

@@ -15,6 +15,9 @@ Designed to run on resource-constrained hardware (Raspberry Pi capturing raw MJP
- [docs/cli/media_ctrl_cli.md](docs/cli/media_ctrl_cli.md) — media device and topology tool
- [docs/cli/v4l2_ctrl_cli.md](docs/cli/v4l2_ctrl_cli.md) — V4L2 camera control tool
- `test_image_cli` — generate test patterns and write PPM output for visual inspection
- `xorg_cli` — display a test pattern in the viewer window; exercises scale/anchor modes and text overlays
- `v4l2_view_cli` — live camera viewer; auto-selects highest-FPS format, displays FPS overlay
## Structure
@@ -23,12 +26,13 @@ src/modules/ C modules, one directory each
include/ public headers
dev/cli/ CLI driver programs for each module
dev/web/ development web UI (Node.js/Express) — connects to live nodes for V4L2 inspection and control
tools/ build-time code generators (e.g. gen_font_atlas — bitmap font atlas for xorg text overlays)
docs/ documentation
```
## Status
Core modules and the video node binary are working. The node can be queried over the wire protocol for device enumeration and V4L2 camera control. The development web UI connects to live nodes for inspection and control. Frame capture and relay have not started.
Core modules and the video node binary are working. The node can be queried over the wire protocol for device enumeration and V4L2 camera control. The development web UI connects to live nodes for inspection and control. The xorg viewer sink is implemented (GLFW+OpenGL, all scale/anchor modes, bitmap font atlas text overlays). A V4L2 capture viewer (`v4l2_view_cli`) demonstrates live camera display without going through the node system. Frame ingest-to-wire and relay have not started.
| Module | Status | Notes |
|---|---|---|
@@ -42,10 +46,11 @@ Core modules and the video node binary are working. The node can be queried over
| `protocol` | done | Typed write/read functions for all message types |
| `node` | done | Video node binary — config, discovery, transport server, V4L2/media request handlers |
| `dev/web` | done | Development web UI — connects to live nodes, V4L2 inspection and control |
| `test_image` | done | Test pattern generator — colour bars, luminance ramp, grid; YUV420/BGRA output |
| `xorg` | done | GLFW+OpenGL viewer sink — YUV420/BGRA/MJPEG input, all scale/anchor modes, bitmap font atlas text overlays; screen grab and XRandR queries not yet implemented |
| `frame_alloc` | not started | Per-frame allocation with byte budget and ref counting |
| `relay` | not started | Input dispatch to output queues (low-latency and completeness modes) |
| `ingest` | not started | V4L2 capture loop — dequeue buffers, emit frames |
| `archive` | not started | Write frames to disk, control messages to binary log |
| `codec` | not started | Per-frame encode/decode (MJPEG, QOI, ZSTD-raw, VA-API H.264) |
| `xorg` | not started | X11 geometry (XRandR), screen grab source, frame viewer sink |
| `web node` | not started | Node.js peer — binary protocol socket side + HTTP/WebSocket to browser |

39
architecture.md
View File

@@ -418,7 +418,7 @@ The initial implementation uses **GLFW** for window and input management and **O
GLFW handles window creation, the event loop, resize, and input callbacks — it also supports Vulkan surface creation using the same API, which makes a future renderer swap straightforward. Input events (keyboard, mouse) are normalised by GLFW before being encoded as protocol messages.
The OpenGL renderer:
1. For **MJPEG**: calls `tjDecompressToYUV2` (libjpeg-turbo) to decompress directly to planar YUV — no CPU-side color conversion. JPEG stores YCbCr internally so this is the minimal decode path: Huffman + DCT output lands directly in YUV planes.
1. For **MJPEG**: calls `tjDecompressToYUVPlanes` (libjpeg-turbo) to decompress directly to planar YUV — no CPU-side color conversion. JPEG stores YCbCr internally so this is the minimal decode path: Huffman + DCT output lands directly in YUV planes.
2. Uploads Y, Cb, Cr as separate `GL_RED` textures (chroma at half resolution for 4:2:0 / 4:2:2 as delivered by most V4L2 cameras).
3. Fragment shader samples the three planes and applies the BT.601 matrix to produce RGB — a few lines of GLSL.
4. Scaling and filtering happen in the same shader pass.
@@ -428,17 +428,19 @@ For **raw pixel formats** (BGRA, YUV planar from the wire): uploaded directly wi
This keeps CPU load minimal — the only CPU work for MJPEG is Huffman decode and DCT, which libjpeg-turbo runs with SIMD. All color conversion and scaling is on the GPU.
#### Text overlays (future)
#### Text overlays
Two tiers are planned, implemented in order:
Two tiers, implemented in order:
**Tier 1 — bitmap font atlas (initial)**
**Tier 1 — bitmap font atlas (done)**
A build-time script (Python Pillow) renders glyphs from a TTF font into a packed PNG atlas and emits a metadata file (JSON or generated C header) with per-glyph UV rects and advance widths. At runtime the atlas is uploaded as a `GL_RGBA` texture and each character is rendered as a small quad, alpha-blended over the frame. Simple skyline packing keeps the atlas compact.
`tools/gen_font_atlas/gen_font_atlas.py` (Python/Pillow) renders glyphs 32–255 from DejaVu Sans at 16pt into a packed grayscale atlas using a skyline bin packer and emits `build/gen/font_atlas.h` — a C header with the pixel data as a `static const uint8_t` array and a `Font_Glyph[256]` metrics table indexed by codepoint.
The generator lives in `tools/gen_font_atlas/` and runs as part of `make build`. Sufficient for ASCII overlays: timestamps, stream labels, debug info.
At runtime the atlas is uploaded as a `GL_R8` texture. Each overlay is rendered as a batch of alpha-blended glyph quads preceded by a semi-transparent dark background rect (using a separate minimal screen-space rect shader driven by `gl_VertexID`). The public API is `xorg_viewer_set_overlay_text(v, idx, x, y, text, r, g, b)` and `xorg_viewer_clear_overlays(v)`. Up to 8 independent overlays are supported.
**Tier 2 — HarfBuzz + FreeType (later)**
The generator runs automatically as a `make` dependency before compiling `xorg.c`. The Pillow build tool is the only Python dependency; there are no runtime font deps.
**Tier 2 — HarfBuzz + FreeType (future)**
A proper runtime font stack for full typography: correct shaping, kerning, ligatures, bidirectional text, non-Latin scripts. Added as a feature flag with its own runtime deps alongside the blit path.
@@ -446,20 +448,27 @@ When Tier 2 is implemented, the Pillow build dependency may be replaced by a pur
#### Render loop
The viewer is driven by incoming frames rather than a fixed-rate loop. The intended pattern for callers:
The viewer is driven by incoming frames rather than a fixed-rate loop. Two polling functions are provided depending on the use case:
**Static image / test tool** — `xorg_viewer_poll(v)` processes events then re-renders from existing textures:
```c
while (xorg_viewer_poll(v)) {
if (new_frame_available()) {
xorg_viewer_push_yuv420(v, ...); /* upload + render */
while (xorg_viewer_poll(v)) { /* wait for close */ }
```
**Live stream** — the push functions (`push_yuv420`, `push_mjpeg`, etc.) already upload and render. Use `xorg_viewer_handle_events(v)` to process window events without an extra render:
```c
while (1) {
/* block on V4L2/network fd until frame or timeout */
if (frame_available) {
xorg_viewer_push_mjpeg(v, data, size); /* upload + render */
}
/* no new frame → no redundant GPU work */
if (!xorg_viewer_handle_events(v)) { break; }
}
```
`xorg_viewer_poll` calls `glfwPollEvents` which dispatches input and resize events. A `framebuffer_size_callback` registered on the window calls `render()` synchronously during the resize, so the image tracks the window edge without a one-frame lag. This avoids both a busy render loop and the latency of waiting for the next poll iteration.
For a static image (test tool, paused stream), `glfwWaitEventsTimeout(interval)` is a better substitute for `glfwPollEvents` — it sleeps until an event arrives or the timeout expires, eliminating idle CPU usage.
A `framebuffer_size_callback` registered on the window calls `render()` synchronously during resize, so the image tracks the window edge without a one-frame lag.
Threading note: the GL context must be used from the thread that created it. In the video node, incoming frames arrive on a network receive thread. A frame queue between the receive thread and the render thread (which owns the GL context) is the correct model — the render thread drains the queue each poll iteration rather than having the network thread call push functions directly.

12
dev/cli/Makefile
View File

@@ -22,7 +22,8 @@ CLI_SRCS = \
protocol_cli.c \
query_cli.c \
test_image_cli.c \
xorg_cli.c
xorg_cli.c \
v4l2_view_cli.c
CLI_OBJS = $(CLI_SRCS:%.c=$(CLI_BUILD)/%.o)
@@ -37,7 +38,8 @@ all: \
$(CLI_BUILD)/protocol_cli \
$(CLI_BUILD)/query_cli \
$(CLI_BUILD)/test_image_cli \
$(CLI_BUILD)/xorg_cli
$(CLI_BUILD)/xorg_cli \
$(CLI_BUILD)/v4l2_view_cli
# Module objects delegate to their sub-makes.
$(COMMON_OBJ): ; $(MAKE) -C $(ROOT)/src/modules/common
@@ -83,6 +85,9 @@ $(CLI_BUILD)/test_image_cli: $(CLI_BUILD)/test_image_cli.o $(TEST_IMAGE_OBJ)
$(CLI_BUILD)/xorg_cli: $(CLI_BUILD)/xorg_cli.o $(TEST_IMAGE_OBJ) $(XORG_OBJ)
$(CC) $(CFLAGS) -o $@ $^ $(PKG_LDFLAGS)
$(CLI_BUILD)/v4l2_view_cli: $(CLI_BUILD)/v4l2_view_cli.o $(XORG_OBJ)
$(CC) $(CFLAGS) -o $@ $^ $(PKG_LDFLAGS)
$(CLI_BUILD):
mkdir -p $@
@@ -98,6 +103,7 @@ clean:
$(CLI_BUILD)/protocol_cli \
$(CLI_BUILD)/query_cli \
$(CLI_BUILD)/test_image_cli \
$(CLI_BUILD)/xorg_cli
$(CLI_BUILD)/xorg_cli \
$(CLI_BUILD)/v4l2_view_cli
-include $(CLI_OBJS:%.o=%.d)

537
dev/cli/v4l2_view_cli.c Normal file
View File

@@ -0,0 +1,537 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <time.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/select.h>
#include <linux/videodev2.h>
#include "xorg.h"
#define N_BUFS 4
#define MAX_OPTS 1024
/* ------------------------------------------------------------------ */
/* Format option — one (pixfmt, size, fps) combination */
/* ------------------------------------------------------------------ */
typedef struct {
uint32_t pixfmt;
int w, h;
int fps_n; /* fps = fps_n / fps_d */
int fps_d;
} Fmt_Option;
/* fps_a > fps_b ? */
static int fps_gt(const Fmt_Option *a, const Fmt_Option *b)
{
return (long long)a->fps_n * b->fps_d > (long long)b->fps_n * a->fps_d;
}
static int fps_eq(const Fmt_Option *a, const Fmt_Option *b)
{
return (long long)a->fps_n * b->fps_d == (long long)b->fps_n * a->fps_d;
}
/* ------------------------------------------------------------------ */
/* Format enumeration */
/* ------------------------------------------------------------------ */
typedef struct {
Fmt_Option *opts;
int n;
int max;
} Opt_List;
static void opt_push(Opt_List *l, uint32_t pixfmt, int w, int h, int fps_n, int fps_d)
{
if (l->n >= l->max) { return; }
l->opts[l->n++] = (Fmt_Option){ pixfmt, w, h, fps_n, fps_d };
}
static int xioctl(int fd, unsigned long req, void *arg)
{
int r;
do { r = ioctl(fd, req, arg); } while (r == -1 && errno == EINTR);
return r;
}
static void collect_intervals(int fd, uint32_t pixfmt, int w, int h, Opt_List *l)
{
struct v4l2_frmivalenum fie = {0};
fie.pixel_format = pixfmt;
fie.width = (uint32_t)w;
fie.height = (uint32_t)h;
for (fie.index = 0; xioctl(fd, VIDIOC_ENUM_FRAMEINTERVALS, &fie) == 0; fie.index++) {
if (fie.type == V4L2_FRMIVAL_TYPE_DISCRETE) {
opt_push(l, pixfmt, w, h,
(int)fie.discrete.denominator,
(int)fie.discrete.numerator);
} else {
/* Stepwise/continuous: record the fastest (minimum) interval. */
opt_push(l, pixfmt, w, h,
(int)fie.stepwise.min.denominator,
(int)fie.stepwise.min.numerator);
break;
}
}
}
static void collect_sizes(int fd, uint32_t pixfmt, Opt_List *l)
{
struct v4l2_frmsizeenum fse = {0};
fse.pixel_format = pixfmt;
for (fse.index = 0; xioctl(fd, VIDIOC_ENUM_FRAMESIZES, &fse) == 0; fse.index++) {
if (fse.type == V4L2_FRMSIZE_TYPE_DISCRETE) {
collect_intervals(fd, pixfmt,
(int)fse.discrete.width,
(int)fse.discrete.height, l);
} else {
/* Stepwise/continuous: only probe the maximum size. */
collect_intervals(fd, pixfmt,
(int)fse.stepwise.max_width,
(int)fse.stepwise.max_height, l);
break;
}
}
}
/*
* Enumerate all (pixfmt, size, fps) combinations the device supports.
* Filtered to formats we can handle (MJPEG, YUYV).
* If fmt_filter is non-zero, only that pixel format is considered.
*/
static int enumerate_formats(int fd, Fmt_Option *buf, int buf_max,
uint32_t fmt_filter)
{
Opt_List l = { buf, 0, buf_max };
struct v4l2_fmtdesc fd_desc = {0};
fd_desc.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
for (fd_desc.index = 0;
xioctl(fd, VIDIOC_ENUM_FMT, &fd_desc) == 0;
fd_desc.index++) {
uint32_t pf = fd_desc.pixelformat;
if (pf != V4L2_PIX_FMT_MJPEG && pf != V4L2_PIX_FMT_YUYV) { continue; }
if (fmt_filter && pf != fmt_filter) { continue; }
collect_sizes(fd, pf, &l);
}
return l.n;
}
/*
* Select the best option from the list:
* 1. Highest FPS
* 2. Largest area (w×h) among equal-FPS entries
* 3. MJPEG preferred over YUYV for equal FPS and area
*/
static const Fmt_Option *select_best(const Fmt_Option *opts, int n)
{
if (n == 0) { return NULL; }
const Fmt_Option *best = &opts[0];
for (int i = 1; i < n; i++) {
const Fmt_Option *o = &opts[i];
if (fps_gt(o, best)) {
best = o;
} else if (fps_eq(o, best)) {
int o_area = o->w * o->h;
int b_area = best->w * best->h;
if (o_area > b_area) {
best = o;
} else if (o_area == b_area &&
o->pixfmt == V4L2_PIX_FMT_MJPEG &&
best->pixfmt != V4L2_PIX_FMT_MJPEG) {
best = o;
}
}
}
return best;
}
/* ------------------------------------------------------------------ */
/* YUYV → planar YUV420 conversion */
/* ------------------------------------------------------------------ */
static void yuyv_to_yuv420(const uint8_t *yuyv, int stride,
int w, int h,
uint8_t *y_out, uint8_t *cb_out, uint8_t *cr_out)
{
for (int row = 0; row < h; row++) {
const uint8_t *src = yuyv + row * stride;
uint8_t *dst = y_out + row * w;
for (int col = 0; col < w; col++) {
dst[col] = src[col * 2];
}
}
for (int row = 0; row < h; row += 2) {
const uint8_t *src = yuyv + row * stride;
int c_row = row / 2;
for (int col = 0; col < w; col += 2) {
cb_out[c_row * (w / 2) + col / 2] = src[col * 2 + 1];
cr_out[c_row * (w / 2) + col / 2] = src[col * 2 + 3];
}
}
}
/* ------------------------------------------------------------------ */
/* Mmap buffers */
/* ------------------------------------------------------------------ */
typedef struct { void *start; size_t length; } Mmap_Buf;
/* ------------------------------------------------------------------ */
/* Usage */
/* ------------------------------------------------------------------ */
static void usage(void)
{
fprintf(stderr,
"usage: v4l2_view_cli [--device PATH]\n"
" [--width N --height N]\n"
" [--format mjpeg|yuyv]\n"
" [--scale stretch|fit|fill|1:1]\n"
" [--anchor center|topleft]\n"
" [--x N] [--y N]\n"
"\n"
"Opens a V4L2 capture device and displays the live feed.\n"
"Without --width/--height, selects the highest-FPS mode\n"
"and within that the largest resolution.\n"
"Q or Escape closes the window.\n"
"\n"
"defaults: /dev/video0 auto fit center at 0,0\n");
}
/* ------------------------------------------------------------------ */
/* Main */
/* ------------------------------------------------------------------ */
int main(int argc, char **argv)
{
const char *device = "/dev/video0";
int req_width = 0;
int req_height = 0;
int win_x = 0;
int win_y = 0;
Xorg_Scale scale = XORG_SCALE_FIT;
Xorg_Anchor anchor = XORG_ANCHOR_CENTER;
uint32_t fmt_filter = 0; /* 0 = auto */
for (int i = 1; i < argc; i++) {
if (strcmp(argv[i], "--device") == 0 && i + 1 < argc) {
device = argv[++i];
} else if (strcmp(argv[i], "--width") == 0 && i + 1 < argc) {
req_width = atoi(argv[++i]);
} else if (strcmp(argv[i], "--height") == 0 && i + 1 < argc) {
req_height = atoi(argv[++i]);
} else if (strcmp(argv[i], "--x") == 0 && i + 1 < argc) {
win_x = atoi(argv[++i]);
} else if (strcmp(argv[i], "--y") == 0 && i + 1 < argc) {
win_y = atoi(argv[++i]);
} else if (strcmp(argv[i], "--format") == 0 && i + 1 < argc) {
i++;
if (strcmp(argv[i], "mjpeg") == 0) { fmt_filter = V4L2_PIX_FMT_MJPEG; }
else if (strcmp(argv[i], "yuyv") == 0) { fmt_filter = V4L2_PIX_FMT_YUYV; }
else { fprintf(stderr, "unknown format: %s\n", argv[i]); usage(); return 1; }
} else if (strcmp(argv[i], "--scale") == 0 && i + 1 < argc) {
i++;
if (strcmp(argv[i], "stretch") == 0) { scale = XORG_SCALE_STRETCH; }
else if (strcmp(argv[i], "fit") == 0) { scale = XORG_SCALE_FIT; }
else if (strcmp(argv[i], "fill") == 0) { scale = XORG_SCALE_FILL; }
else if (strcmp(argv[i], "1:1") == 0) { scale = XORG_SCALE_1_1; }
else { fprintf(stderr, "unknown scale: %s\n", argv[i]); usage(); return 1; }
} else if (strcmp(argv[i], "--anchor") == 0 && i + 1 < argc) {
i++;
if (strcmp(argv[i], "center") == 0) { anchor = XORG_ANCHOR_CENTER; }
else if (strcmp(argv[i], "topleft") == 0) { anchor = XORG_ANCHOR_TOP_LEFT; }
else { fprintf(stderr, "unknown anchor: %s\n", argv[i]); usage(); return 1; }
} else {
usage(); return 1;
}
}
if (!xorg_available()) {
fprintf(stderr, "v4l2_view_cli: built without HAVE_GLFW — viewer not available\n");
return 1;
}
/* ---------------------------------------------------------------- */
/* Open V4L2 device */
/* ---------------------------------------------------------------- */
int fd = open(device, O_RDWR | O_NONBLOCK);
if (fd < 0) { perror(device); return 1; }
struct v4l2_capability cap = {0};
if (xioctl(fd, VIDIOC_QUERYCAP, &cap) < 0) {
perror("VIDIOC_QUERYCAP"); close(fd); return 1;
}
if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)) {
fprintf(stderr, "%s: not a capture device\n", device); close(fd); return 1;
}
if (!(cap.capabilities & V4L2_CAP_STREAMING)) {
fprintf(stderr, "%s: does not support streaming\n", device); close(fd); return 1;
}
/* ---------------------------------------------------------------- */
/* Format selection */
/* ---------------------------------------------------------------- */
int width, height, stride;
int use_mjpeg;
int sel_fps_n, sel_fps_d;
if (req_width > 0 && req_height > 0) {
/*
* Explicit size requested — skip enumeration, negotiate directly.
* Try MJPEG first (or whatever fmt_filter says), fall back to YUYV.
*/
struct v4l2_format fmt = {0};
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.width = (uint32_t)req_width;
fmt.fmt.pix.height = (uint32_t)req_height;
fmt.fmt.pix.field = V4L2_FIELD_ANY;
use_mjpeg = 0;
if (fmt_filter != V4L2_PIX_FMT_YUYV) {
fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_MJPEG;
if (xioctl(fd, VIDIOC_S_FMT, &fmt) == 0 &&
fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_MJPEG) {
use_mjpeg = 1;
}
}
if (!use_mjpeg) {
fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
if (xioctl(fd, VIDIOC_S_FMT, &fmt) < 0 ||
fmt.fmt.pix.pixelformat != V4L2_PIX_FMT_YUYV) {
fprintf(stderr, "%s: could not set %dx%d in MJPEG or YUYV\n",
device, req_width, req_height);
close(fd); return 1;
}
}
width = (int)fmt.fmt.pix.width;
height = (int)fmt.fmt.pix.height;
stride = (int)fmt.fmt.pix.bytesperline;
sel_fps_n = 0; sel_fps_d = 1; /* unknown until G_PARM below */
} else {
/* Enumerate all supported modes and pick the best. */
Fmt_Option *opts = malloc(MAX_OPTS * sizeof(*opts));
if (!opts) { fprintf(stderr, "out of memory\n"); close(fd); return 1; }
int n = enumerate_formats(fd, opts, MAX_OPTS, fmt_filter);
if (n == 0) {
fprintf(stderr, "%s: no usable formats found (MJPEG/YUYV)\n", device);
free(opts); close(fd); return 1;
}
const Fmt_Option *best = select_best(opts, n);
/* Apply the selected format. */
struct v4l2_format fmt = {0};
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.pixelformat = best->pixfmt;
fmt.fmt.pix.width = (uint32_t)best->w;
fmt.fmt.pix.height = (uint32_t)best->h;
fmt.fmt.pix.field = V4L2_FIELD_ANY;
if (xioctl(fd, VIDIOC_S_FMT, &fmt) < 0) {
perror("VIDIOC_S_FMT"); free(opts); close(fd); return 1;
}
use_mjpeg = (fmt.fmt.pix.pixelformat == V4L2_PIX_FMT_MJPEG);
width = (int)fmt.fmt.pix.width;
height = (int)fmt.fmt.pix.height;
stride = (int)fmt.fmt.pix.bytesperline;
sel_fps_n = best->fps_n;
sel_fps_d = best->fps_d;
free(opts);
}
/* Request the selected frame rate (driver may ignore, but try). */
{
struct v4l2_streamparm parm = {0};
parm.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
parm.parm.capture.timeperframe.numerator = (uint32_t)sel_fps_d;
parm.parm.capture.timeperframe.denominator = (uint32_t)sel_fps_n;
xioctl(fd, VIDIOC_S_PARM, &parm);
/* Read back what the driver actually set. */
if (xioctl(fd, VIDIOC_G_PARM, &parm) == 0 &&
parm.parm.capture.timeperframe.denominator > 0) {
sel_fps_n = (int)parm.parm.capture.timeperframe.denominator;
sel_fps_d = (int)parm.parm.capture.timeperframe.numerator;
}
}
printf("device: %s (%s)\n", device, (char *)cap.card);
printf("format: %s %dx%d stride=%d target=%.1f fps\n",
use_mjpeg ? "MJPEG" : "YUYV", width, height, stride,
sel_fps_d > 0 ? (double)sel_fps_n / sel_fps_d : 0.0);
/* ---------------------------------------------------------------- */
/* Mmap buffers + stream on */
/* ---------------------------------------------------------------- */
struct v4l2_requestbuffers req = {0};
req.count = N_BUFS;
req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
req.memory = V4L2_MEMORY_MMAP;
if (xioctl(fd, VIDIOC_REQBUFS, &req) < 0) {
perror("VIDIOC_REQBUFS"); close(fd); return 1;
}
Mmap_Buf bufs[N_BUFS] = {0};
for (unsigned i = 0; i < req.count; i++) {
struct v4l2_buffer buf = {0};
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
buf.index = i;
if (xioctl(fd, VIDIOC_QUERYBUF, &buf) < 0) {
perror("VIDIOC_QUERYBUF"); close(fd); return 1;
}
bufs[i].length = buf.length;
bufs[i].start = mmap(NULL, buf.length, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, buf.m.offset);
if (bufs[i].start == MAP_FAILED) {
perror("mmap"); close(fd); return 1;
}
}
for (unsigned i = 0; i < req.count; i++) {
struct v4l2_buffer buf = {0};
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
buf.index = i;
if (xioctl(fd, VIDIOC_QBUF, &buf) < 0) {
perror("VIDIOC_QBUF"); close(fd); return 1;
}
}
enum v4l2_buf_type stream_type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
if (xioctl(fd, VIDIOC_STREAMON, &stream_type) < 0) {
perror("VIDIOC_STREAMON"); close(fd); return 1;
}
/* ---------------------------------------------------------------- */
/* Open viewer */
/* ---------------------------------------------------------------- */
Xorg_Viewer *v = xorg_viewer_open(win_x, win_y, width, height, "v4l2_view_cli");
if (!v) {
fprintf(stderr, "v4l2_view_cli: failed to open viewer window\n");
xioctl(fd, VIDIOC_STREAMOFF, &stream_type);
close(fd); return 1;
}
xorg_viewer_set_scale(v, scale);
xorg_viewer_set_anchor(v, anchor);
/* ---------------------------------------------------------------- */
/* YUYV conversion buffer */
/* ---------------------------------------------------------------- */
uint8_t *yuv420_buf = NULL;
if (!use_mjpeg) {
yuv420_buf = malloc((size_t)(width * height * 3 / 2));
if (!yuv420_buf) {
fprintf(stderr, "v4l2_view_cli: out of memory\n");
xorg_viewer_close(v);
xioctl(fd, VIDIOC_STREAMOFF, &stream_type);
close(fd); return 1;
}
}
/* ---------------------------------------------------------------- */
/* Capture loop */
/* ---------------------------------------------------------------- */
struct timespec t_fps;
clock_gettime(CLOCK_MONOTONIC, &t_fps);
int fps_frame_count = 0;
float displayed_fps = 0.0f;
/* Set initial info overlay; fps will be filled in once measured. */
const char *fmt_name = use_mjpeg ? "MJPEG" : "YUYV";
{
char info[64];
snprintf(info, sizeof(info), "%s %dx%d @ --.- fps", fmt_name, width, height);
xorg_viewer_set_overlay_text(v, 0, 10, 10, info, 1.0f, 1.0f, 0.8f);
}
while (1) {
fd_set fds;
FD_ZERO(&fds);
FD_SET(fd, &fds);
struct timeval tv = {1, 0};
int r = select(fd + 1, &fds, NULL, NULL, &tv);
if (r < 0) {
if (errno == EINTR) { continue; }
perror("select"); break;
}
if (r == 0) {
fprintf(stderr, "v4l2_view_cli: select timeout — no frames\n");
continue;
}
struct v4l2_buffer buf = {0};
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
if (xioctl(fd, VIDIOC_DQBUF, &buf) < 0) {
if (errno == EAGAIN) { continue; }
perror("VIDIOC_DQBUF"); break;
}
const uint8_t *data = bufs[buf.index].start;
if (use_mjpeg) {
xorg_viewer_push_mjpeg(v, data, buf.bytesused);
} else {
uint8_t *y_p = yuv420_buf;
uint8_t *cb_p = y_p + width * height;
uint8_t *cr_p = cb_p + width * height / 4;
yuyv_to_yuv420(data, stride, width, height, y_p, cb_p, cr_p);
xorg_viewer_push_yuv420(v, y_p, cb_p, cr_p, width, height);
}
if (xioctl(fd, VIDIOC_QBUF, &buf) < 0) {
perror("VIDIOC_QBUF"); break;
}
/* Update FPS overlay every 0.5s. */
fps_frame_count++;
struct timespec now;
clock_gettime(CLOCK_MONOTONIC, &now);
double elapsed = (now.tv_sec - t_fps.tv_sec) +
(now.tv_nsec - t_fps.tv_nsec) * 1e-9;
if (elapsed >= 0.5) {
displayed_fps = (float)(fps_frame_count / elapsed);
fps_frame_count = 0;
t_fps = now;
char info[64];
snprintf(info, sizeof(info), "%s %dx%d @ %.1f fps",
fmt_name, width, height, displayed_fps);
xorg_viewer_set_overlay_text(v, 0, 10, 10, info, 1.0f, 1.0f, 0.8f);
}
if (!xorg_viewer_handle_events(v)) { break; }
}
/* ---------------------------------------------------------------- */
/* Cleanup */
/* ---------------------------------------------------------------- */
xorg_viewer_close(v);
xioctl(fd, VIDIOC_STREAMOFF, &stream_type);
for (unsigned i = 0; i < req.count; i++) {
if (bufs[i].start && bufs[i].start != MAP_FAILED) {
munmap(bufs[i].start, bufs[i].length);
}
}
free(yuv420_buf);
close(fd);
return 0;
}

15
dev/cli/xorg_cli.c
View File

@@ -1,6 +1,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "test_image.h"
#include "xorg.h"
@@ -100,14 +101,24 @@ int main(int argc, char **argv)
width, height, fmt_name, pat_name, scale_name, anchor_name, win_x, win_y);
Xorg_Viewer *v = xorg_viewer_open(win_x, win_y, width, height, "xorg_cli");
xorg_viewer_set_scale(v, scale);
xorg_viewer_set_anchor(v, anchor);
if (!v) {
fprintf(stderr, "xorg_cli: failed to open viewer window\n");
test_image_free(f);
return 1;
}
xorg_viewer_set_scale(v, scale);
xorg_viewer_set_anchor(v, anchor);
/* Overlay: today's date, white text, top-left corner. */
{
char date_buf[32];
time_t now = time(NULL);
struct tm *tm = localtime(&now);
strftime(date_buf, sizeof(date_buf), "%Y-%m-%d", tm);
xorg_viewer_set_overlay_text(v, 0, 10, 10, date_buf, 1.0f, 1.0f, 1.0f);
}
if (fmt == TEST_FMT_YUV420) {
xorg_viewer_push_yuv420(v,
f->plane[0], f->plane[1], f->plane[2],

16
include/xorg.h
View File

@@ -56,11 +56,27 @@ bool xorg_viewer_push_mjpeg(Xorg_Viewer *v,
void xorg_viewer_set_scale(Xorg_Viewer *v, Xorg_Scale scale);
void xorg_viewer_set_anchor(Xorg_Viewer *v, Xorg_Anchor anchor);
/*
* Set a text overlay at window-pixel position (x, y) — top-left of the text.
* Up to 8 overlays (idx 0..7); calling with the same idx replaces it.
* r, g, b are in [0, 1].
*/
void xorg_viewer_set_overlay_text(Xorg_Viewer *v, int idx, int x, int y,
const char *text, float r, float g, float b);
/* Remove all text overlays. */
void xorg_viewer_clear_overlays(Xorg_Viewer *v);
/*
* Process pending window events.
* Returns false when the user has closed the window.
* Must be called from the thread that created the viewer.
*
* poll() — processes events then re-renders; use for static images.
* handle_events() — processes events only, no render; use when push_* drives
* rendering (e.g. live camera feed).
*/
bool xorg_viewer_poll(Xorg_Viewer *v);
bool xorg_viewer_handle_events(Xorg_Viewer *v);
void xorg_viewer_close(Xorg_Viewer *v);

25
planning.md
View File

@@ -20,6 +20,8 @@ video-setup/
serial/ - little-endian binary serialization primitives
transport/ - framed TCP stream, single-write send
protocol/ - typed write_*/read_* message functions
test_image/ - test pattern generator (colour bars, ramp, grid; YUV420/BGRA)
xorg/ - GLFW+OpenGL viewer sink; stub for headless builds
node/ - video node entry point and top-level integration (later)
include/ - public headers
dev/
@@ -27,6 +29,9 @@ video-setup/
web/ - development web UI (Node.js/Express); browser-side equivalent
of the CLI tools; depends on protocol being finalised
experiments/ - freeform experiments
tools/
gen_font_atlas/ - build-time bitmap font atlas generator (Python/Pillow);
outputs build/gen/font_atlas.h consumed by xorg module
tests/ - automated tests (later)
Makefile
architecture.md
@@ -43,22 +48,23 @@ Modules are listed in intended build order. Each depends only on modules above i
| # | Module | Status | Notes |
|---|---|---|---|
| 1 | `common` | done | Error types, base definitions — no dependencies |
| — | `config` | done | INI file loader with schema-driven defaults, typed getters, FLAGS type for bitmask values |
| 2 | `media_ctrl` | done | Media Controller API — device and topology enumeration, pad format config |
| 3 | `v4l2_ctrl` | done | V4L2 controls — enumerate, get, set camera parameters |
| 4 | `serial` | done | `put`/`get` primitives for little-endian binary serialization into byte buffers |
| 5 | `transport` | done | Encapsulated transport — frame header, TCP stream abstraction, single-write send |
| 6 | `discovery` | done | UDP multicast announcements, peer table, found/lost callbacks |
| 8 | `protocol` | done | Typed `write_*`/`read_*` functions for all message types; builds on serial + transport |
| 7 | `protocol` | done | Typed `write_*`/`read_*` functions for all message types; builds on serial + transport |
| — | `node` | done | Video node binary — config, discovery, transport server, V4L2/media control request handlers |
| 9 | `frame_alloc` | not started | Per-frame allocation with bookkeeping (byte budget, ref counting) |
| 10 | `relay` | not started | Input dispatch to output queues (low-latency and completeness modes) |
| 11 | `ingest` | not started | V4L2 capture loop — dequeue buffers, emit one encapsulated frame per buffer |
| 12 | `archive` | not started | Write frames to disk, control messages to binary log |
| 13 | `codec` | not started | Per-frame encode/decode — MJPEG (libjpeg-turbo), QOI, ZSTD-raw, VA-API H.264 intra; used by screen grab source and archive |
| 14 | `xorg` | not started | X11 screen geometry queries (XRandR), screen grab source (calls codec), frame viewer sink — see architecture.md |
| 8 | `test_image` | done | Test pattern generator — colour bars, luminance ramp, grid crosshatch; YUV420/BGRA output |
| 9 | `xorg` | done | GLFW+OpenGL viewer sink — YUV420/BGRA/MJPEG display, all scale/anchor modes, bitmap font atlas text overlays; XRandR queries and screen grab not yet implemented |
| 10 | `frame_alloc` | not started | Per-frame allocation with bookkeeping (byte budget, ref counting) |
| 11 | `relay` | not started | Input dispatch to output queues (low-latency and completeness modes) |
| 12 | `ingest` | not started | V4L2 capture loop — dequeue buffers, emit one encapsulated frame per buffer |
| 13 | `archive` | not started | Write frames to disk, control messages to binary log |
| 14 | `codec` | not started | Per-frame encode/decode — MJPEG (libjpeg-turbo), QOI, ZSTD-raw, VA-API H.264 intra; used by screen grab source and archive |
| 15 | `web node` | not started | Node.js/Express peer — speaks binary protocol on socket side, HTTP/WebSocket to browser; `protocol.mjs` mirrors C protocol module |
| — | `mjpeg_scan` | future | EOI marker scanner for misbehaving hardware that does not deliver clean per-buffer frames; not part of the primary pipeline |
| — | `config` | done | INI file loader with schema-driven defaults, typed getters, FLAGS type for bitmask values |
---
@@ -73,6 +79,9 @@ Each module gets a corresponding CLI driver that exercises its API and serves as
| `media_ctrl_cli` | `media_ctrl` | List media devices, show topology, configure pad formats |
| `v4l2_ctrl_cli` | `v4l2_ctrl` | List controls, get/set values — lightweight `v4l2-ctl` equivalent |
| `transport_cli` | `transport` | Send/receive framed messages, inspect headers |
| `test_image_cli` | `test_image` | Generate test patterns, write PPM for visual inspection |
| `xorg_cli` | `xorg` | Display test pattern in viewer window; exercises scale/anchor modes and text overlays |
| `v4l2_view_cli` | V4L2 + `xorg` | Live camera viewer — auto-selects highest-FPS format, FPS/format overlay; bypasses node system |
### Web UI (`dev/web/`)

18
src/modules/xorg/Makefile
View File

@@ -2,12 +2,20 @@ ROOT := $(abspath ../../..)
include $(ROOT)/common.mk
MODULE_BUILD = $(BUILD)/xorg
GEN_DIR = $(BUILD)/gen
ATLAS_H = $(GEN_DIR)/font_atlas.h
ATLAS_PNG = $(GEN_DIR)/font_atlas.png
GEN_SCRIPT = $(ROOT)/tools/gen_font_atlas/gen_font_atlas.py
# Select real implementation when glfw feature is enabled, stub otherwise.
ifeq ($(filter glfw,$(FEATURES)),glfw)
SRC = xorg.c
DEPS = $(ATLAS_H)
EXTRA_CFLAGS = -I$(GEN_DIR)
else
SRC = xorg_stub.c
DEPS =
EXTRA_CFLAGS =
endif
OBJ = $(MODULE_BUILD)/xorg.o
@@ -16,12 +24,18 @@ OBJ = $(MODULE_BUILD)/xorg.o
all: $(OBJ)
$(OBJ): $(SRC) | $(MODULE_BUILD)
$(CC) $(CFLAGS) $(DEPFLAGS) -c -o $@ $<
$(OBJ): $(SRC) $(DEPS) | $(MODULE_BUILD)
$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(DEPFLAGS) -c -o $@ $<
$(ATLAS_H) $(ATLAS_PNG): $(GEN_SCRIPT) | $(GEN_DIR)
python3 $< --out-header $(ATLAS_H) --out-png $(ATLAS_PNG)
$(MODULE_BUILD):
mkdir -p $@
$(GEN_DIR):
mkdir -p $@
clean:
rm -f $(OBJ) $(MODULE_BUILD)/xorg.d

329
src/modules/xorg/xorg.c
View File

@@ -10,9 +10,10 @@
#endif
#include "xorg.h"
#include "font_atlas.h" /* generated: font_glyphs[], font_atlas_pixels[], FONT_ATLAS_W/H */
/* ------------------------------------------------------------------ */
/* Shader sources */
/* Shader sources — video */
/* ------------------------------------------------------------------ */
/*
@@ -62,14 +63,99 @@ static const char *FRAG_RGB_SRC =
"}\n";
/* ------------------------------------------------------------------ */
/* Viewer state */
/* Shader sources — solid rect */
/* ------------------------------------------------------------------ */
/*
* Draws a screen-space axis-aligned rect using gl_VertexID (no VBO).
* u_rect = (x0, y0, x1, y1) in window pixels.
*/
static const char *VERT_RECT_SRC =
"#version 330 core\n"
"uniform vec4 u_rect;\n"
"uniform vec2 u_fb_size;\n"
"void main() {\n"
" vec2 corners[4] = vec2[4](\n"
" vec2(u_rect.x, u_rect.y),\n"
" vec2(u_rect.x, u_rect.w),\n"
" vec2(u_rect.z, u_rect.y),\n"
" vec2(u_rect.z, u_rect.w)\n"
" );\n"
" vec2 pos = corners[gl_VertexID];\n"
" vec2 ndc = (pos / u_fb_size) * 2.0 - 1.0;\n"
" ndc.y = -ndc.y;\n"
" gl_Position = vec4(ndc, 0.0, 1.0);\n"
"}\n";
static const char *FRAG_RECT_SRC =
"#version 330 core\n"
"out vec4 out_color;\n"
"uniform vec4 u_color;\n"
"void main() {\n"
" out_color = u_color;\n"
"}\n";
/* ------------------------------------------------------------------ */
/* Shader sources — text overlay */
/* ------------------------------------------------------------------ */
/*
* Screen-space vertex shader: a_pos in window pixels (top-left = 0,0),
* converted to NDC. a_uv is the atlas UV coordinate.
*/
static const char *VERT_TEXT_SRC =
"#version 330 core\n"
"layout(location = 0) in vec2 a_pos;\n"
"layout(location = 1) in vec2 a_uv;\n"
"out vec2 v_uv;\n"
"uniform vec2 u_fb_size;\n"
"void main() {\n"
" vec2 ndc = (a_pos / u_fb_size) * 2.0 - 1.0;\n"
" ndc.y = -ndc.y;\n"
" gl_Position = vec4(ndc, 0.0, 1.0);\n"
" v_uv = a_uv;\n"
"}\n";
/*
* Fragment shader: samples the atlas (GL_R8, grayscale) and uses the
* texel value as alpha, blended with the per-overlay colour.
*/
static const char *FRAG_TEXT_SRC =
"#version 330 core\n"
"in vec2 v_uv;\n"
"out vec4 out_color;\n"
"uniform sampler2D u_text_atlas;\n"
"uniform vec3 u_text_color;\n"
"void main() {\n"
" float a = texture(u_text_atlas, v_uv).r;\n"
" out_color = vec4(u_text_color, a);\n"
"}\n";
/* ------------------------------------------------------------------ */
/* Constants and types */
/* ------------------------------------------------------------------ */
#define MAX_OVERLAYS 8
#define MAX_OVERLAY_CHARS 256
/*
* Each glyph quad = 6 vertices × 4 floats (x, y, u, v).
* Maximum vertex buffer covers all overlays at full text length.
*/
#define MAX_TEXT_VERTS (MAX_OVERLAYS * MAX_OVERLAY_CHARS * 6 * 4)
typedef struct {
char text[MAX_OVERLAY_CHARS];
int x, y; /* top-left of text block in window pixels */
float r, g, b;
} Overlay;
typedef enum { MODE_NONE, MODE_YUV, MODE_RGB } Render_Mode;
struct Xorg_Viewer {
GLFWwindow *window;
/* Video programs */
GLuint prog_yuv;
GLint u_tex_y, u_tex_cb, u_tex_cr;
GLint u_uv_scale_yuv, u_uv_offset_yuv;
@@ -86,6 +172,24 @@ struct Xorg_Viewer {
Xorg_Anchor anchor;
int frame_w, frame_h;
/* Solid rect (overlay background) */
GLuint prog_rect;
GLint u_rect_loc;
GLint u_rect_fb_size_loc;
GLint u_rect_color_loc;
/* Text overlay */
GLuint prog_text;
GLint u_fb_size_loc;
GLint u_text_atlas_loc;
GLint u_text_color_loc;
GLuint tex_atlas;
GLuint vao_text;
GLuint vbo_text;
Overlay overlays[MAX_OVERLAYS];
int n_overlays;
#ifdef HAVE_TURBOJPEG
tjhandle tj;
uint8_t *yuv_buf;
@@ -114,9 +218,9 @@ static GLuint compile_shader(GLenum type, const char *src)
return s;
}
static GLuint link_program(const char *frag_src)
static GLuint link_program(const char *vert_src, const char *frag_src)
{
GLuint vs = compile_shader(GL_VERTEX_SHADER, VERT_SRC);
GLuint vs = compile_shader(GL_VERTEX_SHADER, vert_src);
GLuint fs = compile_shader(GL_FRAGMENT_SHADER, frag_src);
if (!vs || !fs) {
glDeleteShader(vs);
@@ -145,7 +249,8 @@ static GLuint link_program(const char *frag_src)
/* Public API */
/* ------------------------------------------------------------------ */
static void render(Xorg_Viewer *v); /* forward declaration */
static void render(Xorg_Viewer *v); /* forward declarations */
static void draw_text_overlays(Xorg_Viewer *v, int fb_w, int fb_h);
bool xorg_available(void) { return true; }
@@ -166,6 +271,58 @@ static void framebuffer_size_callback(GLFWwindow *window, int width, int height)
if (v) { render(v); }
}
/* Initialise text rendering resources — called from xorg_viewer_open. */
static bool init_text_rendering(Xorg_Viewer *v)
{
v->prog_rect = link_program(VERT_RECT_SRC, FRAG_RECT_SRC);
if (!v->prog_rect) { return false; }
v->u_rect_loc = glGetUniformLocation(v->prog_rect, "u_rect");
v->u_rect_fb_size_loc = glGetUniformLocation(v->prog_rect, "u_fb_size");
v->u_rect_color_loc = glGetUniformLocation(v->prog_rect, "u_color");
v->prog_text = link_program(VERT_TEXT_SRC, FRAG_TEXT_SRC);
if (!v->prog_text) { return false; }
v->u_fb_size_loc = glGetUniformLocation(v->prog_text, "u_fb_size");
v->u_text_atlas_loc = glGetUniformLocation(v->prog_text, "u_text_atlas");
v->u_text_color_loc = glGetUniformLocation(v->prog_text, "u_text_color");
/* Upload atlas texture (grayscale GL_R8). */
glGenTextures(1, &v->tex_atlas);
glBindTexture(GL_TEXTURE_2D, v->tex_atlas);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glTexImage2D(GL_TEXTURE_2D, 0, GL_R8,
FONT_ATLAS_W, FONT_ATLAS_H, 0,
GL_RED, GL_UNSIGNED_BYTE, font_atlas_pixels);
glBindTexture(GL_TEXTURE_2D, 0);
/* VAO + dynamic VBO for glyph quads. */
glGenVertexArrays(1, &v->vao_text);
glGenBuffers(1, &v->vbo_text);
glBindVertexArray(v->vao_text);
glBindBuffer(GL_ARRAY_BUFFER, v->vbo_text);
glBufferData(GL_ARRAY_BUFFER,
MAX_TEXT_VERTS * sizeof(float), NULL, GL_DYNAMIC_DRAW);
/* layout(location=0): vec2 a_pos, layout(location=1): vec2 a_uv */
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE,
4 * sizeof(float), (void *)0);
glEnableVertexAttribArray(1);
glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE,
4 * sizeof(float), (void *)(2 * sizeof(float)));
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindVertexArray(0);
return true;
}
Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height,
const char *title)
{
@@ -209,8 +366,8 @@ Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height,
glfwSetWindowUserPointer(win, v);
glfwSetFramebufferSizeCallback(win, framebuffer_size_callback);
v->prog_yuv = link_program(FRAG_YUV_SRC);
v->prog_rgb = link_program(FRAG_RGB_SRC);
v->prog_yuv = link_program(VERT_SRC, FRAG_YUV_SRC);
v->prog_rgb = link_program(VERT_SRC, FRAG_RGB_SRC);
if (!v->prog_yuv || !v->prog_rgb) {
xorg_viewer_close(v);
return NULL;
@@ -238,6 +395,11 @@ Xorg_Viewer *xorg_viewer_open(int x, int y, int width, int height,
}
glBindTexture(GL_TEXTURE_2D, 0);
if (!init_text_rendering(v)) {
xorg_viewer_close(v);
return NULL;
}
#ifdef HAVE_TURBOJPEG
v->tj = tjInitDecompress();
if (!v->tj) {
@@ -260,6 +422,144 @@ void xorg_viewer_set_anchor(Xorg_Viewer *v, Xorg_Anchor anchor)
if (v) { v->anchor = anchor; }
}
void xorg_viewer_set_overlay_text(Xorg_Viewer *v, int idx, int x, int y,
const char *text, float r, float g, float b)
{
if (!v || idx < 0 || idx >= MAX_OVERLAYS) { return; }
Overlay *o = &v->overlays[idx];
strncpy(o->text, text, MAX_OVERLAY_CHARS - 1);
o->text[MAX_OVERLAY_CHARS - 1] = '\0';
o->x = x;
o->y = y;
o->r = r;
o->g = g;
o->b = b;
if (idx >= v->n_overlays) { v->n_overlays = idx + 1; }
}
void xorg_viewer_clear_overlays(Xorg_Viewer *v)
{
if (v) { v->n_overlays = 0; }
}
/* ------------------------------------------------------------------ */
/* Internal: build and draw text quads for all overlays */
/* ------------------------------------------------------------------ */
static void draw_text_overlays(Xorg_Viewer *v, int fb_w, int fb_h)
{
if (v->n_overlays == 0 || !v->prog_text) { return; }
/*
* Vertex layout: (x, y, u, uv) — 4 floats per vertex, 6 verts per glyph.
* Declared static to keep it off the stack (2MB+ otherwise).
*/
static float verts[MAX_TEXT_VERTS];
/* Reset to full-window viewport for overlay drawing. */
glViewport(0, 0, fb_w, fb_h);
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, v->tex_atlas);
#define OVERLAY_MARGIN 4
for (int oi = 0; oi < v->n_overlays; oi++) {
const Overlay *o = &v->overlays[oi];
if (o->text[0] == '\0') { continue; }
/* Measure text to compute bounding box for the background rect. */
int text_w = 0, max_ascent = 0, max_descent = 0;
for (const char *p = o->text; *p; p++) {
unsigned char cp = (unsigned char)*p;
if (cp < 32) { continue; }
const Font_Glyph *g = &font_glyphs[cp];
text_w += g->advance;
if (g->bearing_y > max_ascent) { max_ascent = g->bearing_y; }
if (g->h > 0) {
int desc = g->h - g->bearing_y;
if (desc > max_descent) { max_descent = desc; }
}
}
int text_h = max_ascent + max_descent;
int max_bearing_y = max_ascent;
/* Draw semi-transparent dark background rect. */
float rx0 = (float)(o->x - OVERLAY_MARGIN);
float ry0 = (float)(o->y - OVERLAY_MARGIN);
float rx1 = (float)(o->x + text_w + OVERLAY_MARGIN);
float ry1 = (float)(o->y + text_h + OVERLAY_MARGIN);
glUseProgram(v->prog_rect);
glUniform2f(v->u_rect_fb_size_loc, (float)fb_w, (float)fb_h);
glUniform4f(v->u_rect_loc, rx0, ry0, rx1, ry1);
glUniform4f(v->u_rect_color_loc, 0.0f, 0.0f, 0.0f, 0.55f);
glBindVertexArray(v->vao);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
int baseline_y = o->y + max_bearing_y;
/* Switch back to text program + VBO for glyph quads. */
glUseProgram(v->prog_text);
glUniform2f(v->u_fb_size_loc, (float)fb_w, (float)fb_h);
glUniform1i(v->u_text_atlas_loc, 0);
glBindVertexArray(v->vao_text);
glBindBuffer(GL_ARRAY_BUFFER, v->vbo_text);
/* Build quads for this overlay. */
int n_verts = 0;
int cursor_x = o->x;
for (const char *p = o->text; *p; p++) {
unsigned char cp = (unsigned char)*p;
if (cp < 32) { continue; }
const Font_Glyph *g = &font_glyphs[cp];
/* Advance even for non-printing glyphs (e.g. space). */
if (g->w > 0 && g->h > 0) {
/* Screen-space quad corners (top-left origin). */
float sx0 = (float)(cursor_x + g->bearing_x);
float sy0 = (float)(baseline_y - g->bearing_y);
float sx1 = sx0 + (float)g->w;
float sy1 = sy0 + (float)g->h;
/* Atlas UV corners. */
float u0 = (float)g->x / FONT_ATLAS_W;
float u1 = (float)(g->x + g->w) / FONT_ATLAS_W;
float v0 = (float)g->y / FONT_ATLAS_H;
float v1 = (float)(g->y + g->h) / FONT_ATLAS_H;
if (n_verts + 6 * 4 > MAX_TEXT_VERTS) { break; }
float *d = verts + n_verts;
/* tri 0 */
d[ 0]=sx0; d[ 1]=sy0; d[ 2]=u0; d[ 3]=v0;
d[ 4]=sx0; d[ 5]=sy1; d[ 6]=u0; d[ 7]=v1;
d[ 8]=sx1; d[ 9]=sy0; d[10]=u1; d[11]=v0;
/* tri 1 */
d[12]=sx1; d[13]=sy0; d[14]=u1; d[15]=v0;
d[16]=sx0; d[17]=sy1; d[18]=u0; d[19]=v1;
d[20]=sx1; d[21]=sy1; d[22]=u1; d[23]=v1;
n_verts += 24;
}
cursor_x += g->advance;
}
if (n_verts == 0) { continue; }
glBufferSubData(GL_ARRAY_BUFFER, 0, n_verts * sizeof(float), verts);
glUniform3f(v->u_text_color_loc, o->r, o->g, o->b);
glDrawArrays(GL_TRIANGLES, 0, n_verts / 4);
}
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindVertexArray(0);
glDisable(GL_BLEND);
}
/* ------------------------------------------------------------------ */
/* Internal: compute layout and render from existing textures */
/* ------------------------------------------------------------------ */
@@ -383,6 +683,9 @@ static void render(Xorg_Viewer *v)
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glBindVertexArray(0);
/* Draw text overlays on top, then present. */
draw_text_overlays(v, fb_w, fb_h);
glfwSwapBuffers(v->window);
}
@@ -510,6 +813,13 @@ bool xorg_viewer_poll(Xorg_Viewer *v)
return true;
}
bool xorg_viewer_handle_events(Xorg_Viewer *v)
{
if (!v || glfwWindowShouldClose(v->window)) { return false; }
glfwPollEvents();
return !glfwWindowShouldClose(v->window);
}
void xorg_viewer_close(Xorg_Viewer *v)
{
if (!v) { return; }
@@ -517,6 +827,11 @@ void xorg_viewer_close(Xorg_Viewer *v)
if (v->tj) { tjDestroy(v->tj); }
free(v->yuv_buf);
#endif
if (v->vao_text) { glDeleteVertexArrays(1, &v->vao_text); }
if (v->vbo_text) { glDeleteBuffers(1, &v->vbo_text); }
if (v->tex_atlas) { glDeleteTextures(1, &v->tex_atlas); }
if (v->prog_text) { glDeleteProgram(v->prog_text); }
if (v->prog_rect) { glDeleteProgram(v->prog_rect); }
if (v->vao) { glDeleteVertexArrays(1, &v->vao); }
if (v->tex[0]) { glDeleteTextures(4, v->tex); }
if (v->prog_yuv) { glDeleteProgram(v->prog_yuv); }

7
src/modules/xorg/xorg_stub.c
View File

@@ -34,5 +34,12 @@ bool xorg_viewer_push_mjpeg(Xorg_Viewer *v,
void xorg_viewer_set_scale(Xorg_Viewer *v, Xorg_Scale scale) { (void)v; (void)scale; }
void xorg_viewer_set_anchor(Xorg_Viewer *v, Xorg_Anchor anchor) { (void)v; (void)anchor; }
void xorg_viewer_set_overlay_text(Xorg_Viewer *v, int idx, int x, int y,
const char *text, float r, float g, float b)
{
(void)v; (void)idx; (void)x; (void)y; (void)text; (void)r; (void)g; (void)b;
}
void xorg_viewer_clear_overlays(Xorg_Viewer *v) { (void)v; }
bool xorg_viewer_poll(Xorg_Viewer *v) { (void)v; return false; }
bool xorg_viewer_handle_events(Xorg_Viewer *v) { (void)v; return false; }
void xorg_viewer_close(Xorg_Viewer *v) { (void)v; }

229
tools/gen_font_atlas/gen_font_atlas.py Normal file
View File

@@ -0,0 +1,229 @@
#!/usr/bin/env python3
"""
Generate a bitmap font atlas from a TrueType font for use in the xorg viewer.
Output:
--out-header C header with pixel data (uint8_t array) and glyph metrics
--out-png Optional PNG for visual inspection
The atlas texture is grayscale (1 byte/pixel, GL_R8). Each glyph's alpha
channel is stored directly — the renderer blends using this value.
Usage:
gen_font_atlas.py [--font PATH] [--size PT]
[--out-header PATH] [--out-png PATH]
"""
import sys
import os
import argparse
from PIL import Image, ImageFont, ImageDraw
GLYPH_FIRST = 32
GLYPH_LAST = 255
FONT_SEARCH_PATHS = [
'/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf',
'/usr/share/fonts/dejavu/DejaVuSans.ttf',
'/usr/share/fonts/TTF/DejaVuSans.ttf',
'/usr/share/fonts/dejavu-sans-fonts/DejaVuSans.ttf',
'/usr/local/share/fonts/truetype/dejavu/DejaVuSans.ttf',
]
def find_font():
for p in FONT_SEARCH_PATHS:
if os.path.exists(p):
return p
return None
# ---------------------------------------------------------------------------
# Simple per-column skyline bin packer
# ---------------------------------------------------------------------------
class Skyline:
"""
Skyline bin packer using a per-column height array.
For each candidate x position, the placement y is max(sky[x:x+w]).
O(atlas_width * n_glyphs) — fast enough for small atlases.
"""
def __init__(self, width, height):
self.w = width
self.h = height
self.sky = [0] * width
def place(self, gw, gh):
"""Find best-fit position for a glyph of size (gw, gh). Returns (x, y) or None."""
best_y = None
best_x = None
for x in range(self.w - gw + 1):
y = max(self.sky[x:x + gw])
if y + gh > self.h:
continue
if best_y is None or y < best_y:
best_y = y
best_x = x
if best_x is None:
return None
for i in range(best_x, best_x + gw):
self.sky[i] = best_y + gh
return (best_x, best_y)
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument('--font', default=None,
help='Path to TrueType font file (default: search for DejaVu Sans)')
ap.add_argument('--size', type=int, default=16,
help='Font size in points (default: 16)')
ap.add_argument('--out-header', default='font_atlas.h',
help='Output C header path (default: font_atlas.h)')
ap.add_argument('--out-png', default=None,
help='Optional output PNG for visual inspection')
args = ap.parse_args()
font_path = args.font or find_font()
if not font_path:
sys.exit('error: DejaVu Sans not found; pass --font PATH')
print(f'font: {font_path}', file=sys.stderr)
print(f'size: {args.size}pt', file=sys.stderr)
font = ImageFont.truetype(font_path, args.size)
# ------------------------------------------------------------------
# Collect glyph metrics and bitmaps
# ------------------------------------------------------------------
glyphs = {}
for cp in range(GLYPH_FIRST, GLYPH_LAST + 1):
ch = chr(cp)
# getbbox returns (left, top, right, bottom) in local glyph coords.
# top is negative for glyphs that extend above the origin.
bbox = font.getbbox(ch)
adv = int(font.getlength(ch) + 0.5)
if bbox is None:
glyphs[cp] = {'bitmap': None, 'w': 0, 'h': 0,
'advance': adv, 'bearing_x': 0, 'bearing_y': 0}
continue
l, t, r, b = bbox
gw = r - l
gh = b - t
if gw <= 0 or gh <= 0:
# Non-printing (e.g. space) — store advance only
glyphs[cp] = {'bitmap': None, 'w': 0, 'h': 0,
'advance': adv, 'bearing_x': int(l), 'bearing_y': int(-t)}
continue
img = Image.new('L', (gw, gh), 0)
draw = ImageDraw.Draw(img)
draw.text((-l, -t), ch, font=font, fill=255)
glyphs[cp] = {
'bitmap': img, 'w': gw, 'h': gh,
'advance': adv,
'bearing_x': int(l), # horizontal distance from pen to left edge
'bearing_y': int(-t), # ascent: pixels above baseline (positive)
}
renderable = [cp for cp, g in glyphs.items() if g['bitmap'] is not None]
print(f'glyphs: {len(renderable)} renderable, '
f'{len(glyphs) - len(renderable)} non-printing', file=sys.stderr)
total_area = sum(glyphs[cp]['w'] * glyphs[cp]['h'] for cp in renderable)
# ------------------------------------------------------------------
# Pack into atlas — try increasing sizes until everything fits
# ------------------------------------------------------------------
# Sort by descending height for better skyline packing
sorted_cps = sorted(renderable,
key=lambda cp: (-glyphs[cp]['h'], -glyphs[cp]['w']))
PAD = 1 # 1-pixel gap between glyphs to avoid bilinear bleed
placements = {}
atlas_w = atlas_h = 0
for aw, ah in [(256, 256), (512, 256), (512, 512), (1024, 512)]:
packer = Skyline(aw, ah)
places = {}
failed = False
for cp in sorted_cps:
g = glyphs[cp]
pos = packer.place(g['w'] + PAD, g['h'] + PAD)
if pos is None:
failed = True
break
places[cp] = pos
if not failed:
placements = places
atlas_w, atlas_h = aw, ah
util = 100 * total_area // (aw * ah)
print(f'atlas: {aw}x{ah} ({util}% utilisation)', file=sys.stderr)
break
else:
sys.exit('error: could not fit all glyphs into any supported atlas size')
# ------------------------------------------------------------------
# Render atlas image
# ------------------------------------------------------------------
atlas = Image.new('L', (atlas_w, atlas_h), 0)
for cp, (px, py) in placements.items():
atlas.paste(glyphs[cp]['bitmap'], (px, py))
if args.out_png:
os.makedirs(os.path.dirname(os.path.abspath(args.out_png)), exist_ok=True)
atlas.save(args.out_png)
print(f'png: {args.out_png}', file=sys.stderr)
# ------------------------------------------------------------------
# Write C header
# ------------------------------------------------------------------
os.makedirs(os.path.dirname(os.path.abspath(args.out_header)), exist_ok=True)
pixels = list(atlas.tobytes())
with open(args.out_header, 'w') as f:
f.write('/* Auto-generated by gen_font_atlas.py — do not edit. */\n')
f.write('#pragma once\n\n')
f.write('#include <stdint.h>\n\n')
f.write(f'#define FONT_ATLAS_W {atlas_w}\n')
f.write(f'#define FONT_ATLAS_H {atlas_h}\n\n')
f.write('typedef struct {\n')
f.write('\tint x, y; /* top-left in atlas */\n')
f.write('\tint w, h; /* bitmap size in pixels */\n')
f.write('\tint advance; /* horizontal pen advance */\n')
f.write('\tint bearing_x; /* left-side bearing */\n')
f.write('\tint bearing_y; /* ascent above baseline */\n')
f.write('} Font_Glyph;\n\n')
# Glyph table — indexed directly by codepoint 0..255
f.write('static const Font_Glyph font_glyphs[256] = {\n')
for cp in range(256):
if cp in glyphs:
g = glyphs[cp]
px, py = placements.get(cp, (0, 0))
f.write(f'\t[{cp}] = {{ {px}, {py}, {g["w"]}, {g["h"]}, '
f'{g["advance"]}, {g["bearing_x"]}, {g["bearing_y"]} }},\n')
else:
f.write(f'\t[{cp}] = {{ 0, 0, 0, 0, 0, 0, 0 }},\n')
f.write('};\n\n')
# Pixel data — grayscale, row-major
f.write(f'static const uint8_t font_atlas_pixels[{atlas_w * atlas_h}] = {{\n')
for i in range(0, len(pixels), 16):
chunk = pixels[i:i + 16]
f.write('\t' + ', '.join(str(b) for b in chunk) + ',\n')
f.write('};\n')
print(f'header: {args.out_header}', file=sys.stderr)
if __name__ == '__main__':
main()