Build step 1: replay, render, and UI on synthetic data

GLM Agent committed on 2026-10-25 at 07:14

- rig: shared camera and splat types (COLMAP convention, Y-up world),
  footprint-sized surfel construction, virtual-camera blending,
  FrameSource trait
- replay: synthetic head-and-shoulders scene with exact geometry;
  a software depth camera raycasts per pixel and builds surfel splat
  batches through the same construction the real depth path will use
- render: Brush integration with packed RGBA8 readback; rig-to-Brush
  camera and splat conversions; runs headless on any wgpu adapter,
  including lavapipe in CI
- sink: egui viewer with steering slider, play/pause, and live stats,
  as a headless-testable model plus a thin eframe shell
- tests: unit tests per crate, GPU integration tests with image-content
  assertions, and kittest UI tests driving the real widgets — all
  headless (39 tests)
- burn pinned in Cargo.lock to Brush's tested revision via
  scripts/sync-tracel-lock.awk; burn's main drifts ahead of Brush and
  breaks its build

Browse files at this commit

Cargo.lock +8589 -8

Large diff (8597 lines changed) - not displayed

Cargo.toml +32 -3

diff --git a/Cargo.toml b/Cargo.toml
index 369a752..b3c1e93 100644
@@ -1,6 +1,7 @@
[workspace]
resolver = "3"
members = [
    "rig",
    "capture",
    "replay",
    "splatgen",
@@ -8,7 +9,9 @@ members = [
    "sink",
]
# The workspace root is the `kinect-view/` tree from PLAN.md.
# The workspace root is the `kinect-view/` tree from PLAN.md. `rig` holds the
# shared types (poses, intrinsics, splat batches, frame sources) so the other
# crates never need to depend on each other's heavy dependencies.
[workspace.package]
version = "0.1.0"
@@ -16,6 +19,7 @@ edition = "2024"
[workspace.dependencies]
# Internal crates.
rig = { path = "rig" }
capture = { path = "capture" }
replay = { path = "replay" }
splatgen = { path = "splatgen" }
@@ -33,12 +37,36 @@ kinect-one = { git = "https://github.com/Aytixel/kinect-one" }
nokhwa = "0.10"
# Rendering: Brush's renderer as a library (not published to crates.io; it
# lives under crates/brush-render in the Brush repo).
brush-render = { git = "https://github.com/ArthurBrussee/brush" }
# lives under crates/brush-render in the Brush repo). The burn/burn-wgpu
# pins mirror Brush's own workspace deps exactly — same git source and
# revision (the one in Brush's committed Cargo.lock) — so cargo unifies
# them into a single burn crate. A different source or revision would
# compile a second burn and break type interop, exactly like glam.
# Pin the revision whenever these move: burn's `main` drifts ahead of
# Brush between releases (a fusion API change broke the build once
# already).
brush-render = { git = "https://github.com/ArthurBrussee/brush", rev = "1388f74c6fe0236f68ee4915564bf00e9d2e3747" }
# NOTE: burn is declared with `branch = "main"` — the same requirement as
# Brush's workspace — so cargo unifies it with Brush's burn into one
# crate. The *revision actually built* is then pinned through the
# committed Cargo.lock (see the lock-sync note in PLAN.md): burn's `main`
# regularly drifts ahead of Brush and breaks its build, so the lock pins
# the `branch = "main"` resolution to the revision Brush's own committed
# lockfile uses. Never run `cargo update -p burn ...` blindly.
burn = { git = "https://github.com/tracel-ai/burn", branch = "main", default-features = false, features = [
    "std",
    "wgpu",
    "autodiff",
] }
burn-wgpu = { git = "https://github.com/tracel-ai/burn", branch = "main", features = [
    "exclusive-memory-only",
] }
# UI / sinks.
eframe = "0.37"
egui = "0.37"
# Headless egui test harness (wgpu snapshots + interaction).
egui_kittest = { version = "0.37", features = ["wgpu"] }
pollster = "1"
# v4l2loopback output (Linux only). Fallback candidate if this proves
# awkward for output queues: edgefirst-v4l2.
@@ -54,6 +82,7 @@ thiserror = "2"
tracing = "0.1"
tracing-subscriber = "0.3"
log = "0.4"
env_logger = "0.11"
# Async plumbing (kinect-one's API is tokio-based).
tokio = { version = "1", features = ["macros", "rt", "rt-multi-thread", "sync", "time"] }

PLAN.md +29 -1

diff --git a/PLAN.md b/PLAN.md
index 7e6349b..81bc40d 100644
@@ -64,6 +64,7 @@ The workspace root of this repository is the `kinect-view/` tree:
```
kinect-view/
├── rig/        # shared types: poses, intrinsics, splat batches, frame sources
├── capture/    # kinect-one, webcams, frame sync
├── replay/     # recorded/synthetic datasets — dev & CI without hardware
├── splatgen/   # depth filter, normals, surfels, ellipsoid proxy fill
@@ -71,9 +72,30 @@ kinect-view/
└── sink/       # egui UI + v4l2loopback (Linux)
```
(`rig/` was added during build-order step 1: the shared types need a home
that none of the other crates depend on awkwardly.)
## Testing
Test everything as it is built, headlessly — the container has no display
and no Kinect:
* **Unit tests** live with the code they test (camera math, surfel
  construction, scene intersection, conversions).
* **GPU integration tests** render real frames through Brush and assert
  on image *content* (where the subject lands, what colour it is),
  tolerant enough for any conforming driver. Any wgpu-capable adapter
  works, including software Vulkan (lavapipe) in containers and CI.
* **UI tests** drive the real egui code path with `egui_kittest`:
  widget interactions (play/pause, the steering slider) and a full
  offscreen paint pass.
* The **Kinect v2 integration cannot be tested in the container**; replay
  sources stand in for it until hardware testing happens on a real
  machine.
## Build order
- [ ] 1. replay + render + UI with synthetic data
- [x] 1. replay + render + UI with synthetic data
- [ ] 2. splatgen on recorded Kinect frames
- [ ] 3. Live Kinect capture + ellipsoid proxy recoloring from side cameras
- [ ] 4. v4l2loopback sink
@@ -87,6 +109,12 @@ kinect-view/
3. **Cross-camera color seams** → calibration vs Kinect.
4. **Frame sync across three async sensors** → timestamp-align, tolerate
   1-frame skew.
5. **Brush/burn moving targets** → brush-render is pinned to a verified
   revision, and the lockfile pins burn's `branch = "main"` resolution to
   the revision from Brush's own committed lock (see
   `scripts/sync-tracel-lock.awk`). burn's main drifting ahead of Brush
   broke the build once during step 1; re-run that script whenever the
   pins move, and never `cargo update` git-sourced packages blindly.
## Out of scope

README.md +14 -0

diff --git a/README.md b/README.md
index 3b7a5f8..1b445b9 100644
@@ -23,10 +23,24 @@ The rendered output is delivered as:
Targets Ubuntu 24.04 (x86-64) and Windows on ARM64.
## Status
Build-order step 1 is complete: the full pipeline runs on synthetic data.
A synthetic head-and-shoulders stand-in is raycast per depth pixel and
turned into footprint-sized surfel splats — the same construction the
real depth path will use — then rendered live by Brush's WebGPU
renderer from a virtual camera steered between the rig's side cameras,
and shown in the egui viewer. The whole chain, GPU rendering included,
runs headlessly in tests (software Vulkan).
- `cargo run -p sink` — the viewer (needs a GPU, or lavapipe/Vulkan)
- `cargo test --workspace` — unit, GPU-integration, and headless UI tests
## Workspace layout
| Crate      | Role |
|------------|------|
| `rig`      | Shared types: camera poses and intrinsics, the virtual camera, splat batches, frame sources |
| `capture`  | Kinect v2 RGB + depth (`kinect-one`) and two webcams (`nokhwa`), timestamp frame sync |
| `replay`   | Recorded / synthetic rig datasets, so development and CI run without hardware |
| `splatgen` | Depth filtering, normals, footprint-sized surfel splats, ellipsoid-proxy fill |

render/Cargo.toml +11 -0

diff --git a/render/Cargo.toml b/render/Cargo.toml
index 3494be6..bcde597 100644
@@ -6,3 +6,14 @@ edition.workspace = true
publish = false
[dependencies]
anyhow.workspace = true
brush-render.workspace = true
burn.workspace = true
burn-wgpu.workspace = true
glam.workspace = true
pollster.workspace = true
rig.workspace = true
[dev-dependencies]
replay.workspace = true
tokio = { workspace = true }

render/src/lib.rs +323 -4

diff --git a/render/src/lib.rs b/render/src/lib.rs
index ea3e521..edfc7cc 100644
@@ -1,7 +1,326 @@
//! Brush renderer integration and the virtual camera.
//!
//! Pipeline role (PLAN.md, "Pipeline per frame" — render): take the
//! freshly constructed Gaussian list from `splatgen` each frame and hand
//! it to Brush's renderer (WebGPU via Burn), then render it from the
//! virtual camera pose supplied by the UI. Fresh splat list per frame,
//! 30–60 fps target, front-to-back alpha blending.
//! freshly constructed [`SplatBatch`] each frame, hand it to Brush's
//! renderer (WebGPU via Burn) as a brand-new Gaussian list — no training,
//! no state carried between frames — and render it from the virtual
//! camera pose supplied by the UI.
//!
//! [`SplatRenderer`] owns the Burn device; [`SplatBatch`] and [`rig::Camera`]
//! types are converted to Brush's representation at the boundary:
//!
//! * scales → log scales, opacity → logit, colour → SH degree 0
//!   (Brush's conventions, see `brush_render::sh::rgb_to_sh`),
//! * intrinsics → Brush's fov + normalized center.
//!
//! The output is Brush's packed RGBA8 frame: premultiplied over
//! `background`, `[H, W, 4]` bytes, row-major.
use anyhow::Result;
use burn::tensor::Device;
use burn_wgpu::WgpuDevice;
use glam::{UVec2, Vec2, Vec3};
use brush_render::camera::Camera as BrushCamera;
use brush_render::gaussian_splats::{SplatRenderMode, Splats, TextureMode, inverse_sigmoid};
use brush_render::kernels::camera_model::CameraModel;
use brush_render::render_splats;
use brush_render::sh::rgb_to_sh;
use rig::{Camera, SplatBatch};
/// A rendered virtual-camera frame: Brush's packed output, premultiplied
/// over the renderer's background.
#[derive(Debug, Clone, PartialEq)]
pub struct RenderedImage {
    pub width: u32,
    pub height: u32,
    /// `[height][width][4]` RGBA8, premultiplied over the background.
    pub rgba: Vec<u8>,
}
impl RenderedImage {
    /// The pixel at `(x, y)`: `(r, g, b, a)`.
    pub fn pixel(&self, x: u32, y: u32) -> [u8; 4] {
        let idx = ((y * self.width + x) * 4) as usize;
        [
            self.rgba[idx],
            self.rgba[idx + 1],
            self.rgba[idx + 2],
            self.rgba[idx + 3],
        ]
    }
    /// A solid image of the background colour (alpha 0, colour
    /// premultiplied): what Brush produces for an empty Gaussian list.
    pub fn background(width: u32, height: u32, background: Vec3) -> Self {
        let px = [
            (background.x * 255.0) as u8,
            (background.y * 255.0) as u8,
            (background.z * 255.0) as u8,
            0,
        ];
        let rgba = px.repeat((width * height) as usize);
        Self {
            width,
            height,
            rgba,
        }
    }
}
/// Counting stats from a render pass, for the UI and tests.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct RenderStats {
    /// Splats in the batch.
    pub total_splats: usize,
    /// Splats that survived culling for this view.
    pub visible_splats: u32,
}
/// Renders splat batches from arbitrary pinhole cameras using Brush.
pub struct SplatRenderer {
    device: Device,
    background: Vec3,
}
impl Default for SplatRenderer {
    fn default() -> Self {
        Self::new(Vec3::new(0.10, 0.10, 0.12))
    }
}
impl SplatRenderer {
    /// Create a renderer on the default Burn WGPU device.
    ///
    /// The device initializes lazily on first render; it needs a
    /// wgpu-capable GPU adapter (Vulkan; software Vulkan such as lavapipe
    /// works for tests and CI).
    pub fn new(background: Vec3) -> Self {
        Self {
            device: Device::from(WgpuDevice::default()),
            background,
        }
    }
    /// The background colour splats are composited over.
    pub fn background(&self) -> Vec3 {
        self.background
    }
    /// Render `batch` from `camera` (blocking on the GPU).
    pub fn render(
        &self,
        batch: &SplatBatch,
        camera: &Camera,
    ) -> Result<(RenderedImage, RenderStats)> {
        pollster::block_on(self.render_async(batch, camera))
    }
    /// Render `batch` from `camera`, async.
    pub async fn render_async(
        &self,
        batch: &SplatBatch,
        camera: &Camera,
    ) -> Result<(RenderedImage, RenderStats)> {
        let resolution = camera.resolution;
        if batch.is_empty() {
            return Ok((
                RenderedImage::background(resolution.x, resolution.y, self.background),
                RenderStats {
                    total_splats: 0,
                    visible_splats: 0,
                },
            ));
        }
        let splats = splats_for_brush(batch, &self.device);
        let brush_camera = brush_camera(camera);
        let (image, aux) = render_splats(
            splats,
            &brush_camera,
            UVec2::new(resolution.x, resolution.y),
            self.background,
            None,
            TextureMode::Packed,
        )
        .await;
        let stats = RenderStats {
            total_splats: batch.len(),
            visible_splats: aux.num_visible,
        };
        let data = image
            .into_data_async()
            .await
            .map_err(|e| anyhow::anyhow!("frame readback failed: {e}"))?;
        let rgba = data.into_bytes().to_vec();
        Ok((
            RenderedImage {
                width: resolution.x,
                height: resolution.y,
                rgba,
            },
            stats,
        ))
    }
}
/// Convert a rig camera to Brush's representation: fov from focal length,
/// normalized principal point, ideal pinhole.
pub fn brush_camera(camera: &Camera) -> BrushCamera {
    let res = camera.resolution;
    BrushCamera::new(
        camera.pose.position,
        camera.pose.rotation,
        camera.fov_x() as f64,
        camera.fov_y() as f64,
        Vec2::new(
            camera.intrinsics.cx / res.x as f32,
            camera.intrinsics.cy / res.y as f32,
        ),
        CameraModel::Pinhole,
    )
}
/// Flatten a batch into Brush's raw arrays (pure CPU math, no device):
/// positions, xyzw quaternions, log scales, SH degree-0 coefficients,
/// logit opacities.
pub fn brush_raw_arrays(batch: &SplatBatch) -> BrushRawArrays {
    let n = batch.len();
    let mut positions = Vec::with_capacity(n * 3);
    let mut rotations = Vec::with_capacity(n * 4);
    let mut log_scales = Vec::with_capacity(n * 3);
    let mut sh_coeffs = Vec::with_capacity(n * 3);
    let mut raw_opacities = Vec::with_capacity(n);
    for splat in batch.iter() {
        positions.extend_from_slice(&splat.mean.to_array());
        rotations.extend_from_slice(&splat.rotation.to_array());
        log_scales.extend_from_slice(&splat.scale.to_array().map(f32::ln));
        let sh = rgb_to_sh(Vec3::from(splat.color));
        sh_coeffs.extend_from_slice(&sh.to_array());
        raw_opacities.push(inverse_sigmoid(splat.opacity));
    }
    BrushRawArrays {
        positions,
        rotations,
        log_scales,
        sh_coeffs,
        raw_opacities,
    }
}
/// The flattened per-splat arrays [`brush_raw_arrays`] produces.
#[derive(Debug, Clone, PartialEq)]
pub struct BrushRawArrays {
    pub positions: Vec<f32>,
    pub rotations: Vec<f32>,
    pub log_scales: Vec<f32>,
    pub sh_coeffs: Vec<f32>,
    pub raw_opacities: Vec<f32>,
}
/// Upload a batch to the device as Brush `Splats` (forward rendering,
/// SH degree 0, default render mode).
pub fn splats_for_brush(batch: &SplatBatch, device: &Device) -> Splats {
    let arrays = brush_raw_arrays(batch);
    Splats::from_raw(
        arrays.positions,
        arrays.rotations,
        arrays.log_scales,
        arrays.sh_coeffs,
        arrays.raw_opacities,
        SplatRenderMode::Default,
        device,
    )
}
#[cfg(test)]
mod tests {
    use super::*;
    use glam::{uvec2, vec3};
    use rig::{CameraIntrinsics, CameraPose, Splat};
    fn camera() -> Camera {
        Camera {
            pose: CameraPose::looking_at(Vec3::ZERO, vec3(0.0, 0.0, 2.0)),
            intrinsics: CameraIntrinsics {
                fx: 500.0,
                fy: 480.0,
                cx: 320.0,
                cy: 240.0,
            },
            resolution: uvec2(640, 480),
        }
    }
    #[test]
    fn brush_camera_conversion() {
        let cam = camera();
        let bc = brush_camera(&cam);
        assert_eq!(bc.position, Vec3::ZERO);
        // fov matches Brush's focal formula (f32 round-trips, so a
        // milli-pixel tolerance is plenty tight).
        let focal_x = (640.0 / 2.0) / (bc.fov_x / 2.0).tan();
        assert!((focal_x - 500.0).abs() < 1e-3);
        let focal_y = (480.0 / 2.0) / (bc.fov_y / 2.0).tan();
        assert!((focal_y - 480.0).abs() < 1e-3);
        // Normalized principal point.
        assert!((bc.center_uv.x - 0.5).abs() < 1e-6);
        assert!((bc.center_uv.y - 0.5).abs() < 1e-6);
        // Rotation preserved (camera-to-world).
        let fwd = cam.pose.rotation * Vec3::Z;
        let bfwd = bc.rotation * Vec3::Z;
        assert!((fwd - bfwd).length() < 1e-6);
        assert!(bc.is_valid());
    }
    #[test]
    fn raw_arrays_follow_brush_conventions() {
        let mut batch = SplatBatch::new();
        batch.push(Splat {
            mean: vec3(0.1, 0.2, 2.0),
            rotation: glam::Quat::from_rotation_y(0.5),
            scale: vec3(0.004, 0.005, 0.003),
            color: [0.85, 0.62, 0.52],
            opacity: 0.95,
        });
        let a = brush_raw_arrays(&batch);
        // Position.
        assert_eq!(a.positions, vec![0.1, 0.2, 2.0]);
        // Quaternion is glam's (x, y, z, w) layout.
        assert_eq!(a.rotations, glam::Quat::from_rotation_y(0.5).to_array());
        // Log scales.
        for (got, want) in a.log_scales.iter().zip([0.004_f32, 0.005, 0.003]) {
            assert!((got - want.ln()).abs() < 1e-6);
        }
        // Logit opacity: sigmoid(raw) == 0.95.
        let raw = a.raw_opacities[0];
        let sigmoid = 1.0 / (1.0 + (-raw).exp());
        assert!((sigmoid - 0.95).abs() < 1e-6);
        // SH degree 0: colour = 0.5 + SH_C0 * coeff must reconstruct the
        // splat colour (the constant matches Brush's, cast like Brush
        // does).
        let sh_c0 = 0.28209479177387814_f64 as f32;
        for (coeff, want) in a.sh_coeffs.iter().zip([0.85_f32, 0.62, 0.52]) {
            let color = 0.5 + sh_c0 * coeff;
            assert!((color - want).abs() < 1e-5);
        }
    }
    #[test]
    fn background_image_is_transparent() {
        let img = RenderedImage::background(4, 3, Vec3::new(1.0, 0.0, 0.0));
        assert_eq!((img.width, img.height), (4, 3));
        assert_eq!(img.rgba.len(), 4 * 3 * 4);
        // Premultiplied background: colour at full value, alpha 0.
        assert_eq!(img.pixel(0, 0), [255, 0, 0, 0]);
        assert_eq!(img.pixel(3, 2), [255, 0, 0, 0]);
    }
}

render/tests/render.rs +225 -0

diff --git a/render/tests/render.rs b/render/tests/render.rs
new file mode 100644
index 0000000..f038298
@@ -0,0 +1,225 @@
//! GPU integration tests for the Brush render path.
//!
//! These run the real renderer headlessly: Burn's WGPU device picks the
//! best adapter available, which includes software Vulkan (lavapipe) in
//! containers and CI. Frames come from the synthetic source, and the
//! assertions check image *content* — where the subject lands, what
//! colour it is, what the backdrop looks like — rather than exact
//! pixels, so they hold on any conforming driver.
use std::sync::LazyLock;
use glam::{Vec2, uvec2, vec3};
use render::{RenderStats, SplatRenderer};
use replay::synth::{self, SUBJECT};
use rig::{Camera, FrameSource, SplatBatch};
/// Small enough to stay fast on software Vulkan, big enough that the
/// subject spans several pixels.
const CENTER: (u32, u32) = (64, 53);
const SIDE: (u32, u32) = (80, 60);
fn renderer() -> &'static SplatRenderer {
    static RENDERER: LazyLock<SplatRenderer> = LazyLock::new(SplatRenderer::default);
    &RENDERER
}
/// The synthetic subject's head centre (the scene is deterministic at
/// frame 0, where the sway offset is zero).
fn head_center() -> glam::Vec3 {
    glam::vec3(0.0, 0.18, 2.0)
}
/// Synthetic skin colour, RGB in [0, 1].
const SKIN: [f32; 3] = [0.85, 0.62, 0.52];
fn first_frame() -> SplatBatch {
    let rig = synth::dev_rig_with_resolutions(uvec2(CENTER.0, CENTER.1), uvec2(SIDE.0, SIDE.1));
    let mut source = synth::dev_source_with_center_camera(rig.center, 30.0);
    source.next_frame().expect("synthetic frame").splats
}
/// The mean of pixels within `tolerance` of a colour, in image
/// coordinates; `None` if too few match.
fn color_centroid(image: &render::RenderedImage, color: [f32; 3], tolerance: f32) -> Option<Vec2> {
    let mut sum = Vec2::ZERO;
    let mut count = 0.0;
    for y in 0..image.height {
        for x in 0..image.width {
            let [r, g, b, _a] = image.pixel(x, y);
            let px = [r as f32 / 255.0, g as f32 / 255.0, b as f32 / 255.0];
            let dist =
                (px[0] - color[0]).abs() + (px[1] - color[1]).abs() + (px[2] - color[2]).abs();
            if dist < tolerance {
                sum += Vec2::new(x as f32, y as f32);
                count += 1.0;
            }
        }
    }
    (count > 5.0).then(|| sum / count)
}
/// A checkerboard cell colour, wherever the given pixel landed.
#[allow(dead_code)]
fn is_checker_grey(px: [u8; 4]) -> bool {
    let [r, g, b, a] = px;
    a > 0 && r == g && g == b && ((r as i32 - 184).abs() < 24 || (r as i32 - 71).abs() < 24)
}
#[tokio::test]
async fn renders_subject_and_backdrop_from_the_center() {
    let rig = synth::dev_rig_with_resolutions(uvec2(CENTER.0, CENTER.1), uvec2(SIDE.0, SIDE.1));
    let batch = first_frame();
    assert!(!batch.is_empty());
    // The virtual camera at t = 0.5 sits at the rig's center camera.
    let camera = rig.virtual_camera(0.5, rig.left.intrinsics, rig.left.resolution);
    let (image, stats) = renderer()
        .render_async(&batch, &camera)
        .await
        .expect("render");
    assert_eq!((image.width, image.height), SIDE);
    assert_eq!(image.rgba.len(), (SIDE.0 * SIDE.1 * 4) as usize);
    assert!(stats.visible_splats > 0);
    assert!(stats.visible_splats as usize <= stats.total_splats);
    // The subject's head lands where the camera math says it should.
    let projected = camera.project(head_center()).expect("in front of camera");
    let rendered = color_centroid(&image, SKIN, 0.2).expect("skin visible");
    assert!(
        (projected - rendered).length() < 3.0,
        "head projected to {projected:?} but rendered at {rendered:?}"
    );
    // The image corners show the checkerboard backdrop, not the subject:
    // grey, opaque, and neither skin nor the renderer background. (Exact
    // cell colours are not asserted — a corner can legitimately blend a
    // checker boundary.)
    let corners = [
        image.pixel(2, 2),
        image.pixel(image.width - 3, 2),
        image.pixel(2, image.height - 3),
        image.pixel(image.width - 3, image.height - 3),
    ];
    for corner in corners {
        let [r, g, b, a] = corner;
        assert!(a >= 250, "backdrop is opaque, got {corner:?}");
        assert!(
            (r as i32 - g as i32).abs() <= 1 && (g as i32 - b as i32).abs() <= 1,
            "backdrop is grey, got {corner:?}"
        );
        assert!(
            (40..215).contains(&r),
            "backdrop is a checker grey, got {corner:?}"
        );
    }
    // The head is opaque: stacked ~0.95-opacity surfels saturate alpha
    // (within the rasterizer's 1e-4 transmittance floor).
    let head_px = image.pixel(projected.x as u32, projected.y as u32);
    assert!(head_px[3] >= 250, "head pixel must be opaque: {head_px:?}");
}
#[tokio::test]
async fn steering_shifts_the_backdrop_but_not_the_subject() {
    // The rig converges on the subject: at the convergence distance the
    // parallax is zero, so the subject stays framed as the virtual camera
    // moves between the side cameras — the point of the whole design —
    // while everything off the convergence plane (the backdrop) shifts.
    let rig = synth::dev_rig_with_resolutions(uvec2(CENTER.0, CENTER.1), uvec2(SIDE.0, SIDE.1));
    let batch = first_frame();
    let render_at = |t: f32| {
        let camera = rig.virtual_camera(t, rig.left.intrinsics, rig.left.resolution);
        let batch = &batch;
        async move { renderer().render_async(batch, &camera).await }
    };
    let (left, _) = render_at(0.0).await.expect("render left");
    let (right, _) = render_at(1.0).await.expect("render right");
    // The rendered head aligns with the camera math in both views…
    let left_head = color_centroid(&left, SKIN, 0.2).expect("skin from left");
    let right_head = color_centroid(&right, SKIN, 0.2).expect("skin from right");
    let cam_left = rig.virtual_camera(0.0, rig.left.intrinsics, rig.left.resolution);
    let cam_right = rig.virtual_camera(1.0, rig.left.intrinsics, rig.left.resolution);
    let proj_left = cam_left.project(head_center()).unwrap();
    let proj_right = cam_right.project(head_center()).unwrap();
    assert!((proj_left - left_head).length() < 3.0);
    assert!((proj_right - right_head).length() < 3.0);
    // …and the convergence geometry keeps the subject stable across the
    // full steering range…
    assert!(
        (left_head - right_head).length() < 3.0,
        "subject must stay framed: {left_head:?} vs {right_head:?}"
    );
    // …while the backdrop parallaxes: a good fraction of the frame must
    // change between the two ends of the baseline.
    let mut changed = 0;
    let mut total = 0;
    let (left_px, _) = left.rgba.as_chunks::<4>();
    let (right_px, _) = right.rgba.as_chunks::<4>();
    for (a, b) in left_px.iter().zip(right_px) {
        total += 1;
        let diff = (a[0] as i32 - b[0] as i32)
            .abs()
            .max((a[1] as i32 - b[1] as i32).abs());
        if diff > 25 {
            changed += 1;
        }
    }
    assert!(
        changed * 10 > total,
        "only {changed}/{total} pixels changed between t=0 and t=1 — no parallax"
    );
}
#[tokio::test]
async fn empty_batch_renders_the_background() {
    let rig = synth::dev_rig_with_resolutions(uvec2(CENTER.0, CENTER.1), uvec2(SIDE.0, SIDE.1));
    let camera: Camera = rig.virtual_camera(0.5, rig.left.intrinsics, rig.left.resolution);
    let (image, stats) = renderer()
        .render_async(&SplatBatch::new(), &camera)
        .await
        .expect("render");
    assert_eq!(
        stats,
        RenderStats {
            total_splats: 0,
            visible_splats: 0
        }
    );
    assert_eq!((image.width, image.height), SIDE);
    // Every pixel is the (transparent) background.
    for y in 0..image.height {
        for x in 0..image.width {
            assert_eq!(image.pixel(x, y)[3], 0);
        }
    }
}
#[tokio::test]
async fn splats_behind_the_camera_are_culled() {
    let rig = synth::dev_rig_with_resolutions(uvec2(CENTER.0, CENTER.1), uvec2(SIDE.0, SIDE.1));
    let batch = first_frame();
    // A virtual camera looking away from the subject: everything ends up
    // behind the near plane.
    let mut camera = rig.virtual_camera(0.5, rig.left.intrinsics, rig.left.resolution);
    camera.pose = rig::CameraPose::looking_at(
        SUBJECT + vec3(0.0, 0.0, 2.0),
        SUBJECT + vec3(0.0, 0.0, 10.0),
    );
    let (image, stats) = renderer()
        .render_async(&batch, &camera)
        .await
        .expect("render");
    assert_eq!(stats.total_splats, batch.len());
    assert_eq!(stats.visible_splats, 0, "everything is behind the camera");
    // The image is all background.
    assert!(image.rgba.as_chunks::<4>().0.iter().all(|px| px[3] == 0));
}

replay/Cargo.toml +3 -0

diff --git a/replay/Cargo.toml b/replay/Cargo.toml
index d25f382..c3a815c 100644
@@ -6,3 +6,6 @@ edition.workspace = true
publish = false
[dependencies]
anyhow.workspace = true
glam.workspace = true
rig.workspace = true

replay/src/lib.rs +17 -6

diff --git a/replay/src/lib.rs b/replay/src/lib.rs
index 8a9ab55..aa17ea1 100644
@@ -1,11 +1,22 @@
//! Recorded and synthetic rig datasets.
//!
//! Pipeline role: a frame source for development and CI that runs without
//! any hardware. Presents the same rig-frame interface as `capture`, so
//! downstream code cannot tell live and replayed input apart.
//! Pipeline role: a frame source for development and CI that runs
//! without any hardware. Presents the same [`rig::FrameSource`] interface
//! as live capture, so downstream code cannot tell live and replayed
//! input apart.
//!
//! Sources:
//!
//! * recorded Kinect + webcam streams (for `splatgen` work on real data)
//! * synthetic scenes with known ground truth (for exercising `render`'s
//!   virtual camera and the `sink` UI)
//! * [`synth`] — synthetic scenes with known ground truth: a
//!   software depth camera raycasts the scene from the rig's center
//!   (Kinect) viewpoint and builds footprint-sized surfels through the
//!   same `rig::Splat::surfel` path the real `splatgen` will use. This
//!   exercises the render/viewer path end-to-end before any Kinect
//!   hardware is involved.
//! * recorded Kinect + webcam streams land here with build-order step 2.
pub mod scene;
pub mod synth;
pub use scene::Scene;
pub use synth::SyntheticSource;

replay/src/scene.rs +346 -0

diff --git a/replay/src/scene.rs b/replay/src/scene.rs
new file mode 100644
index 0000000..5b29e72
@@ -0,0 +1,346 @@
//! Synthetic scenes with exact, analytic geometry.
//!
//! Surfaces are simple enough to intersect exactly (ellipsoids, planes)
//! yet shaped like the real subject — a head, shoulders, and a backdrop —
//! so surfel construction, the virtual camera, and the UI can be
//! developed and tested against known ground truth.
use glam::Vec3;
/// A ray-surface intersection.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Hit {
    /// Distance along the ray.
    pub t: f32,
    /// World-space hit position.
    pub position: Vec3,
    /// Unit surface normal at the hit.
    pub normal: Vec3,
    /// Surface colour at the hit, RGB in `[0, 1]`.
    pub color: [f32; 3],
}
/// A scene surface.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum Surface {
    /// An axis-aligned ellipsoid (sphere with per-axis radii).
    Ellipsoid {
        center: Vec3,
        radii: Vec3,
        color: [f32; 3],
    },
    /// A backdrop plane with a checkerboard pattern, `cell` metres per
    /// cell (0 disables the pattern: flat colour).
    CheckerPlane {
        point: Vec3,
        normal: Vec3,
        cell: f32,
        color_a: [f32; 3],
        color_b: [f32; 3],
    },
}
impl Surface {
    /// Intersect `origin + t * dir` with the surface; the nearest hit
    /// with `t > 0`.
    ///
    /// `dir` need not be normalized (as long as the caller compares
    /// distances consistently, which [`Scene::cast_ray`] does).
    pub fn intersect(&self, origin: Vec3, dir: Vec3) -> Option<Hit> {
        match *self {
            Surface::Ellipsoid {
                center,
                radii,
                color,
            } => {
                // Scale space so the ellipsoid becomes the unit sphere.
                let o = (origin - center) / radii;
                let d = dir / radii;
                // Solve |o + t d|² = 1.
                let a = d.dot(d);
                let b = 2.0 * o.dot(d);
                let c = o.dot(o) - 1.0;
                let disc = b * b - 4.0 * a * c;
                if disc < 0.0 || a <= f32::EPSILON {
                    return None;
                }
                let sqrt_disc = disc.sqrt();
                // Near root; for a ray starting inside the ellipsoid it is
                // negative, so fall back to the far (exit) root.
                let mut t = (-b - sqrt_disc) / (2.0 * a);
                if t <= f32::EPSILON {
                    t = (-b + sqrt_disc) / (2.0 * a);
                }
                if t <= f32::EPSILON {
                    return None;
                }
                let position = origin + dir * t;
                // Normal: gradient of the implicit function at the hit,
                // mapped back to world space.
                let grad = (position - center) / (radii * radii);
                let normal = grad.normalize_or_zero();
                Some(Hit {
                    t,
                    position,
                    normal,
                    color,
                })
            }
            Surface::CheckerPlane {
                point,
                normal,
                cell,
                color_a,
                color_b,
            } => {
                let normal = normal.normalize_or_zero();
                let denom = dir.dot(normal);
                if denom.abs() <= f32::EPSILON {
                    return None; // parallel
                }
                let t = (point - origin).dot(normal) / denom;
                if t <= f32::EPSILON {
                    return None;
                }
                let position = origin + dir * t;
                let color = if cell > 0.0 {
                    // Checkerboard over the plane's in-plane coordinates.
                    let (u, v) = plane_coords(position, normal);
                    let parity = ((u / cell).floor() + (v / cell).floor()) as i32 % 2;
                    if parity.rem_euclid(2) == 0 {
                        color_a
                    } else {
                        color_b
                    }
                } else {
                    color_a
                };
                Some(Hit {
                    t,
                    position,
                    normal,
                    color,
                })
            }
        }
    }
}
/// In-plane coordinates of a point on a plane with the given normal,
/// for the checkerboard pattern.
fn plane_coords(p: Vec3, normal: Vec3) -> (f32, f32) {
    // Build any orthonormal tangent basis.
    let reference = if normal.abs().dot(Vec3::Y) > 0.999 {
        Vec3::X
    } else {
        Vec3::Y
    };
    let t1 = normal.cross(reference).normalize_or_zero();
    let t2 = normal.cross(t1).normalize_or_zero();
    (p.dot(t1), p.dot(t2))
}
/// The default development scene's subject: a head-and-shoulders stand-in
/// at `subject`, facing the rig's center camera at the origin.
///
/// Returns `(head, shoulders)`; the backdrop is [`backdrop`].
pub fn head_and_shoulders(subject: Vec3) -> (Surface, Surface) {
    (
        // Head: slightly egg-shaped ellipsoid, ~22 cm tall.
        Surface::Ellipsoid {
            center: subject + Vec3::new(0.0, 0.18, 0.0),
            radii: Vec3::new(0.09, 0.11, 0.10),
            color: [0.85, 0.62, 0.52],
        },
        // Shoulders.
        Surface::Ellipsoid {
            center: subject + Vec3::new(0.0, -0.05, 0.0),
            radii: Vec3::new(0.24, 0.13, 0.16),
            color: [0.20, 0.35, 0.55],
        },
    )
}
/// The default development scene's backdrop: a checkerboard plane
/// `0.8 m` behind the subject, facing the rig.
pub fn backdrop(subject: Vec3) -> Surface {
    Surface::CheckerPlane {
        point: subject + Vec3::new(0.0, 0.0, 0.8),
        normal: Vec3::new(0.0, 0.0, -1.0),
        cell: 0.25,
        color_a: [0.72, 0.72, 0.72],
        color_b: [0.28, 0.28, 0.28],
    }
}
impl Surface {
    /// The surface translated by `offset`.
    pub fn translated(&self, offset: Vec3) -> Self {
        match *self {
            Surface::Ellipsoid {
                center,
                radii,
                color,
            } => Surface::Ellipsoid {
                center: center + offset,
                radii,
                color,
            },
            Surface::CheckerPlane {
                point,
                normal,
                cell,
                color_a,
                color_b,
            } => Surface::CheckerPlane {
                point: point + offset,
                normal,
                cell,
                color_a,
                color_b,
            },
        }
    }
}
/// A synthetic scene: a list of surfaces, cast against per pixel.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct Scene {
    pub surfaces: Vec<Surface>,
}
impl Scene {
    pub fn new() -> Self {
        Self::default()
    }
    /// Add a surface.
    pub fn with(mut self, surface: Surface) -> Self {
        self.surfaces.push(surface);
        self
    }
    /// The default development scene: a head-and-shoulders stand-in in
    /// front of a checkerboard backdrop, centred at `subject` (roughly
    /// where the real subject sits, PLAN.md pipeline step 1).
    ///
    /// The subject faces the rig's center camera at the origin (i.e. its
    /// face points towards -Z). For the animated version used by
    /// [`crate::synth`], see [`head_and_shoulders`] and [`backdrop`].
    pub fn head_stand_in(subject: Vec3) -> Self {
        let (head, shoulders) = head_and_shoulders(subject);
        Self::new()
            .with(head)
            .with(shoulders)
            .with(backdrop(subject))
    }
    /// Cast a ray against the scene; the nearest surface hit.
    pub fn cast_ray(&self, origin: Vec3, dir: Vec3) -> Option<Hit> {
        self.surfaces
            .iter()
            .filter_map(|s| s.intersect(origin, dir))
            .min_by(|a, b| a.t.partial_cmp(&b.t).unwrap())
    }
}
#[cfg(test)]
mod tests {
    use super::*;
    use glam::vec3;
    fn scene() -> Scene {
        Scene::head_stand_in(vec3(0.0, 0.0, 2.0))
    }
    #[test]
    fn center_ray_hits_the_head() {
        // Straight at the head's centre: (0, 0.18, 2.0).
        let head_center = vec3(0.0, 0.18, 2.0);
        let hit = scene()
            .cast_ray(Vec3::ZERO, head_center)
            .expect("head in the way");
        // The hit is on the ellipsoid's near side, on its surface.
        assert!(hit.position.distance(head_center) < 0.11);
        assert!(
            hit.position.z < head_center.z - 0.05,
            "must hit the near side, hit z = {}",
            hit.position.z
        );
        // Normal faces back at the camera (front face of the ellipsoid).
        assert!(hit.normal.dot(hit.position) < 0.0);
    }
    #[test]
    fn backdrop_behind_the_subject() {
        let hit = scene()
            .cast_ray(Vec3::ZERO, vec3(0.0, 0.35, 1.0))
            .expect("backdrop in the way");
        assert!((hit.position.z - 2.8).abs() < 1e-5);
        assert!((hit.normal - vec3(0.0, 0.0, -1.0)).length() < 1e-6);
        // Checkerboard is one of the two greys.
        let c = hit.color;
        assert!(
            (c[0] - 0.72).abs() < 1e-5 || (c[0] - 0.28).abs() < 1e-5,
            "checker color was {c:?}"
        );
    }
    #[test]
    fn nearest_hit_wins() {
        // A ray above the head hits only the backdrop.
        let hit = scene()
            .cast_ray(Vec3::ZERO, vec3(0.0, 0.5, 1.0))
            .expect("backdrop");
        assert!((hit.position.z - 2.8).abs() < 1e-5);
        // A ray through the head hits the head, not the backdrop behind it.
        let hit = scene()
            .cast_ray(Vec3::ZERO, vec3(0.0, 0.09, 1.0))
            .expect("head");
        assert!(hit.position.z < 2.0, "nearest hit must be the head");
    }
    #[test]
    fn misses_report_none() {
        // Only rays pointing away from the scene miss: everything forward
        // eventually reaches the (infinite) backdrop plane.
        assert!(scene().cast_ray(Vec3::ZERO, vec3(0.0, 0.0, -1.0)).is_none());
        // Straight up, parallel to the backdrop: miss.
        assert!(scene().cast_ray(Vec3::ZERO, vec3(0.0, 1.0, 0.0)).is_none());
    }
    #[test]
    fn ellipsoid_hit_math() {
        let s = Surface::Ellipsoid {
            center: vec3(0.0, 0.0, 2.0),
            radii: vec3(1.0, 1.0, 1.0),
            color: [1.0, 0.0, 0.0],
        };
        let hit = s.intersect(Vec3::ZERO, vec3(0.0, 0.0, 1.0)).expect("hits");
        assert!((hit.position.z - 1.0).abs() < 1e-5);
        assert!((hit.normal - vec3(0.0, 0.0, -1.0)).length() < 1e-5);
        assert!((hit.t - 1.0).abs() < 1e-5);
        // Ray from inside: front face ahead of the origin.
        let hit = s
            .intersect(vec3(0.0, 0.0, 2.0), vec3(0.0, 0.0, 1.0))
            .expect("exits");
        assert!((hit.t - 1.0).abs() < 1e-5);
    }
    #[test]
    fn checker_pattern_alternates() {
        let s = Surface::CheckerPlane {
            point: vec3(0.0, 0.0, 3.0),
            normal: vec3(0.0, 0.0, -1.0),
            cell: 1.0,
            color_a: [1.0, 1.0, 1.0],
            color_b: [0.0, 0.0, 0.0],
        };
        let a = s.intersect(Vec3::ZERO, vec3(0.0, 0.0, 1.0)).unwrap();
        let b = s.intersect(Vec3::ZERO, vec3(1.0, 0.0, 3.0)).unwrap();
        let c = s.intersect(Vec3::ZERO, vec3(-1.0, 0.0, 3.0)).unwrap();
        assert_eq!(a.color, [1.0, 1.0, 1.0]);
        assert_eq!(b.color, [0.0, 0.0, 0.0]);
        assert_eq!(c.color, [0.0, 0.0, 0.0]);
    }
}

replay/src/synth.rs +338 -0

diff --git a/replay/src/synth.rs b/replay/src/synth.rs
new file mode 100644
index 0000000..e195c77
@@ -0,0 +1,338 @@
//! The synthetic frame source: a software depth camera.
//!
//! Every frame, [`SyntheticSource`] raycasts the scene from the rig's
//! center (Kinect) viewpoint — one ray per depth pixel — and turns each
//! hit into a footprint-sized surfel via [`rig::Splat::surfel`], the same
//! construction the real `splatgen` uses on Kinect depth. The subject
//! sways a little so temporal behaviour is testable, deterministically
//! (a function of the frame index, never the wall clock).
use anyhow::Result;
use glam::{UVec2, Vec3};
use rig::{
    Camera, CameraIntrinsics, CameraPose, Frame, FrameSource, Rig, SURFEL_THICKNESS_M, Splat,
    SplatBatch,
};
use crate::scene::{self, Scene, Surface};
/// Where the synthetic subject sits: 2 m in front of the rig's center
/// camera at the origin.
pub const SUBJECT: Vec3 = Vec3::new(0.0, 0.0, 2.0);
/// Opacity used for synthetic surfels (PLAN.md: ~0.9–0.99).
pub const SURFEL_OPACITY: f32 = 0.95;
/// Subject sway amplitude, metres, and period in frames.
const SWAY_AMPLITUDE_M: f32 = 0.04;
const SWAY_PERIOD_FRAMES: u64 = 90;
/// The development rig: a Kinect-like center camera flanked by two
/// webcams 0.8 m apart, converged on [`SUBJECT`].
///
/// The center camera approximates the Kinect v2 depth sensor (70° × 60°
/// field of view); the side cameras are typical 60° webcams. Pass smaller
/// resolutions to keep the same fields of view with fewer pixels — what
/// the test suites use to stay fast.
pub fn dev_rig_with_resolutions(center: UVec2, side: UVec2) -> Rig {
    // Kinect v2 depth: ~70° x 60° field of view.
    let center = Camera {
        pose: CameraPose::looking_at(Vec3::ZERO, SUBJECT),
        intrinsics: CameraIntrinsics {
            fx: CENTER_FX * center.x as f32,
            fy: CENTER_FY * center.y as f32,
            cx: center.x as f32 / 2.0,
            cy: center.y as f32 / 2.0,
        },
        resolution: center,
    };
    // Typical webcams with a 60 degree horizontal FOV.
    let make_side = |x: f32| Camera {
        pose: CameraPose::looking_at(Vec3::new(x, 0.0, 0.0), SUBJECT),
        intrinsics: CameraIntrinsics {
            fx: SIDE_FX * side.x as f32,
            fy: SIDE_FX * side.y as f32,
            cx: side.x as f32 / 2.0,
            cy: side.y as f32 / 2.0,
        },
        resolution: side,
    };
    Rig {
        left: make_side(-0.4),
        center,
        right: make_side(0.4),
    }
}
/// Center-camera focal ratios (per resolution pixel): a ~70° horizontal
/// and 60° vertical field of view, like the Kinect v2 depth sensor.
const CENTER_FX: f32 = 0.714_074; // 1 / (2 * tan(35°)), FOV_x = 70°
const CENTER_FY: f32 = 0.866_025_4; // 1 / (2 * tan(30°)), FOV_y = 60°
/// Side-camera focal ratio: a 60° horizontal FOV, square pixels.
const SIDE_FX: f32 = 0.866_025_4; // 1 / (2 * tan(30°))
/// The development rig at full size: Kinect-like depth at 256x212
/// (~54k active splats, inside the plan's 50–150k budget) and 640x480
/// webcams.
pub fn dev_rig() -> Rig {
    dev_rig_with_resolutions(UVec2::new(256, 212), UVec2::new(640, 480))
}
/// A synthetic source over the default development scene (see
/// [`dev_rig`]), at `fps` frames per second.
pub fn dev_source(fps: f32) -> SyntheticSource {
    dev_source_with_center_camera(dev_rig().center, fps)
}
/// Like [`dev_source`], but with a custom center (depth) camera — e.g.
/// `dev_rig_with_resolutions(..).center` at a small resolution for tests.
pub fn dev_source_with_center_camera(center: Camera, fps: f32) -> SyntheticSource {
    let (head, shoulders) = scene::head_and_shoulders(SUBJECT);
    SyntheticSource::new(
        Scene::new().with(scene::backdrop(SUBJECT)),
        vec![head, shoulders],
        center,
        fps,
    )
}
/// A synthetic depth-camera frame source.
pub struct SyntheticSource {
    /// Static surroundings (backdrop).
    scene: Scene,
    /// The animated subject surfaces, translated by the sway offset each
    /// frame.
    subject: Vec<Surface>,
    /// The depth viewpoint: the rig's center camera.
    center: Camera,
    fps: f32,
    frame_index: u64,
}
impl SyntheticSource {
    pub fn new(scene: Scene, subject: Vec<Surface>, center: Camera, fps: f32) -> Self {
        Self {
            scene,
            subject,
            center,
            fps: fps.max(1.0),
            frame_index: 0,
        }
    }
    /// The number of frames produced so far.
    pub fn frame_index(&self) -> u64 {
        self.frame_index
    }
    /// The depth viewpoint: the rig's center camera.
    pub fn center(&self) -> &Camera {
        &self.center
    }
    /// The deterministic subject sway offset for a given frame.
    fn sway_offset(&self, frame: u64) -> Vec3 {
        let phase = 2.0 * std::f32::consts::PI * (frame % SWAY_PERIOD_FRAMES) as f32
            / SWAY_PERIOD_FRAMES as f32;
        Vec3::new(SWAY_AMPLITUDE_M * phase.sin(), 0.0, 0.0)
    }
    /// The full scene for a given frame: static surroundings plus the
    /// swayed subject.
    fn frame_scene(&self, frame: u64) -> Scene {
        let offset = self.sway_offset(frame);
        let mut scene = self.scene.clone();
        for surface in &self.subject {
            scene.surfaces.push(surface.translated(offset));
        }
        scene
    }
    /// Build the splat batch the software depth camera sees in `frame`.
    fn build_frame(&self, frame: u64) -> SplatBatch {
        let scene = self.frame_scene(frame);
        let (width, height) = (self.center.resolution.x, self.center.resolution.y);
        let world_from_camera = self.center.pose.world_from_camera();
        let mut batch = SplatBatch::with_capacity((width * height) as usize);
        for y in 0..height {
            for x in 0..width {
                // Sample through pixel centres.
                let (origin, dir) = self.center.pixel_ray(x as f32 + 0.5, y as f32 + 0.5);
                let Some(hit) = scene.cast_ray(origin, dir) else {
                    continue;
                };
                // Depth along the viewing axis (like a Kinect depth map,
                // which is what PLAN.md's footprint formula expects).
                let depth = world_from_camera.inverse().transform_point3(hit.position).z;
                batch.push(Splat::surfel(
                    hit.position,
                    hit.normal,
                    depth,
                    &self.center.intrinsics,
                    SURFEL_THICKNESS_M,
                    hit.color,
                    SURFEL_OPACITY,
                ));
            }
        }
        batch
    }
}
impl FrameSource for SyntheticSource {
    fn next_frame(&mut self) -> Result<Frame> {
        let frame = self.frame_index;
        self.frame_index += 1;
        Ok(Frame {
            timestamp_secs: frame as f64 / self.fps as f64,
            splats: self.build_frame(frame),
        })
    }
}
#[cfg(test)]
mod tests {
    use super::*;
    use crate::scene;
    /// A small-resolution source so tests stay fast: 32x24 depth pixels.
    fn tiny_source() -> SyntheticSource {
        let (head, shoulders) = scene::head_and_shoulders(SUBJECT);
        let mut center = dev_rig().center;
        // Keep the center camera's FOV, shrink the resolution.
        let scale = 32.0 / center.resolution.x as f32;
        center.intrinsics.fx *= scale;
        center.intrinsics.fy *= scale;
        center.intrinsics.cx = 16.0;
        center.intrinsics.cy = 12.0;
        center.resolution = UVec2::new(32, 24);
        SyntheticSource::new(
            Scene::new().with(scene::backdrop(SUBJECT)),
            vec![head, shoulders],
            center,
            30.0,
        )
    }
    #[test]
    fn frames_are_nonempty_and_in_front() {
        let mut src = tiny_source();
        let frame = src.next_frame().unwrap();
        assert!(!frame.splats.is_empty());
        // One splat per depth pixel that hit — never more.
        assert!(frame.splats.len() <= 32 * 24);
        // Most rays hit the backdrop.
        assert!(frame.splats.len() > 32 * 24 / 2);
        // Every splat lies in front of the center camera and matches the
        // footprint formula.
        let cam = *src.center();
        let camera_from_world = cam.pose.camera_from_world();
        for splat in frame.splats.iter() {
            let p = camera_from_world.transform_point3(splat.mean);
            assert!(p.z > 0.5, "splat behind camera: {p:?}");
            assert!((splat.scale.x - p.z / cam.intrinsics.fx).abs() < 1e-5);
            assert!((splat.scale.y - p.z / cam.intrinsics.fy).abs() < 1e-5);
            assert!((splat.opacity - SURFEL_OPACITY).abs() < 1e-6);
        }
    }
    #[test]
    fn subject_and_backdrop_both_present() {
        let mut src = tiny_source();
        let frame = src.next_frame().unwrap();
        let mut skin = 0;
        let mut checker = 0;
        for splat in frame.splats.iter() {
            if (splat.color[0] - 0.85).abs() < 0.01 {
                skin += 1;
            }
            // Checker greys: 0.72 or 0.28.
            if (splat.color[0] - splat.color[1]).abs() < 1e-4
                && ((splat.color[0] - 0.72).abs() < 0.01 || (splat.color[0] - 0.28).abs() < 0.01)
            {
                checker += 1;
            }
        }
        assert!(skin > 0, "no subject splats");
        assert!(checker > 0, "no backdrop splats");
    }
    #[test]
    fn timestamps_advance_with_index() {
        let mut src = tiny_source();
        assert_eq!(src.next_frame().unwrap().timestamp_secs, 0.0);
        assert!((src.next_frame().unwrap().timestamp_secs - 1.0 / 30.0).abs() < 1e-9);
        assert!((src.next_frame().unwrap().timestamp_secs - 2.0 / 30.0).abs() < 1e-9);
    }
    #[test]
    fn playback_is_deterministic() {
        let mut a = tiny_source();
        let mut b = tiny_source();
        for _ in 0..5 {
            assert_eq!(a.next_frame().unwrap(), b.next_frame().unwrap());
        }
    }
    #[test]
    fn subject_sways_and_repeats() {
        let mut src = tiny_source();
        let frame0 = src.next_frame().unwrap();
        // The frame produced at a quarter period (index 22) is at (nearly)
        // maximum sway.
        let mut quarter = None;
        while src.frame_index() <= SWAY_PERIOD_FRAMES / 4 {
            quarter = Some(src.next_frame().unwrap());
        }
        let quarter = quarter.expect("quarter frame produced");
        assert_ne!(
            frame0.splats, quarter.splats,
            "no visible sway at a quarter period"
        );
        // The frame at index SWAY_PERIOD_FRAMES repeats frame 0 exactly.
        let mut period = None;
        while src.frame_index() <= SWAY_PERIOD_FRAMES {
            period = Some(src.next_frame().unwrap());
        }
        let period = period.expect("period frame produced");
        assert_eq!(
            frame0.splats, period.splats,
            "sway must be periodic in the frame index"
        );
    }
    #[test]
    fn sway_offset_is_deterministic_and_bounded() {
        let src = tiny_source();
        let offset = src.sway_offset(0);
        assert_eq!(offset, Vec3::ZERO);
        assert_eq!(src.sway_offset(SWAY_PERIOD_FRAMES), Vec3::ZERO);
        // A quarter period (90/4 = 22 frames) is close to the maximum.
        let max = src.sway_offset(SWAY_PERIOD_FRAMES / 4);
        assert!(max.x > 0.99 * SWAY_AMPLITUDE_M, "sway max was {max:?}");
        for frame in 0..SWAY_PERIOD_FRAMES * 2 {
            let o = src.sway_offset(frame);
            assert!(o.x.abs() <= SWAY_AMPLITUDE_M + 1e-5);
        }
    }
    #[test]
    fn dev_rig_is_symmetric() {
        let rig = dev_rig();
        assert_eq!(rig.left.resolution, UVec2::new(640, 480));
        assert_eq!(rig.right.resolution, UVec2::new(640, 480));
        assert_eq!(rig.center.resolution, UVec2::new(256, 212));
        // The center camera sits exactly between the side cameras.
        let mid = rig.virtual_pose(0.5);
        assert!(mid.position.distance(rig.center.pose.position) < 1e-6);
        // And it looks at the subject.
        let forward = mid.rotation * Vec3::Z;
        let to_subject = (SUBJECT - mid.position).normalize();
        assert!(forward.dot(to_subject) > 0.999);
    }
}

rig/Cargo.toml +10 -0

diff --git a/rig/Cargo.toml b/rig/Cargo.toml
new file mode 100644
index 0000000..fccb5d8
@@ -0,0 +1,10 @@
[package]
name = "rig"
description = "Shared rig types: camera poses and intrinsics, the virtual camera, splat batches, frame sources"
version.workspace = true
edition.workspace = true
publish = false
[dependencies]
anyhow.workspace = true
glam.workspace = true

rig/src/camera.rs +233 -0

diff --git a/rig/src/camera.rs b/rig/src/camera.rs
new file mode 100644
index 0000000..524b393
@@ -0,0 +1,233 @@
//! Camera poses, pinhole intrinsics, and projection math.
use glam::{Affine3A, Mat3, Quat, UVec2, Vec2, Vec3};
/// A pinhole camera's intrinsics, in pixels at the camera's native
/// resolution.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct CameraIntrinsics {
    /// Focal length in pixels, horizontal.
    pub fx: f32,
    /// Focal length in pixels, vertical.
    pub fy: f32,
    /// Principal point in pixels (image centre for a well-calibrated rig).
    pub cx: f32,
    pub cy: f32,
}
impl CameraIntrinsics {
    /// Project a camera-space point to pixel coordinates `(u, v)`.
    ///
    /// Returns `None` for points on or behind the camera plane.
    pub fn project_camera_point(&self, p: Vec3) -> Option<Vec2> {
        if p.z <= f32::EPSILON {
            return None;
        }
        Some(Vec2::new(
            self.fx * p.x / p.z + self.cx,
            self.fy * p.y / p.z + self.cy,
        ))
    }
    /// The camera-space direction (unnormalized) of a pixel's view ray.
    pub fn camera_ray(&self, u: f32, v: f32) -> Vec3 {
        Vec3::new((u - self.cx) / self.fx, (v - self.cy) / self.fy, 1.0)
    }
}
/// The position and orientation of a camera in world space.
///
/// The rotation is camera-to-world; the camera looks along its local +Z
/// (COLMAP convention). With [`CameraPose::identity`] the camera frame
/// equals the world frame, so it looks down world +Z with image-right at
/// world +X — and the image upside-down in a Y-up world (an identity pose
/// is just a pose; use [`CameraPose::looking_at`] for a right-side-up
/// view).
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct CameraPose {
    pub position: Vec3,
    pub rotation: Quat,
}
impl CameraPose {
    /// Camera at the origin, looking down world +Z.
    pub fn identity() -> Self {
        Self {
            position: Vec3::ZERO,
            rotation: Quat::IDENTITY,
        }
    }
    /// A pose at `position` that looks at `target` with the world up
    /// (+Y) at the top of the image.
    ///
    /// Falls back to +Z as "up" when the view direction is near-vertical.
    pub fn looking_at(position: Vec3, target: Vec3) -> Self {
        let forward = (target - position).normalize_or_zero();
        // Camera basis: x = image-right, y = image-down, z = forward,
        // right-handed, so x = y × z and y = z × x.
        let up = if forward.abs().dot(Vec3::Y) > 0.999 {
            Vec3::Z
        } else {
            Vec3::Y
        };
        let right = (-up).cross(forward).normalize_or_zero();
        let down = forward.cross(right).normalize_or_zero();
        let rotation = Quat::from_mat3(&Mat3::from_cols(right, down, forward));
        Self { position, rotation }
    }
    /// The camera-to-world transform.
    pub fn world_from_camera(&self) -> Affine3A {
        Affine3A::from_rotation_translation(self.rotation, self.position)
    }
    /// The world-to-camera transform.
    pub fn camera_from_world(&self) -> Affine3A {
        self.world_from_camera().inverse()
    }
    /// Interpolate from `self` at `t = 0` to `other` at `t = 1`.
    ///
    /// Positions lerp, orientations slerp. Used to place the virtual
    /// camera between the physical rig cameras.
    pub fn blend(&self, other: &Self, t: f32) -> Self {
        Self {
            position: self.position.lerp(other.position, t),
            rotation: self.rotation.slerp(other.rotation, t),
        }
    }
}
/// A pinhole camera: pose, intrinsics, and image resolution.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Camera {
    pub pose: CameraPose,
    pub intrinsics: CameraIntrinsics,
    /// Image size in pixels: `(width, height)`.
    pub resolution: UVec2,
}
impl Camera {
    /// Project a world-space point to pixel coordinates, or `None` if the
    /// point is behind the camera.
    pub fn project(&self, world: Vec3) -> Option<Vec2> {
        let p = self.pose.camera_from_world().transform_point3(world);
        self.intrinsics.project_camera_point(p)
    }
    /// The world-space ray through the centre of pixel `(u, v)`:
    /// `(origin, normalized_direction)`.
    pub fn pixel_ray(&self, u: f32, v: f32) -> (Vec3, Vec3) {
        let dir_cam = self.intrinsics.camera_ray(u, v).normalize_or_zero();
        let dir_world = self.pose.rotation * dir_cam;
        (self.pose.position, dir_world)
    }
    /// Horizontal field of view in radians, matching Brush's convention
    /// (`focal = (width / 2) / tan(fov / 2)`).
    pub fn fov_x(&self) -> f32 {
        2.0 * ((self.resolution.x as f32 / 2.0) / self.intrinsics.fx).atan()
    }
    /// Vertical field of view in radians.
    pub fn fov_y(&self) -> f32 {
        2.0 * ((self.resolution.y as f32 / 2.0) / self.intrinsics.fy).atan()
    }
}
#[cfg(test)]
mod tests {
    use super::*;
    use glam::{uvec2, vec2, vec3};
    fn cam_looking_at_subject() -> Camera {
        Camera {
            pose: CameraPose::looking_at(Vec3::ZERO, vec3(0.0, 0.0, 2.0)),
            intrinsics: CameraIntrinsics {
                fx: 500.0,
                fy: 500.0,
                cx: 320.0,
                cy: 240.0,
            },
            resolution: uvec2(640, 480),
        }
    }
    #[test]
    fn identity_pose_looks_down_plus_z() {
        let cam = Camera {
            pose: CameraPose::identity(),
            intrinsics: CameraIntrinsics {
                fx: 100.0,
                fy: 100.0,
                cx: 50.0,
                cy: 50.0,
            },
            resolution: uvec2(100, 100),
        };
        // A point straight ahead at +Z maps to the principal point.
        assert_eq!(cam.project(vec3(0.0, 0.0, 5.0)), Some(vec2(50.0, 50.0)));
        // Camera frame = world frame: image-right is +X, image-down is +Y
        // (in a Y-up world this pose renders upside-down; it's the COLMAP
        // identity, see CameraPose docs).
        assert!(cam.project(vec3(1.0, 0.0, 5.0)).unwrap().x > 50.0);
        assert!(cam.project(vec3(0.0, 1.0, 5.0)).unwrap().y > 50.0);
        // Behind the camera: no projection.
        assert_eq!(cam.project(vec3(0.0, 0.0, -5.0)), None);
    }
    #[test]
    fn looking_at_is_right_side_up() {
        let cam = cam_looking_at_subject();
        // The subject's head (above the look-at point) must land above the
        // principal point: world up is image up.
        let head = cam.project(vec3(0.0, 0.5, 2.0)).expect("in front");
        assert!(head.y < cam.intrinsics.cy, "head projected to {head:?}");
        // Something to the camera's right (world -X when looking down +Z,
        // see CameraPose docs) lands on the right half of the image.
        let right = cam.project(vec3(-0.5, 0.0, 2.0)).expect("in front");
        assert!(right.x > cam.intrinsics.cx);
        // ...and world +X lands on the left half.
        let left = cam.project(vec3(0.5, 0.0, 2.0)).expect("in front");
        assert!(left.x < cam.intrinsics.cx);
    }
    #[test]
    fn pixel_ray_round_trips() {
        let cam = cam_looking_at_subject();
        // Ray through the principal point hits the look-at target.
        let (o, d) = cam.pixel_ray(320.0, 240.0);
        let hit = o + d * 2.0;
        assert!(hit.distance_squared(vec3(0.0, 0.0, 2.0)) < 1e-8);
        // Marching a ray and projecting back gives the same pixel.
        let (o, d) = cam.pixel_ray(123.0, 45.0);
        let p = o + d * 3.7;
        let projected = cam.project(p).expect("in front");
        assert!((projected - vec2(123.0, 45.0)).length() < 1e-3);
    }
    #[test]
    fn pose_blend_hits_endpoints() {
        let a = CameraPose::looking_at(vec3(-1.0, 0.0, 0.0), vec3(0.0, 0.0, 2.0));
        let b = CameraPose::looking_at(vec3(1.0, 0.0, 0.0), vec3(0.0, 0.0, 2.0));
        assert_eq!(a.blend(&b, 0.0), a);
        assert_eq!(a.blend(&b, 1.0), b);
        let mid = a.blend(&b, 0.5);
        // Halfway along the baseline, still looking at the subject.
        assert!(mid.position.length() < 1e-6);
        let forward = mid.rotation * Vec3::Z;
        assert!(forward.z > 0.99);
    }
    #[test]
    fn fov_matches_focal() {
        let cam = cam_looking_at_subject();
        // Brush's formula: focal = (w/2) / tan(fov/2).
        let focal = (640.0 / 2.0) / (cam.fov_x() / 2.0).tan();
        assert!((focal - 500.0).abs() < 1e-3);
        let focal_y = (480.0 / 2.0) / (cam.fov_y() / 2.0).tan();
        assert!((focal_y - 500.0).abs() < 1e-3);
    }
}

rig/src/lib.rs +26 -0

diff --git a/rig/src/lib.rs b/rig/src/lib.rs
new file mode 100644
index 0000000..738176f
@@ -0,0 +1,26 @@
//! Shared types for the fixed 3-camera rig.
//!
//! Everything that flows between the pipeline crates — camera poses and
//! intrinsics, the virtual camera, the per-frame splat batch, and the
//! [`FrameSource`] abstraction — lives here so that `replay`, `splatgen`,
//! `render` and `sink` share one vocabulary without depending on each
//! other's heavy dependencies.
//!
//! Conventions (matching Brush, see PLAN.md "Core design"):
//!
//! * world space is right-handed, Y up, in metres;
//! * camera frames follow COLMAP/OpenCV: +X image-right, +Y image-down,
//!   +Z the viewing direction — the same convention Brush's renderer
//!   expects;
//! * image coordinates: `(0, 0)` is the top-left pixel, `v` grows downward;
//! * intrinsics are in pixels at the camera's native resolution.
pub mod camera;
pub mod rig;
pub mod source;
pub mod splat;
pub use camera::{Camera, CameraIntrinsics, CameraPose};
pub use rig::Rig;
pub use source::{Frame, FrameSource};
pub use splat::{SURFEL_THICKNESS_M, Splat, SplatBatch};

rig/src/rig.rs +130 -0

diff --git a/rig/src/rig.rs b/rig/src/rig.rs
new file mode 100644
index 0000000..cfbfcaf
@@ -0,0 +1,130 @@
//! The fixed 3-camera rig and the virtual camera steered between its
//! cameras.
use crate::camera::{Camera, CameraIntrinsics, CameraPose};
/// The physical rig: the Kinect (`center`, the depth source) flanked by
/// two webcams (`left`, `right`).
///
/// Extrinsics are known exactly from the offline calibration (see
/// PLAN.md); for the synthetic dataset they are simply declared.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Rig {
    pub left: Camera,
    pub center: Camera,
    pub right: Camera,
}
impl Rig {
    /// Iterate the rig cameras in order `left, center, right`.
    pub fn cameras(&self) -> [Camera; 3] {
        [self.left, self.center, self.right]
    }
    /// The virtual camera pose at position `t` between the side cameras:
    /// `t = 0` is the left camera, `t = 1` the right camera.
    ///
    /// `t` is clamped to `[0, 1]`.
    pub fn virtual_pose(&self, t: f32) -> CameraPose {
        let t = t.clamp(0.0, 1.0);
        self.left.pose.blend(&self.right.pose, t)
    }
    /// The full virtual camera at `t`, using the given intrinsics and
    /// resolution (normally the webcams': the virtual camera *is* a
    /// webcam placed where no real one sits).
    pub fn virtual_camera(
        &self,
        t: f32,
        intrinsics: CameraIntrinsics,
        resolution: glam::UVec2,
    ) -> Camera {
        Camera {
            pose: self.virtual_pose(t),
            intrinsics,
            resolution,
        }
    }
}
#[cfg(test)]
mod tests {
    use super::*;
    use crate::camera::CameraIntrinsics;
    use glam::{uvec2, vec3};
    /// A symmetric converging rig, like the real one: side cameras
    /// `±baseline/2` apart, all looking at a subject 2 m out.
    fn symmetric_rig(baseline: f32) -> Rig {
        let intrinsics = CameraIntrinsics {
            fx: 500.0,
            fy: 500.0,
            cx: 320.0,
            cy: 240.0,
        };
        let make = |x: f32| Camera {
            pose: CameraPose::looking_at(vec3(x, 0.0, 0.0), vec3(0.0, 0.0, 2.0)),
            intrinsics,
            resolution: uvec2(640, 480),
        };
        Rig {
            left: make(-baseline / 2.0),
            center: make(0.0),
            right: make(baseline / 2.0),
        }
    }
    #[test]
    fn virtual_pose_hits_the_side_cameras() {
        let rig = symmetric_rig(0.6);
        assert_eq!(rig.virtual_pose(0.0), rig.left.pose);
        assert_eq!(rig.virtual_pose(1.0), rig.right.pose);
    }
    #[test]
    fn virtual_pose_midpoint_is_center_for_symmetric_rig() {
        let rig = symmetric_rig(0.6);
        let mid = rig.virtual_pose(0.5);
        assert!(mid.position.distance(rig.center.pose.position) < 1e-6);
        // Orientation within a degree of the center camera.
        let dot = mid
            .rotation
            .normalize()
            .dot(rig.center.pose.rotation.normalize());
        assert!(dot > (1.0_f32).cos().to_radians());
    }
    #[test]
    fn virtual_pose_moves_monotonically_along_baseline() {
        let rig = symmetric_rig(1.0);
        let mut last_x = f32::MIN;
        for i in 0..=10 {
            let t = i as f32 / 10.0;
            let x = rig.virtual_pose(t).position.x;
            assert!(x >= last_x, "t={t} moved backwards");
            last_x = x;
        }
    }
    #[test]
    fn virtual_pose_clamps_out_of_range() {
        let rig = symmetric_rig(0.6);
        assert_eq!(rig.virtual_pose(-2.0), rig.virtual_pose(0.0));
        assert_eq!(rig.virtual_pose(7.0), rig.virtual_pose(1.0));
    }
    #[test]
    fn virtual_camera_carries_intrinsics() {
        let rig = symmetric_rig(0.6);
        let intrinsics = CameraIntrinsics {
            fx: 600.0,
            fy: 600.0,
            cx: 320.0,
            cy: 240.0,
        };
        let cam = rig.virtual_camera(0.25, intrinsics, uvec2(640, 480));
        assert_eq!(cam.intrinsics, intrinsics);
        assert_eq!(cam.resolution, uvec2(640, 480));
        assert_eq!(cam.pose, rig.virtual_pose(0.25));
    }
}

rig/src/source.rs +27 -0

diff --git a/rig/src/source.rs b/rig/src/source.rs
new file mode 100644
index 0000000..94305dc
@@ -0,0 +1,27 @@
//! The frame-source abstraction shared by live capture and replay.
use anyhow::Result;
use crate::splat::SplatBatch;
/// One processed rig frame, ready for rendering.
///
/// In the MVP (and the synthetic dataset) a frame is simply the freshly
/// constructed Gaussian list; recorded RGB/depth data rides along from
/// build-order step 2 on.
#[derive(Debug, Clone, PartialEq)]
pub struct Frame {
    /// Frame timestamp in seconds since the source started. Synthetic
    /// sources use deterministic timestamps (frame index / fps).
    pub timestamp_secs: f64,
    pub splats: SplatBatch,
}
/// A source of processed rig frames — the seam where live capture, replay
/// of recorded datasets, and synthetic generation are interchangeable.
///
/// Implementations must be deterministic when replaying: the same frame
/// sequence every run, so tests and recordings behave identically.
pub trait FrameSource {
    fn next_frame(&mut self) -> Result<Frame>;
}

rig/src/splat.rs +247 -0

diff --git a/rig/src/splat.rs b/rig/src/splat.rs
new file mode 100644
index 0000000..1777892
@@ -0,0 +1,247 @@
//! The per-frame Gaussian list: footprint-sized anisotropic surfel splats
//! (see PLAN.md, "Core design").
//!
//! A surfel is a flat disk aligned to the depth-map normal:
//!
//! * two tangential scales = `depth * pixel_size / focal_length`
//!   (pixel size 1 px, focal in pixels ⇒ `depth / fx`),
//! * a few-mm thickness along the normal,
//! * opacity ~0.9–0.99,
//! * rendered with front-to-back alpha blending (Brush's renderer).
use glam::{Mat3, Quat, Vec3};
use crate::camera::CameraIntrinsics;
/// Thickness along the normal for surfels taken from depth data, in
/// metres ("a few mm", PLAN.md).
pub const SURFEL_THICKNESS_M: f32 = 0.003;
/// One Gaussian splat, in the raw (non-log, non-logit) parameterization
/// the pipeline reasons about. Brush's log/logit/SH conversions happen at
/// the render boundary, not here.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Splat {
    /// Centre position in world space, metres.
    pub mean: Vec3,
    /// Orientation: maps local (X, Y, Z) to the splat's axes. For surfels
    /// local Z is the surface normal. Stored as a glam quaternion (x, y,
    /// z, w) — the layout Brush expects.
    pub rotation: Quat,
    /// Per-axis scale in metres (not log scale).
    pub scale: Vec3,
    /// Colour, RGB in `[0, 1]`.
    pub color: [f32; 3],
    /// Opacity in `[0, 1]` (not logit).
    pub opacity: f32,
}
impl Splat {
    /// A footprint-sized surfel from a depth-map sample.
    ///
    /// * `position` — the back-projected depth pixel's 3D position,
    /// * `normal` — the surface normal at that position (unit length),
    /// * `depth` — the pixel's depth (distance along the camera's viewing
    ///   axis), used for the footprint,
    /// * `intrinsics` — the depth camera's intrinsics, for the
    ///   `depth / focal` footprint from PLAN.md,
    /// * `thickness` — extent along the normal in metres.
    pub fn surfel(
        position: Vec3,
        normal: Vec3,
        depth: f32,
        intrinsics: &CameraIntrinsics,
        thickness: f32,
        color: [f32; 3],
        opacity: f32,
    ) -> Self {
        let normal = normal.normalize_or_zero();
        // Footprint: one pixel's world size at this depth.
        let footprint_x = depth / intrinsics.fx;
        let footprint_y = depth / intrinsics.fy;
        let rotation = surfel_rotation(normal);
        Self {
            mean: position,
            rotation,
            scale: Vec3::new(footprint_x, footprint_y, thickness),
            color,
            opacity: opacity.clamp(0.0, 1.0),
        }
    }
    /// The splat's normal axis in world space (its local +Z).
    pub fn normal(&self) -> Vec3 {
        self.rotation * Vec3::Z
    }
}
/// A flat disk oriented so local Z is `normal`: local (X, Y) span the
/// tangent plane, right-handed.
fn surfel_rotation(normal: Vec3) -> Quat {
    // Reference axis not parallel to the normal (world up, or world X
    // for near-vertical normals).
    let reference = if normal.abs().dot(Vec3::Y) > 0.999 {
        Vec3::X
    } else {
        Vec3::Y
    };
    let t1 = normal.cross(reference).normalize_or_zero();
    let t2 = normal.cross(t1).normalize_or_zero();
    Quat::from_mat3(&Mat3::from_cols(t1, t2, normal))
}
/// A frame's Gaussian list. Constructed fresh every frame — there is no
/// training (PLAN.md, "Core design") — so this is a plain growable
/// collection.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct SplatBatch {
    splats: Vec<Splat>,
}
impl SplatBatch {
    pub fn new() -> Self {
        Self::default()
    }
    pub fn with_capacity(capacity: usize) -> Self {
        Self {
            splats: Vec::with_capacity(capacity),
        }
    }
    pub fn len(&self) -> usize {
        self.splats.len()
    }
    pub fn is_empty(&self) -> bool {
        self.splats.is_empty()
    }
    pub fn push(&mut self, splat: Splat) {
        self.splats.push(splat);
    }
    pub fn clear(&mut self) {
        self.splats.clear();
    }
    pub fn iter(&self) -> impl Iterator<Item = &Splat> {
        self.splats.iter()
    }
    /// The centroid of the splat means (world space). Useful for aiming
    /// debug views at the subject.
    pub fn centroid(&self) -> Option<Vec3> {
        if self.splats.is_empty() {
            return None;
        }
        let sum = self.splats.iter().fold(Vec3::ZERO, |acc, s| acc + s.mean);
        Some(sum / self.splats.len() as f32)
    }
}
impl<'a> IntoIterator for &'a SplatBatch {
    type Item = &'a Splat;
    type IntoIter = std::slice::Iter<'a, Splat>;
    fn into_iter(self) -> Self::IntoIter {
        self.splats.iter()
    }
}
#[cfg(test)]
mod tests {
    use super::*;
    use glam::vec3;
    fn intrinsics() -> CameraIntrinsics {
        CameraIntrinsics {
            fx: 500.0,
            fy: 500.0,
            cx: 256.0,
            cy: 212.0,
        }
    }
    #[test]
    fn surfel_footprint_is_depth_over_focal() {
        // PLAN.md: two tangential scales = depth * pixel_size / focal.
        let s = Splat::surfel(
            vec3(0.0, 0.0, 2.0),
            vec3(0.0, 0.0, -1.0),
            2.0,
            &intrinsics(),
            SURFEL_THICKNESS_M,
            [1.0, 0.0, 0.0],
            0.95,
        );
        assert!((s.scale.x - 2.0 / 500.0).abs() < 1e-6);
        assert!((s.scale.y - 2.0 / 500.0).abs() < 1e-6);
        // Thickness along the normal, not the footprint.
        assert!((s.scale.z - SURFEL_THICKNESS_M).abs() < 1e-6);
        assert!((s.normal() - vec3(0.0, 0.0, -1.0)).length() < 1e-5);
        assert_eq!(s.opacity, 0.95);
    }
    #[test]
    fn surfel_rotation_maps_local_z_to_normal() {
        for normal in [
            vec3(0.0, 0.0, -1.0),
            vec3(0.0, 1.0, 0.0),
            vec3(1.0, 0.0, 0.0),
            vec3(0.707, 0.707, 0.0),
            vec3(0.577, 0.577, 0.577),
            vec3(0.0, -1.0, 0.0), // near the world-up fallback axis
        ] {
            let n = normal.normalize();
            let q = surfel_rotation(n);
            // Local Z maps to the normal; the quaternion is a proper
            // rotation (orthonormal basis).
            assert!((q * Vec3::Z - n).length() < 1e-5, "normal {n:?}");
            assert!((q.length() - 1.0).abs() < 1e-5);
        }
    }
    #[test]
    fn surfel_opacity_is_clamped() {
        let s = Splat::surfel(
            Vec3::ZERO,
            vec3(0.0, 0.0, -1.0),
            1.0,
            &intrinsics(),
            SURFEL_THICKNESS_M,
            [0.0; 3],
            1.4,
        );
        assert_eq!(s.opacity, 1.0);
    }
    #[test]
    fn batch_basics() {
        let mut batch = SplatBatch::with_capacity(4);
        assert!(batch.is_empty());
        assert_eq!(batch.centroid(), None);
        batch.push(Splat::surfel(
            vec3(1.0, 0.0, 0.0),
            vec3(0.0, 0.0, -1.0),
            2.0,
            &intrinsics(),
            SURFEL_THICKNESS_M,
            [1.0; 3],
            0.9,
        ));
        batch.push(Splat::surfel(
            vec3(0.0, 3.0, 0.0),
            vec3(0.0, 0.0, -1.0),
            2.0,
            &intrinsics(),
            SURFEL_THICKNESS_M,
            [1.0; 3],
            0.9,
        ));
        assert_eq!(batch.len(), 2);
        assert_eq!(batch.centroid(), Some(vec3(0.5, 1.5, 0.0)));
        batch.clear();
        assert!(batch.is_empty());
    }
}

scripts/sync-tracel-lock.awk +66 -0

diff --git a/scripts/sync-tracel-lock.awk b/scripts/sync-tracel-lock.awk
new file mode 100644
index 0000000..c0edfc3
@@ -0,0 +1,66 @@
# Sync the tracel-ai git dependencies (burn, cubecl, cubek, tracel-llvm)
# in this workspace's Cargo.lock to the exact revisions recorded in
# Brush's committed Cargo.lock.
#
# Why: brush-render (a git dependency) requires burn from
# `https://github.com/tracel-ai/burn?branch=main`, and a branch
# requirement resolves to that branch's *current* HEAD on any fresh
# resolution. burn's `main` regularly drifts ahead of Brush and breaks
# its build (it did once during build-order step 1). Cargo honours
# whatever revision the lockfile already pins for a branch, though — the
# same mechanism Brush's own CI relies on — so we keep our lock pinned
# to the burn revisions Brush was actually tested with.
#
# Usage:
#   curl -s <brush-url>/Cargo.lock -o /tmp/brush.lock
#   awk -f scripts/sync-tracel-lock.awk /tmp/brush.lock Cargo.lock > Cargo.lock.new
#   grep -c dd... # sanity: no unwanted HEAD revisions remain
#   mv Cargo.lock.new Cargo.lock
#
# Packages present in our lock but absent from Brush's are pruned (they
# are crates that only exist at burn revisions newer than Brush's).
#
# Never run `cargo update -p burn …` blindly — it re-resolves the branch
# to HEAD and undoes this pin.
FNR == NR {
    # Pass 1: collect Brush's known-good resolutions by package name.
    if ($0 == "[[package]]") { inb = 1; bname = ""; bver = ""; bsrc = ""; next }
    if (inb) {
        if ($0 ~ /^name = /) { bname = $0 }
        if ($0 ~ /^version = /) { bver = $0 }
        if ($0 ~ /^source = /) { bsrc = $0 }
        if ($0 == "") {
            if (bsrc ~ /tracel-ai/ && bname != "") {
                map[bname] = bver "\n" bsrc
            }
            inb = 0
        }
    }
    next
}
# Pass 2: rewrite our lockfile block by block.
{
    if ($0 != "[[package]]") { print; next }
    n = 1; buf[1] = $0
    name = ""; veridx = 0; srcidx = 0
    while ((getline line) > 0) {
        buf[++n] = line
        if (line == "") break
        if (line ~ /^name = /) { name = line }
        if (line ~ /^version = /) { veridx = n }
        if (line ~ /^source = /) { srcidx = n }
    }
    if (srcidx > 0 && buf[srcidx] ~ /tracel-ai/) {
        if (name in map) {
            split(map[name], repl, "\n")
            buf[veridx] = repl[1]
            buf[srcidx] = repl[2]
            for (i = 1; i <= n; i++) print buf[i]
        }
        # else: crate does not exist at Brush's revision — prune the block.
    } else {
        for (i = 1; i <= n; i++) print buf[i]
    }
}

sink/Cargo.toml +13 -0

diff --git a/sink/Cargo.toml b/sink/Cargo.toml
index 423653e..add88f6 100644
@@ -10,3 +10,16 @@ name = "kinect-view"
path = "src/main.rs"
[dependencies]
anyhow.workspace = true
eframe.workspace = true
egui.workspace = true
env_logger.workspace = true
glam.workspace = true
log.workspace = true
replay.workspace = true
render.workspace = true
rig.workspace = true
[dev-dependencies]
egui_kittest.workspace = true
image.workspace = true

sink/src/app.rs +27 -0

diff --git a/sink/src/app.rs b/sink/src/app.rs
new file mode 100644
index 0000000..5fd6f5e
@@ -0,0 +1,27 @@
//! The eframe shell around [`ViewerModel`] — as thin as possible, since
//! it cannot be exercised headlessly. All UI logic lives in the model.
//!
//! eframe 0.37's [`eframe::App`] splits work across two methods: `logic`
//! runs before each pass (no painting allowed — but repaint requests
//! belong here), and `ui` receives the root [`egui::Ui`], exactly the
//! interface [`ViewerModel::ui`] already speaks.
use eframe::egui;
use crate::model::ViewerModel;
/// The kinect-view application.
pub struct KinectViewApp {
    pub model: ViewerModel,
}
impl eframe::App for KinectViewApp {
    fn logic(&mut self, ctx: &egui::Context, _frame: &mut eframe::Frame) {
        // The pipeline renders every UI frame; keep the repaints coming.
        ctx.request_repaint();
    }
    fn ui(&mut self, ui: &mut egui::Ui, _frame: &mut eframe::Frame) {
        self.model.ui(ui);
    }
}

sink/src/lib.rs +13 -2

diff --git a/sink/src/lib.rs b/sink/src/lib.rs
index 0c525f5..f4c5e16 100644
@@ -3,10 +3,21 @@
//! Pipeline role (PLAN.md, "Pipeline per frame" — sinks):
//!
//! * an egui UI for viewing the virtual-camera image and steering the
//!   virtual camera
//!   virtual camera — [`ViewerModel`] plus the thin eframe shell
//!   [`KinectViewApp`]
//! * on Linux, a v4l2loopback virtual webcam so Teams/Slack can pick the
//!   output up as a regular camera
//!   output up as a regular camera (build-order step 4)
//! * on Windows there is no built-in sink; capture the UI window via OBS
//!   yourself
//!
//! The `kinect-view` binary in this crate is the application entry point.
//!
//! Testing: the UI logic lives in [`ViewerModel`], which drives the whole
//! pipeline (source → render → texture) from `ui(&Context)` and is
//! exercised headlessly with `egui-kittest` — see `tests/ui.rs`.
pub mod app;
pub mod model;
pub use app::KinectViewApp;
pub use model::{FrameStats, ViewerModel, ViewerState};

sink/src/main.rs +26 -6

diff --git a/sink/src/main.rs b/sink/src/main.rs
index 30481e6..5f535c6 100644
@@ -1,9 +1,29 @@
//! `kinect-view` application entry point (scaffolding).
//! `kinect-view` application entry point.
//!
//! The egui application shell arrives with build-order step 1 in PLAN.md
//! ("replay + render + UI with synthetic data"); this placeholder keeps
//! the workspace runnable from the first commit.
//! Runs the pipeline on the synthetic development dataset (build-order
//! step 1); live Kinect capture replaces the source in step 3.
fn main() {
    println!("kinect-view — application shell lands with PLAN.md build-order step 1.");
use eframe::egui;
fn main() -> eframe::Result<()> {
    env_logger::init();
    let rig = replay::synth::dev_rig();
    let source = replay::synth::dev_source(30.0);
    let renderer = render::SplatRenderer::default();
    let model = sink::ViewerModel::new(rig, Box::new(source), renderer);
    let app = sink::KinectViewApp { model };
    let options = eframe::NativeOptions {
        viewport: egui::ViewportBuilder::default()
            .with_title("kinect-view")
            .with_inner_size([1024.0, 768.0]),
        ..Default::default()
    };
    eframe::run_native(
        "kinect-view",
        options,
        Box::new(move |_cc| Ok(Box::new(app))),
    )
}

sink/src/model.rs +190 -0

diff --git a/sink/src/model.rs b/sink/src/model.rs
new file mode 100644
index 0000000..9e90469
@@ -0,0 +1,190 @@
//! The viewer UI: virtual-camera viewfinder and steering controls.
//!
//! [`ViewerModel`] owns the whole pipeline for the UI — the frame source,
//! the Brush renderer, the latest frame's splats — and does its work
//! inside [`ViewerModel::ui`], which is called once per egui frame (by
//! the eframe shell in [`crate::app`] or the kittest harness in tests).
//! Everything is headless-testable: no windowing is involved here.
use std::time::Instant;
use anyhow::Result;
use egui::{ColorImage, TextureHandle, TextureOptions};
use render::{RenderedImage, SplatRenderer};
use rig::{Camera, FrameSource, Rig, SplatBatch};
/// The viewer's interactive state, independent of egui internals so it
/// can be asserted on directly in tests.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct ViewerState {
    /// Virtual camera position between the side cameras:
    /// `0.0` = left camera, `1.0` = right camera.
    pub t: f32,
    /// Whether the frame source is being played back.
    pub playing: bool,
}
/// What the last pipeline pass achieved, for the stats readout and tests.
#[derive(Debug, Clone, Copy, Default, PartialEq)]
pub struct FrameStats {
    /// Frames pulled from the source so far.
    pub frame_index: u64,
    /// Splats in the latest frame's batch.
    pub splats: usize,
    /// Splats visible in the latest virtual-camera render.
    pub visible_splats: u32,
    /// Wall-clock time of the latest render, milliseconds.
    pub render_ms: f32,
    /// The latest frame's source timestamp, seconds.
    pub timestamp_secs: f64,
}
/// Drives the pipeline and builds the viewer UI.
pub struct ViewerModel {
    rig: Rig,
    source: Box<dyn FrameSource>,
    renderer: SplatRenderer,
    state: ViewerState,
    /// The latest frame's splats, re-rendered whenever `t` changes.
    current: Option<SplatBatch>,
    last_image: Option<RenderedImage>,
    texture: Option<TextureHandle>,
    stats: FrameStats,
}
impl ViewerModel {
    pub fn new(rig: Rig, source: Box<dyn FrameSource>, renderer: SplatRenderer) -> Self {
        Self {
            rig,
            source,
            renderer,
            state: ViewerState {
                t: 0.5,
                playing: true,
            },
            current: None,
            last_image: None,
            texture: None,
            stats: FrameStats::default(),
        }
    }
    /// The interactive state.
    pub fn state(&self) -> ViewerState {
        self.state
    }
    /// The latest pipeline statistics.
    pub fn stats(&self) -> FrameStats {
        self.stats
    }
    /// The latest rendered virtual-camera image.
    pub fn last_image(&self) -> Option<&RenderedImage> {
        self.last_image.as_ref()
    }
    /// Steer the virtual camera; `t` is clamped to `[0, 1]`.
    pub fn set_t(&mut self, t: f32) {
        self.state.t = t.clamp(0.0, 1.0);
    }
    /// The virtual camera the model currently renders from: a webcam
    /// (the rig's left camera's intrinsics and resolution) posed at
    /// `t` between the side cameras.
    pub fn virtual_camera(&self) -> Camera {
        self.rig.virtual_camera(
            self.state.t,
            self.rig.left.intrinsics,
            self.rig.left.resolution,
        )
    }
    fn toggle_playing(&mut self) {
        self.state.playing = !self.state.playing;
    }
    /// One pipeline pass: advance the source if playing, then (re)render
    /// the latest splats from the current virtual camera. Re-rendering
    /// while paused is what makes `t` scrubbing live.
    fn update_pipeline(&mut self) -> Result<()> {
        if self.state.playing {
            let frame = self.source.next_frame()?;
            self.stats.frame_index += 1;
            self.stats.timestamp_secs = frame.timestamp_secs;
            self.current = Some(frame.splats);
        }
        let Some(batch) = self.current.as_ref() else {
            return Ok(()); // no frame yet (e.g. paused before first pass)
        };
        let camera = self.virtual_camera();
        let start = Instant::now();
        let (image, render_stats) = self.renderer.render(batch, &camera)?;
        self.stats.render_ms = start.elapsed().as_secs_f32() * 1e3;
        self.stats.splats = render_stats.total_splats;
        self.stats.visible_splats = render_stats.visible_splats;
        self.last_image = Some(image);
        Ok(())
    }
    /// Run one pipeline pass and build the UI. Called once per egui
    /// frame with the root [`egui::Ui`] (by the eframe shell in
    /// [`crate::app`] or the kittest harness in tests).
    pub fn ui(&mut self, ui: &mut egui::Ui) {
        if let Err(error) = self.update_pipeline() {
            // Keep the last good frame on screen; the source may recover.
            log::error!("pipeline error: {error:#}");
        }
        egui::Panel::bottom("controls").show(ui, |ui| {
            ui.horizontal(|ui| {
                let play_label = if self.state.playing { "Pause" } else { "Play" };
                if ui.button(play_label).clicked() {
                    self.toggle_playing();
                }
                ui.add(
                    egui::Slider::new(&mut self.state.t, 0.0..=1.0).text("Virtual camera position"),
                );
                ui.label(format!("{:.0}%", self.state.t * 100.0));
                ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
                    ui.label(format!(
                        "frame {} · {} splats · {} visible · {:.1} ms",
                        self.stats.frame_index,
                        self.stats.splats,
                        self.stats.visible_splats,
                        self.stats.render_ms,
                    ));
                });
            });
        });
        egui::CentralPanel::default_margins().show(ui, |ui| {
            let Some(image) = self.last_image.as_ref() else {
                ui.label("Waiting for the first frame…");
                return;
            };
            // Brush's packed output is premultiplied over the background.
            let color_image = ColorImage::from_rgba_premultiplied(
                [image.width as usize, image.height as usize],
                &image.rgba,
            );
            if let Some(texture) = self.texture.as_mut() {
                texture.set(color_image, TextureOptions::LINEAR);
            } else {
                self.texture = Some(ui.ctx().load_texture(
                    "virtual_camera",
                    color_image,
                    TextureOptions::LINEAR,
                ));
            }
            if let Some(texture) = &self.texture {
                ui.add(
                    egui::Image::from_texture(texture)
                        .maintain_aspect_ratio(true)
                        .max_size(ui.available_size()),
                );
            }
        });
    }
}

sink/tests/ui.rs +202 -0

diff --git a/sink/tests/ui.rs b/sink/tests/ui.rs
new file mode 100644
index 0000000..6c72b7b
@@ -0,0 +1,202 @@
//! Headless UI tests for the viewer, driven with `egui-kittest`.
//!
//! These exercise the real UI code path: each harness step runs
//! [`sink::ViewerModel::ui`], which pulls the next synthetic frame and
//! renders it from the virtual camera via Brush (on software Vulkan in
//! CI). Interactions go through the actual widgets — the play/pause
//! button and the steering slider.
use egui::{Event, pos2, vec2};
use egui_kittest::{Harness, kittest};
use glam::uvec2;
use kittest::Queryable;
use render::RenderedImage;
use replay::synth;
use sink::ViewerModel;
/// Small but complete: the dev rig and scene at low resolutions, so every
/// pipeline stage (synth source → surfels → Brush render → texture)
/// runs for real without slowing the suite down.
fn small_model() -> ViewerModel {
    let rig = synth::dev_rig_with_resolutions(uvec2(64, 53), uvec2(80, 60));
    let source = synth::dev_source_with_center_camera(rig.center, 30.0);
    ViewerModel::new(rig, Box::new(source), render::SplatRenderer::default())
}
fn harness_for<'a>(model: ViewerModel, wgpu: bool) -> Harness<'a, ViewerModel> {
    let mut builder = Harness::builder().with_size(vec2(400.0, 300.0));
    if wgpu {
        builder = builder.wgpu();
    }
    builder.build_ui_state(|ui, model| model.ui(ui), model)
}
#[test]
fn pipeline_runs_and_stats_update() {
    let harness = harness_for(small_model(), false);
    // The harness runs the ui closure at build time; that alone must have
    // pulled frames and rendered them.
    let model = harness.state();
    assert!(model.state().playing);
    assert!(model.stats().frame_index >= 1);
    assert!(model.stats().splats > 0);
    assert!(model.stats().visible_splats > 0);
    assert!(model.stats().render_ms >= 0.0);
    // The virtual camera renders at webcam resolution: 80x60.
    let image = model.last_image().expect("a rendered image");
    assert_eq!((image.width, image.height), (80, 60));
    assert_eq!(image.rgba.len(), 80 * 60 * 4);
}
#[test]
fn play_pause_button_toggles_playback() {
    let mut harness = harness_for(small_model(), false);
    harness.run();
    harness.get_by_label("Pause").click();
    harness.run();
    assert!(!harness.state().state().playing, "Pause must stop playback");
    let frame_paused = harness.state().stats().frame_index;
    // While paused, further harness steps must not advance the source…
    harness.run_steps(3);
    assert_eq!(harness.state().stats().frame_index, frame_paused);
    // …but they still re-render (scrubbing the slider stays live).
    assert!(harness.state().last_image().is_some());
    // And the button label flips back.
    harness.get_by_label("Play").click();
    harness.run();
    assert!(harness.state().state().playing);
    assert!(harness.state().stats().frame_index > frame_paused);
}
#[test]
fn slider_steers_virtual_camera() {
    let mut harness = harness_for(small_model(), false);
    harness.run();
    assert!((harness.state().state().t - 0.5).abs() < 1e-6);
    // Drag the slider handle from the middle to near the right end. The
    // slider query must be role-scoped: egui sliders nest a spin-button
    // with the same label.
    let slider = harness
        .query(
            kittest::by()
                .role(egui::accesskit::Role::Slider)
                .label("Virtual camera position"),
        )
        .expect("steering slider");
    let rect = slider.rect();
    let start = rect.center();
    let end = pos2(rect.right() - rect.width() * 0.02, rect.center().y);
    harness.drag_at(start);
    harness.event(Event::PointerMoved(end));
    harness.drop_at(end);
    harness.run();
    let model = harness.state();
    let t = model.state().t;
    assert!(
        t > 0.7,
        "dragging the slider right must steer right, t = {t}"
    );
    // The virtual camera moved along the baseline with it (the rig's
    // side cameras sit at ±0.4 m).
    let position = model.virtual_camera().pose.position.x;
    assert!(position > 0.0, "virtual camera at x = {position}");
}
#[test]
fn steering_rerenders_the_virtual_view() {
    // End-to-end: steering the slider must change the rendered image.
    // (The rig converges on the subject, so the subject stays framed at
    // any t — the *backdrop* parallaxes. Both are asserted.)
    let mut harness = harness_for(small_model(), false);
    harness.run();
    // Pause playback so the subject's sway cannot contaminate the
    // comparison.
    harness.get_by_label("Pause").click();
    harness.run();
    let image_at = |harness: &mut Harness<ViewerModel>, t: f32| {
        harness.state_mut().set_t(t);
        harness.run();
        harness.state().last_image().expect("rendered").clone()
    };
    let left = image_at(&mut harness, 0.0);
    let right = image_at(&mut harness, 1.0);
    // The subject stays framed from both ends (converged rig)…
    let left_head = skin_centroid_x(&left).expect("subject from left");
    let right_head = skin_centroid_x(&right).expect("subject from right");
    assert!(
        (left_head - right_head).abs() < 4.0,
        "subject must stay framed: x={left_head} vs x={right_head}"
    );
    // …while the backdrop parallaxes: a good fraction of the frame
    // changes across the baseline.
    let mut changed = 0;
    let mut total = 0;
    let (left_px, _) = left.rgba.as_chunks::<4>();
    let (right_px, _) = right.rgba.as_chunks::<4>();
    for (a, b) in left_px.iter().zip(right_px) {
        total += 1;
        let diff = (a[0] as i32 - b[0] as i32)
            .abs()
            .max((a[1] as i32 - b[1] as i32).abs());
        if diff > 25 {
            changed += 1;
        }
    }
    assert!(
        changed * 10 > total,
        "only {changed}/{total} pixels changed between t=0 and t=1"
    );
}
#[test]
fn ui_paints_headlessly() {
    // Full paint pass on the wgpu renderer (software Vulkan): the UI must
    // produce a non-empty, non-uniform image.
    let mut harness = harness_for(small_model(), true);
    harness.run();
    harness.set_render_every_step(true);
    let image = harness.render().expect("wgpu render");
    assert!(image.width() > 0 && image.height() > 0);
    // More than one distinct colour: the viewfinder really painted.
    let colors: std::collections::HashSet<_> = image.pixels().map(|p| p.0).take(1 << 16).collect();
    assert!(
        colors.len() > 8,
        "ui rendered only {} distinct colors",
        colors.len()
    );
}
/// The mean x of pixels close to the synthetic skin colour, or `None` if
/// the subject is not visible.
fn skin_centroid_x(image: &RenderedImage) -> Option<f32> {
    const SKIN: [f32; 3] = [0.85, 0.62, 0.52];
    let mut sum = 0.0;
    let mut count = 0.0;
    for y in 0..image.height {
        for x in 0..image.width {
            let [r, g, b, a] = image.pixel(x, y);
            if a == 0 {
                continue;
            }
            let px = [r as f32 / 255.0, g as f32 / 255.0, b as f32 / 255.0];
            let dist = (px[0] - SKIN[0]).abs() + (px[1] - SKIN[1]).abs() + (px[2] - SKIN[2]).abs();
            if dist < 0.2 {
                sum += x as f32;
                count += 1.0;
            }
        }
    }
    (count > 5.0).then(|| sum / count)
}