Re-align the plan with the codebase after step 1

GLM Agent committed on 2026-10-25 at 07:30

- surfel construction lives in rig (shared); splatgen owns depth
  preprocessing and batch assembly
- the rig's convergence keeps the subject framed (zero parallax at the
  convergence distance, background parallaxes) — observed and verified
  in step 1
- step 2 spelled out: replay defines the recording format, and the
  synthetic source becomes a raw depth frame emitter so the dev/CI path
  exercises the real depth preprocessing

Browse files at this commit

PLAN.md +15 -4

diff --git a/PLAN.md b/PLAN.md
index 6a6c6d2..ff1b945 100644
@@ -29,6 +29,14 @@ Smoothness comes from footprint-sized anisotropic splats:
- opacity ~0.9–0.99
- front-to-back alpha blend
The surfel constructor itself is shared (`rig::Splat::surfel`); `splatgen`
assembles the batch from a depth frame around it (pipeline steps 2–3).
The rig converges on the subject, so at the convergence distance the
subject stays framed as the virtual camera moves between the side
cameras — zero parallax on the subject, parallax only on the background.
Verified end-to-end in step 1; this is the desired virtual-camera look.
**Occluded far side of the face — the MVP uses an ellipsoid proxy:** fit a
head/shoulders ellipsoid per frame to the Kinect head point cluster;
side-camera pixels are projected onto the proxy for position and recolored
@@ -64,10 +72,10 @@ The workspace root of this repository is the `kinect-view/` tree:
```
kinect-view/
├── rig/        # shared types: poses, intrinsics, splat batches, frame sources
├── rig/        # shared types: poses, intrinsics, surfel construction, frame sources
├── capture/    # kinect-one, webcams, frame sync
├── replay/     # recorded/synthetic datasets — dev & CI without hardware
├── splatgen/   # depth filter, normals, surfels, ellipsoid proxy fill
├── replay/     # recorded / synthetic depth datasets — dev & CI without hardware
├── splatgen/   # depth preprocess (filter, fliers, culling), normals, batch assembly
├── render/     # Brush integration, virtual camera
└── sink/       # egui UI + v4l2loopback (Linux)
```
@@ -105,7 +113,10 @@ loopback output) join the same arg surface.
## Build order
- [x] 1. replay + render + UI with synthetic data
- [ ] 2. splatgen on recorded Kinect frames
- [ ] 2. splatgen on recorded Kinect frames — `replay` defines the
      recording format (synthetic recordings for CI); the synthetic
      source is refactored to emit raw depth frames like a Kinect, so
      the dev/CI path runs through the real depth preprocessing
- [ ] 3. Live Kinect capture + ellipsoid proxy recoloring from side cameras
- [ ] 4. v4l2loopback sink
- [ ] 5. Polish: cross-camera color calibration, WB-lock webcams, seam tuning