Live3D

Point a camera at a space, whether a phone's browser, an uploaded video, or a drone or CCTV stream, and a 3D model of it grows live in the browser. When recording stops, the server builds a refined model in real meters, saves it to a history, and lets anyone measure it, walk through it, see its surfaces labeled and export it to Blender. Everything runs on one Apple Silicon workstation with no cloud GPU. The pretrained models inside it are used as released. The work is the system around them: verification, refinement, confidence and evaluation.

A live model has to stay one model. As the camera moves, small errors in its estimated position add up, and when it comes back to a place it has already seen, the map has to notice and pull itself straight without tearing. Live3D checks every suspected loop in layers before trusting it, and only then lets a pose-graph solve move the map. The second difficulty is being honest about depth: every vertex of the final model carries a calibrated confidence, so the model can say where it is unsure.

Capture: a browser page that streams over WebSocket and WebRTC, video upload, and RTSP or RTMP streams for drones and CCTV, with lens calibration, timing and rolling-shutter correction, blur filtering, and blurring of faces and number plates. The engine: metric depth from Depth Anything 3, a keyframe-anchored map, layered loop verification and a Sim(3) pose graph in GTSAM, then a final pass in meters with photo-consistency refinement, TSDF fusion, and a perceptual judge that picks the best model to keep. A preview of scene understanding labels surfaces into 29 classes plus "not yet understood". A viewer measures, walks and exports. About 16,900 lines of Python.

The benchmark is 36 datasets, three runs each: 108 runs. The 95th-percentile live delay is under one second on 34 of the 36, from 0.18 to 1.69 seconds, with a median of 0.47. Camera-path error is 0.08% and 0.21% on two TUM sequences, 0.13% on Newer College, 0.52% on EuRoC and 1.05% on KITTI. Surface accuracy of the refined model, as F-score at 5 cm, has a median of 43.1% over the 18 datasets with ground truth, and runs from 2.0% to 70.7%. The project also measured its own limits: metric scale drifts by roughly 13 to 17% on long outdoor walks, and the model from a 12-minute street test, which ran on a third-party walking video, has 14.7% doubled surfaces and leaves 18 to 24% "not yet understood".

A screen recording of a model growing live, which would be the best way to show this and does not exist yet. The paper is a draft and has not been submitted anywhere. Of the seven layers in the plan, capture and reconstruction are complete, scene understanding and asset export are partial, and AI completion, correction and multi-session scale are still plans.