Skip to content

← The Lab

Apple's Depth Pro 2 makes steady, metric depth maps from ordinary video, at full resolution

Depth Pro 2 estimates the distance of every pixel in meters from a single photo, a whole video or a live camera feed, at the source's full resolution, with depth that stays steady from frame to frame. For video people, that means cleaner depth passes for fog, relighting, depth of field and 2D-to-3D moves. The weights are research-only, though, so it's not for paid client work.

What Apple released. The code went public on GitHub on Nov 19 (initial commit 1:10 AM MST), and the arXiv paper was linked that evening. There are four sizes: S (24M parameters, 97 MB), B (97M), L (334M) and H+ (872M, 3.5 GB, the default), plus a shared field-of-view estimator that predicts the focal length when your footage doesn't record it. One command, uv run depth-pro-2 predict -i clip.mp4 -o out/, runs on an NVIDIA GPU, an Apple silicon GPU or the CPU. Per Apple's figures as quoted by early posters, S processes 1080p video at about 53 fps and H+ at about 8 fps on one B200.

Why it's useful in post. Older single-frame depth models flicker when you run them on video, which makes their passes hard to use for anything moving. Depth Pro 2 processes video in overlapping 16-frame windows (or one frame at a time from the 15 before it, for live feeds) and gives one focal length per clip, so the depth holds steady. Per the project page, it beats earlier single- and multi-frame methods on both edge sharpness and frame-to-frame jitter (Apple's own benchmark, on the Sintel and Spring datasets). Add --save-png16 and you get 16-bit inverse-depth PNGs per frame for Nuke (read as Raw) or Blender (Non-Color). That's the pass you'd use for depth fog, a fake rack focus, relighting or a parallax push-in. --save-npz gives true depth in meters for 3D work.

On a Mac. There's a minimal live-camera demo app on Core ML (Apple silicon, macOS 15+), plus exported Core ML weights. Early community benchmarks compare Core ML, PyTorch and MLX on an M5 Pro Mac mini. Community ports are appearing too (ONNX and a first ComfyUI node), but they're a day old and unofficial.

The catch: license. The code has Apple's own license, and the weights are licensed 'exclusively for Research Purposes', defined as non-commercial research and academic work. Some viral posts call it 'free to download', and it is, but that doesn't make it free to use on a client job. For commercial depth passes, stick with tools whose terms allow it, and use Depth Pro 2 to see what's coming.

Signals: research release plus a big untracked-curator post (@_akhaliq, Nov 19 6:16 PM MST, about 600 likes), Hugging Face's @pcuenq, and tracked curator @RoundtableSpace (Nov 20 5:15 AM MST). We didn't see an official Apple post or a GitHub Trending slot, so it isn't HOT by our rule.

Skills & repos in our directory

More from the Lab

Topics: Open-weight video & audio models