cd ~/
Depth SensingToFStructured LightComputer VisionMagic Leap

Depth Sensing for MR: ToF vs Structured Light

A deep technical comparison of Time-of-Flight and structured light depth sensing approaches for mixed reality applications.

Evyatar Bluzer
3 min read

Most of this month went into evaluating depth sensing technologies, the second of the three perception problems I named on arrival. Depth is the layer everything else in spatial computing sits on: without accurate, real-time depth data you can't place virtual objects on real surfaces, detect collisions, or track hands. So the choice between the two leading approaches deserves a careful look.

Time-of-Flight (ToF) Basics

ToF sensors measure the time it takes for light to travel to a surface and back:

Distance = (Speed of Light × Time) / 2

In practice we use phase detection rather than direct time measurement: the sensor emits modulated IR light and measures the phase shift of the returned signal. The appeal is that it works at any distance within its range, returns full-frame depth in a single shot, and calibration is relatively simple. In exchange you get multi-path interference in complex scenes and flying pixels at depth discontinuities, and power consumption scales with range.

Structured Light

Structured light projects a known IR pattern and uses triangulation to compute depth from the pattern's deformation. At close range it beats ToF on accuracy, it handles sharp depth edges better, and it can reach very high resolution. But accuracy degrades with distance because the approach is baseline-limited, textureless surfaces give it trouble, and ambient IR - sunlight above all - interferes with the projected pattern.

What Does an MR Headset Actually Need?

Indoor range of 0.3m - 5m. Outdoor capability, which means handling sunlight. A power budget under 500mW for the depth subsystem, and latency under 20ms for responsive interaction.

Neither technology is perfect against that list. ToF wins on outdoor robustness and on consistent accuracy across distances; structured light wins on close-range precision for hand tracking.

ToF versus structured light accuracy over distanceA line chart of depth accuracy against distance from 0.3 to 5 metres: a solid structured-light curve starts high and falls off, a dashed time-of-flight line stays flat, and two highlighted labels under the chart assign the near field to structured light and room scale to ToF.0.31.02.03.04.05.0distance from the headset (m)depth accuracy, higher is betterStructured lighttriangulation is baseline-limited:accuracy degrades with distanceTime of flightphase shift; consistent accuracy across its rangenear field: hand tracking (structured light)room scale, SLAM, sunlight (ToF)
Depth accuracy against distance over the headset's 0.3 m to 5 m range: structured light is sharper up close and falls off because it is baseline-limited, ToF stays flat, and that is the case for the hybrid split. Curve shapes are qualitative; the post gives no accuracy figures.

Hybrid Approaches

The most promising path may be a hybrid system: ToF for room-scale understanding and SLAM, structured light for near-field hand tracking. The integration cost is real - different sensors, different calibrations, different failure modes - and I don't want to understate it.

The open question, and where I'm headed next month, is sensor fusion: how to combine these modalities without doubling power consumption.

Comments