Evaluating Next-Gen Depth Sensors
Systematic evaluation of depth sensors for V2 - from datasheets to real-world performance characterization.
Four vendor sample kits are sitting in the lab right now, and one of them is going to end up in V2. Sensor selection is close enough - the January plan had orders locking within four months - that gut feel no longer cuts it, so we put together a proper evaluation.
Evaluation Framework
Datasheets tell one story and real performance tells another, which is the whole reason this framework exists.
Accuracy
We measure depth error against range at 0.5m, 1m, 2m, 3m, 4m, and 5m, and against angle by comparing the center of the field to the corners. Just as important is separating systematic bias, which is consistent and correctable, from random noise, which we characterize as per-frame variation against a static target.
Resolution
Three concrete questions here: can the sensor resolve 1cm features at 2m, what is the minimum detectable depth difference, and how sharp are the depth discontinuities at edges. Edge quality matters more than it sounds - mushy edges poison everything downstream that reasons about object boundaries.
Robustness
- Sunlight immunity: performance at 10K, 50K, 100K lux
- Multi-path handling: corner geometry test
- Dynamic range: black to white surfaces
- Temperature stability: -10°C to 50°C operating
System Integration
- Power consumption: under various operating modes
- Latency: trigger to data ready
- Interface: USB, MIPI, custom
- Calibration: factory vs field requirements
Test Infrastructure
We built a dedicated sensor evaluation lab for this:
┌──────────────────────────────────────────────────────────┐
│ Controlled Environment │
│ │
│ ┌────────────┐ ┌─────────────┐ ┌────────────┐ │
│ │ Light │ │ Reference │ │ Sensor │ │
│ │ Sources │ │ Targets │ │ Under Test │ │
│ │ (variable) │ │ (calibrated)│ │ │ │
│ └────────────┘ └─────────────┘ └────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────┐ │
│ │ Motion Stage (sub-mm repeatability) │ │
│ └────────────────────────────────────────────────────┘ │
│ │
│ Temperature Chamber: -20°C to 60°C │
└──────────────────────────────────────────────────────────┘
The reference targets are flat plates at calibrated distances, a depth wedge for resolution testing, a multi-reflectance chart running 5% to 95%, and a corner geometry rig for multi-path.
Preliminary Results
Sensor A (ToF, indirect) has the best accuracy of the four (±1% at 2m) at moderate power (400mW), but it struggles with sunlight above 30K lux and shows strong multi-path artifacts.
Sensor B (ToF, direct) trades the other way: good sunlight immunity (still works at 80K lux) and minimal multi-path, paid for with a higher noise floor and high power draw (700mW).
Sensor C (structured light) has the best resolution at close range and the lowest power (250mW), but it fails above 10K lux and tops out at 3m of range, which is roughly the near-field role I assigned structured light back in 2016.
Sensor D (stereo) is passive, so zero depth-specific power, and it performs well outdoors. Its resolution depends on scene texture, though, and it struggles on uniform surfaces.
Which Sensor Wins?
None of them, on all dimensions. So we ranked what we actually need:
- Must have: Outdoor operation (>50K lux survival)
- Must have: Range to 4m (room-scale)
- Should have: Low power (under 500mW)
- Should have: Resolution for hands (under 5mm at 60cm)
- Nice to have: Multi-path robustness
Against that list the current leader is Sensor B, despite the power concerns.
On launch day I wrote that ToF was the right call and that V2 would not be fighting sensor limitations. The first half is holding, since the leader is still ToF. The second half is not: none of the four kits wins on every dimension, and the leader draws 700mW against a 500mW should-have.
Evaluation continues through August. Final decision in September.