cd ~/
OpticsLens DesignDepth SensingHardwareMagic Leap

Optical Design for Depth Cameras: Lens Trade-offs

Understanding the optical system design choices for depth sensing - from field of view to distortion to manufacturability.

Evyatar Bluzer
3 min read

Every choice in a depth camera's optical system trades something away, and the trades ripple through the entire system. This is the layer where product requirements collide with physics.

Key Parameters

Field of view: wider sees more but reduces angular resolution. F-number: lower gathers more light but has shallower depth of field. Distortion: fisheye lenses are compact but complicate calibration. And MTF - the Modulation Transfer Function - determines sharpness at different frequencies.

Why Is Wide FOV So Hard?

MR devices want wide FOV (>90°) to track objects entering from the periphery, and wide-angle optics fight back at every step. Start with pixel projection: a 120° FOV on a 640-pixel sensor means each pixel spans ~0.2°, which at the 5m indoor range we're holding to works out to 1.7cm per pixel - terrible depth resolution at range. Geometric distortion is next: wide angles require strong barrel distortion or physically large lenses, and distortion can exceed 30% at the edges. Light falloff follows cos⁴θ, so edges receive ~25% of center intensity at 60° off-axis. And chromatic aberration is harder to correct across wide angles, causing color fringing and depth errors.

One pixel of a 120 degree depth camera at 5 mA cone opening upward from a small sensor box to an arc marked 5 m, spanning 120 degrees, with a thin highlighted wedge along its center axis labeled as one pixel covering 1.7 cm at 5 m, and notes at the left edge of the field on distortion and light falloff. 640-pixel depth sensor 120° field of view wanted: over 90°, to see objects entering from the periphery one pixel: ~0.2°, which is 1.7 cm across at 5 m (wedge exaggerated) 5 m the indoor range at the edge of the field: distortion can exceed 30% ~25% of the center's light
A 120 degree field on a 640-pixel sensor drawn out to the 5 m indoor range: one pixel's wedge, exaggerated here, covers 1.7 cm at 5 m, and the edge of the field carries the distortion and light-falloff figures from the paragraph above.

Lens Architectures

Standard Rectilinear

Straight lines stay straight. Familiar, but physically large for wide FOV.

Fisheye (Equidistant/Equisolid)

Compact, very wide FOV, strong distortion. Common in VR tracking cameras.

Freeform

Computer-optimized surfaces that don't follow standard equations. Better performance is possible, but manufacturing and alignment get harder.

Depth-Specific Considerations

For ToF, the lens must transmit NIR efficiently; many standard glass types absorb at 940nm, the wavelength last month's illumination math was done at. For structured light, the lens NA must match the illumination cone. Too narrow = vignetting. Too wide = wasted light. For stereo, matched distortion between cameras simplifies rectification.

Manufacturing Reality

The best optical design means nothing if it can't be manufactured at scale. We're learning to run tolerance sensitivity analysis early in the design, to pick glass with supply chain availability in mind, and to budget for assembly alignment from the start. The system architecture post that ties all of this together comes next month. Until then, our optical engineer's mantra stands: "The best lens is one that can be built."

Comments