cd ~/
Diffractive OpticsEye TrackingWaveguidesAR DisplaysMagic Leap

Diffractive Optics for Eye Imaging

Using diffractive optical elements to enable eye tracking in waveguide-based AR displays - one optical stack doing two jobs.

Evyatar Bluzer
3 min read

Waveguide displays use diffractive optical elements (DOEs) to guide light to the eye. Can similar structures image the eye for tracking? If they can, the overall system gets simpler.

The Waveguide Display Primer

AR displays need to project images while the user sees through to the real world. In a waveguide, light enters at an input coupler (a DOE), propagates via total internal reflection, and exits at an output coupler (another DOE) toward the eye. The user sees a virtual image overlaid on the real world.

The Eye Tracking Challenge

Standard eye tracking adds cameras alongside the display, which brings additional hardware cost and complexity, optical paths that must not interfere with each other, and a hit to form factor.

What If the Display Waveguide Could Also Image the Eye?

The concept: send IR illumination through the waveguide (on a different layer or wavelength), let the eye's reflection couple back into the waveguide, and have it exit at a sensor location.

         ┌─────────────────────────┐
    Eye →│ Waveguide with DOEs     │← Display light in
         │                         │
         └────────────┬────────────┘
                      │
                      ▼
               Eye camera
         (captures light from eye via waveguide)
Waveguide eye imaging layoutA long horizontal slab with a grating at each coupler: display light enters at the right coupler, zigzags left and exits down to a circle for the eye; a highlighted path rises from the eye into the middle coupler, zigzags left and ends at a sensor box on the left edge, with an eye box bracket under the eye. display waveguide, total internal reflection input coupler (DOE) output coupler (DOE) display light in visible, 450-640 nm eye eye box, roughly 20 mm sensor CMOS at the edge NIR at 850 nm, reflected from the eye, couples back in and exits at the sensor leaks run both ways: display light into the eye-track path as haze, eye-track light into the display as ghost images
One waveguide, two jobs: display light enters at the input coupler and leaves the output coupler toward the eye, while 850 nm light reflected from the eye couples back into the waveguide and exits to a sensor at the edge.

DOE Design Considerations

Wavelength Separation

Display uses visible light (450-640nm); eye tracking uses NIR (850nm). DOE response is wavelength-dependent, so the DOEs have to couple display wavelengths efficiently for image delivery, couple NIR efficiently for eye imaging, and keep crosstalk between the two paths low.

Angular Bandwidth

Eye imaging needs to capture across the eye box, roughly 20mm, so the DOE must have sufficient angular bandwidth to collect light from different eye positions.

Efficiency Trade-offs

Every DOE has efficiency losses, and dual-use makes them bite twice. Some display light leaks into the eye-track path as background noise, and some eye-track light leaks into the display as ghost images. Managing those leaks is the design challenge.

Prototype Results

We built a bench prototype from a commercial single-layer waveguide, a custom NIR DOE overlay, and a CMOS sensor at the edge. The pupil is clearly visible, glints from co-propagated IR LEDs are detectable, and image quality is sufficient for basic gaze estimation. Less encouraging: efficiency is lower than a dedicated camera path, crosstalk creates a background haze, and manufacturing the custom DOE is expensive. Last August I wrote that if the display waveguide could do double duty, the optical architecture would get dramatically simpler. The bench says the first half is true and the second half is not yet: the eye is imaged, and nothing about the system got simpler.

Path to Product

For V1 we're going with separate eye tracking cameras: proven, lower risk, and the same call we made on the speckle tracker. For V2 and beyond, integrated diffractive eye imaging could reduce component count, thin the form factor, and share manufacturing steps with the display waveguide. That's worth continuing the R&D for, and worth the patent filings.

[Patent granted 2021: US11237631 "Eye-Imaging Apparatus Using Diffractive Optical Elements"]

Comments