cd ~/
Eye TrackingOpticsSpecklePatentsMagic Leap

Speckle-Based Eye Tracking: A Novel Approach

Using laser speckle patterns for high-precision eye tracking - the physics, implementation, and path to a patent.

Evyatar Bluzer
3 min read

Shine coherent light - a laser - at a rough surface and the scattered light forms an interference pattern of bright and dark spots called "speckle." We've been building an unconventional eye tracker around that effect, and it's now mature enough to file a patent.

What is Speckle?

The pattern behaves like a fingerprint of the surface that produced it: unique to that surface, sensitive to viewing angle (it changes with observer position), and full of fine-grained, high spatial frequency structure.

Speckle on the Eye

The human cornea and iris have surface texture that creates speckle when illuminated with coherent NIR light. The property that makes this useful: as the eye rotates, the speckle pattern translates. Track the pattern's movement and you can track eye rotation with very high precision.

Speckle eye tracking benchLeft to right: a VCSEL box, an arrow to a circle for the eye with a small rotation arc, an arrow labeled speckle to a camera box, and below the camera two frames of the same dot pattern, the second shifted right with a highlighted arrow marking the offset; formulas for offset and rotation sit under the frames. VCSEL coherent NIR, eye-safe eye rotates eye cornea and iris texture scatter it speckle high-speed camera 3 μm pixels, above 200 Hz, NIR frame n-1 frame n offset = argmax correlation(frame n, frame n-1) rotation = f(offset, geometry) sub-pixel correlation gives sub-degree rotation bench prototype: 0.1 degree, slow movements only tears and blinks decorrelate the pattern
A VCSEL lights the cornea, a fast camera images the speckle, and the tracker measures how far the pattern moved between consecutive frames; that offset maps to eye rotation.

Implementation Concept

VCSEL Illumination → Corneal Reflection → Speckle Pattern →
High-Speed Camera → Pattern Correlation → Gaze Estimation

Illumination

Vertical-Cavity Surface-Emitting Lasers (VCSELs) are coherent enough for speckle generation, eye-safe at low power, and compact and efficient.

Imaging

The speckle pattern gets captured on a specialized camera: pixel size of 3μm or smaller to resolve the speckle, frame rate above 200Hz to keep up with fast eye movements, and NIR sensitivity matched to the VCSEL wavelength.

Pattern Tracking

Cross-correlation between consecutive frames:

offset = argmax(correlation(frame_n, frame_n-1))
rotation = f(offset, geometry)

Sub-pixel correlation gives sub-degree rotation precision.

Advantages Over Traditional Eye Tracking

The glint-and-pupil pipeline I described last summer is limited by image resolution; speckle provides sub-pixel precision. Pattern correlation is also computationally simpler than feature detection. And the method is less sensitive to pupil dilation and partial occlusion.

Challenges

Speckle decorrelation is the big one. If the surface changes (tears, blinking), the pattern changes, so decorrelation events have to be detected and handled. The hardware requirements - a coherent source plus a high-resolution sensor - add cost and power. And putting coherent radiation on the eye requires careful safety analysis; we must meet IEC 62471 Class 1.

Patent Strategy

We're filing on three things: the method of eye tracking using speckle correlation, an apparatus with the specific VCSEL + sensor configuration, and decorrelation detection and recovery. The combination of eye safety constraints, speckle imaging requirements, and correlation algorithms is novel.

Path Forward

Current status: a bench prototype showing 0.1 degree accuracy, limited to slow movements. Getting it into a product needs faster correlation (a GPU or DSP implementation), reliable decorrelation handling, and integration with the display optics, the same packaging problem the conventional eye camera has. This may not make the V1 product, but it's promising for future iterations.

[Patent granted 2020: US10948981 "Method and System for Eye Tracking Using Speckle Patterns"]

Comments