Camera Calibration: Theory Meets Manufacturing Reality

Calibration is the unglamorous foundation under every perception system, and if you get it wrong, nothing else you build matters.

What Needs Calibrating

Four things. Intrinsic parameters: focal length, principal point, and distortion coefficients for each camera. Extrinsic parameters: the relative pose between every sensor pair in the frozen suite - camera to camera, camera to IMU, camera to depth. Temporal alignment, meaning the latency offsets between sensors that destabilized fusion. And the depth-specific set: ToF phase offsets, systematic depth errors, multi-path correction tables.

The Textbook Approach

The standard recipe is to wave a checkerboard in front of the device, detect corners, and run bundle adjustment. That works fine in a lab with one prototype and an expert operator. It fails completely on a manufacturing line, where you get about 10 seconds per device, operators with minimal training, environmental variation in temperature and lighting, and thousands of devices per day. It's the same manufacturing reality that decided the lens design, applied to software.

Production Calibration Architecture

We're designing a calibration cell:

┌────────────────────────────────────────────┐
│              Calibration Cell              │
│  ┌────────────────────────────────────┐   │
│  │      Controlled Environment        │   │
│  │  - Temperature: 25°C ± 1°C        │   │
│  │  - Humidity: 45% ± 5%             │   │
│  │  - Lighting: 500 lux diffuse      │   │
│  └────────────────────────────────────┘   │
│                                            │
│  ┌──────────────┐    ┌──────────────┐    │
│  │  Reference   │    │    Device    │    │
│  │   Targets    │    │   Under      │    │
│  │  (3D known   │    │   Test       │    │
│  │   geometry)  │    │              │    │
│  └──────────────┘    └──────────────┘    │
│                                            │
│  ┌────────────────────────────────────┐   │
│  │      Automated Motion System       │   │
│  └────────────────────────────────────┘   │
└────────────────────────────────────────────┘

The motion system moves reference targets through the field of view, collecting data across the full calibration manifold automatically.

What We're Learning

Temperature matters more than I expected. Optical systems change with it, so a camera calibrated at 25°C may be off at its 35°C operating temperature, and we need thermal models to bridge that gap. Parameters also drift over time from mechanical settling and thermal cycling, which raises a question we haven't answered yet: how often does a unit need recalibrating?

Then there's the split between per-unit and golden calibration. Some parameters can use nominal values while others must be measured on every unit, and where you draw that boundary largely determines your line throughput. Ideally the device can also recalibrate itself in the field. For a product that has to keep working for years, I don't see how we get away without some form of self-calibration.

How accurate does calibration need to be?

Working backwards from user experience:

• Virtual object placement: ±1cm accuracy
• Requires 6DoF tracking: ±1mm, ±0.1°
• Requires intrinsic calibration: under 0.2 pixel error
• Requires extrinsic calibration: under 0.1° rotation, under 0.5mm translation

These are tight tolerances for a consumer device. Whether we can hold them at thousands of units a day is the question the calibration cell has to answer. (Depth calibration gets its own post next month.)