cd ~/
HardwareSpecificationDecision MakingLeadershipMagic Leap

Lessons from Spec Freeze: Hardware-Software Co-Design

What I learned from our first major hardware spec freeze - the art of committing to specifications when uncertainty remains.

Evyatar Bluzer
3 min read

We've hit spec freeze for the perception system, on the December date I committed to in the architecture post. Sensor selections, interfaces, and the hardware architecture are now locked; software can still evolve, but the physical substrate is committed. Terrifying and necessary in roughly equal parts.

Why freeze hardware while the software is still immature?

Because the lead times leave no choice. Custom sensor development runs 18-24 months, ASIC design 12-18 months, and manufacturing tooling another 6-12. Software lives on a different clock entirely: algorithm improvements land in days to weeks, integration in weeks to months. That mismatch means we commit the hardware long before the software that will run on it is anywhere near mature, and then we live with what we picked.

Hardware lead times against software paceFive horizontal range bars on a log time axis; the top three hardware bars sit at the months-to-years end and are highlighted, the two software bars sit at the days-to-weeks end.1 day1 week1 month1 year2 yearstime to change, log scaleCustom sensor developmentASIC designManufacturing toolingSoftware integrationAlgorithm improvements18-24 months12-18 months6-12 monthsweeks to monthsdays to weeksLOCKED AT THE FREEZESTILL EVOLVING
How long each item frozen in December takes to change against the pace software moves at; the three highlighted bars are the commitments that cannot be revisited, with lead times as given in the post.

What We Committed To

Depth is ToF at 320x240, 30Hz, with a 0.3-5m range. Back in March I expected a hybrid, ToF for the room and structured light for near-field hands; we committed to ToF alone, and structured light's only role turned out to be the experiment that de-risked it. Better outdoor performance than structured light drove the choice; the risk is that the low resolution limits hand tracking quality. Tracking cameras are global shutter, 640x480 at 60Hz - global shutter is essential for fast motion, and we pay for it in power relative to rolling shutter. Processing is ARM plus DSP plus custom accelerators, which gives us the best power efficiency for the workloads we know about today, with less flexibility than a pure GPU approach. That is the acceleration I argued in July we could not avoid; the flexibility we gave up for it is the part that could still bite.

How We Made Each Call

For every decision we asked the same four questions. What can't be changed later? What don't we know? What flexibility can we preserve as an escape hatch? And if we're wrong, what does the worst case actually look like? None of the questions are clever, but they kept the arguments concrete.

Mistakes I'm Worried About

Being honest with myself:

  • Depth resolution: 320x240 may not be enough. Software can compensate somewhat through temporal filtering and super-resolution, but there's a floor.
  • No dedicated ML accelerator: We're betting we can do most inference on DSP/GPU. If neural networks become central to tracking, we'll be power-constrained.
  • Eye tracking camera resolution: Balanced against power, but it may limit pupil tracking accuracy in bright conditions.

Key Learnings

Get the algorithms to 80% before the freeze. You need that much fidelity to make informed hardware choices. Build margins too - we specified a 20% power margin and a 30% compute margin, and I have no doubt we'll use all of it. Prototype the risky bets: our structured light experiment is what de-risked the ToF decision. And document assumptions, because every spec has assumptions underneath it, and when one changes you want to know exactly which specs to revisit. That only works if somebody wrote them down.

The last learning is the one that matters most. The spec is a contract. If it's in the spec, it's a commitment, so don't spec what you can't deliver.

2017 will be about delivering on these commitments.

Comments