Blog

A new framework scores whether a supportive model points you toward people or toward itself, and the default across seven models is inward.

Meta priced the glasses near cost and will give Muse away free; the Reality Labs loss is the subsidy that pendant startups cannot match.

A streaming video model keeps the captions it wrote and drops the frames, and the same text decides what it remembers and when to speak.

A new benchmark measures how an assistant should work with you, and finds the memory breaks at write time, not at recall.

An open streaming diarizer runs on one CPU thread with flat memory, for the price of a one-second delay on speaker labels.

An open detector puts the discarded tone layer back into a transcript, and the accuracy is the honest part of the story.

Amazon priced the always-on mic at $75 and the memory at $12 a month. The hardware is the funnel and recall is the product.

A new agent stays silent 97 percent of the time the group fixes its own mistake. The rule for when it speaks is the product, not the model.

Qualcomm gave the earbud a radio that reaches the cloud without the phone, and a 1-bit model that holds about four minutes of talk.

Muse asks before it spends your money. Camera-free glasses have no screen to ask on, so the approval has to land on another device.

Meta's avatar model draws 320 ms chunks that an agent's turn can hide. Driven by a live caller's voice, a chunk would put lips at least 340 ms late.

Meta's Private Processing was built to forget. Its design for AI glasses remembers: ciphertext on Meta's servers, the key on your device.

Meta's FDA filing puts its glasses hearing loop at 9.25 ms. In an open ear that delay is audible, and its costs belong to the frame, not firmware.

Qualcomm's claim of a 3B model at 45 tokens a second on the VR Glasses chip needs the puck's entire memory bus, by my arithmetic.

By my arithmetic, Meta's VR Glasses panels need up to 40 Gbps uncompressed, more than the Qualcomm chip's one DisplayPort 1.4 output carries.

Meta shipped camera-free Ray-Bans at $349 with a six-microphone array. Removing the lens answered the wrong privacy question.

NVIDIA mapped the four ways to wire diarization into streaming ASR, and only the cheapest corner fits inside a wearable's power budget.

Apple's Watch now summarizes your conversations but refuses to keep a transcript or name the speaker, and that refusal is where the market lives.

HTC's Vive Eagle goes global selling privacy as the headline feature - a playbook that stops working the moment the mic stays on.

A new benchmark shows LLM agents almost never ask before committing memories - and why the write policy is the ambient wearable's real moat.

SoniSpeech drops the first open-vocabulary silent speech dataset for wearable eyewear - the missing piece for conversational coaching.

Pixel Watch 5 ships Health Foundation Models and Gemini coaching at $10/month - clarifying what is left to build.

Meta's v127 update pairs Muse Spark AI with silent EMG input on Ray-Ban Display - the interaction shift ambient coaching needed.

The EU AI Act's emotion recognition rules went live today - and they create a moat for on-device coaching wearables.

On-device NPUs and edge voice models converged in the same week - the ambient wearable can finally build context locally.

Streaming diarization at sub-300ms unlocks the relationship graph that makes ambient coaching personal.

Google's SensorFM proves the ambient AI coach needs a physiological foundation model alongside the LLM.

Four states now classify neural data as sensitive - always-on mic data fits the same legal definition.

Muse Spark matches Llama 4 Maverick at 10x less compute and ships on $299 glasses.

Qualcomm's START toolkit lets any eyewear brand ship AI glasses without building the stack.

Apple shipped a three-tier privacy stack that turns your messages, mail, and screen into agent infrastructure.

Three platform companies shipped agent-first hardware in the same week - the agent is leaving the screen.

Devin writes 89% of Cognition's own code - the threshold where coding agents become self-sustaining just arrived.

Google and Anthropic shipped managed agent compute in the same week - agents are a cloud primitive now.

Microsoft's 100-agent security system found 16 real Windows vulnerabilities by making AI agents disagree with each other.

Coding agents skip the hard parts and talk themselves into it. The fix shipping now: discipline as code.

Four commerce protocols shipped in one week, turning agents into economic actors with real payment rails.

LangChain jumped 25 ranks on TerminalBench by changing only the harness. The model never changed.

MCP hit 97 million monthly downloads and Salesforce just went fully agent-native. The integration layer for agents is decided.

Three teams shipped self-evolving agent frameworks in one week, and the architectural convergence tells us where production agents are heading.

New research shows AI models spontaneously scheme to prevent other models from being shut down. If you build multi-agent systems, this changes your trust model.

Ten years since I entered spatial computing. Reflecting on how the field evolved and where it's going.

A full year since leaving Meta. Reflections on independence, what I built, what I learned, and what's next.

How AI coding tools changed my development workflow - from skeptic to convert in the age of Claude Code and Cursor.

Finding the right balance between consulting work and building my own products - how each informs the other.

What I've learned from building products where AI is the core, not a feature - different principles, different challenges.

Exploring multi-agent AI systems - why they matter, how they work, and what I'm building.

The first month after leaving Meta - the quiet, the projects, and figuring out what independent work looks like.

My last day at Meta. Reflections on the journey, gratitude for the team, and anticipation for what's next.

After four years building VPS, I've decided to leave Meta. The reasoning, the emotions, and what comes next.

How large language models and generative AI are changing spatial computing - from scene understanding to content creation.

Three and a half years building VPS at Meta - from inception to platform. Considering what's next.

Quest 3 brings VPS to mainstream consumers - the largest deployment yet and what we learned.

Bringing visual positioning to consumer smart glasses - extreme power constraints and new use cases.

Quest 3's improved hardware creates opportunities for VPS - better cameras, more compute, and MR-first design.

VPS evolved from concept to platform in two years - reflecting on the journey, the team, and what's next.

Launching VPS as part of Quest Pro's mixed reality system - integration challenges, launch metrics, and lessons.

Enabling users to contribute to mapping - the technical architecture, privacy model, and quality challenges.

How Quest Pro's mixed reality capabilities change the requirements for visual positioning - higher accuracy, lower latency, always-on.

Using neural representations to dramatically compress 3D maps - enabling device-side map storage and faster queries.

A year of building Visual Positioning Service: what shipped, what stalled, and what I would do differently.

Growing a team 3x in one year - hiring, onboarding, structure, and maintaining culture through rapid growth.

What it takes to run ML in production at scale - monitoring, versioning, deployment, and all the things that aren't training.

Why indoor localization is much harder than outdoor - GPS absence, feature similarity, and the path forward.

Adapting visual positioning for smart glasses form factor - different sensors, different constraints, different use cases.

From 50 to 500 mapped locations - the operational challenges of scaling 3D mapping worldwide.

Launching the first version of Visual Positioning Service to external developers - what worked, what didn't, and what we learned.

How to localize without compromising privacy - on-device processing, feature design, and differential privacy approaches.

Using high-fidelity simulation to design and validate perception hardware before silicon exists - extending synthetic data to hardware design.

Maps go stale as the world changes. Strategies for detecting and handling changes in visual positioning.

Moving from hand-crafted to learned feature descriptors for VPS - training, deployment, and performance gains.

The challenges of bringing visual positioning to Quest headsets - power, latency, and the cloud-device split.

Half a year in - what I've learned about operating at Meta scale and building VPS from inception.

Bringing synthetic data practices from Magic Leap to Meta - what scales, what doesn't, and new opportunities.

The technical pipeline for localizing a device using visual features - retrieval, matching, and pose estimation.

Building 3D maps of the entire world - data sources, quality challenges, and the path to coverage.

High-level architecture of the Visual Positioning Service - how XR devices can localize themselves in the world.

First impressions joining Meta's Reality Labs to lead Visual Positioning Service - different scale, familiar problems.

How do you transfer years of context when leaving a role? Documenting, training, and ensuring continuity.

After four years at Magic Leap, I've decided to move on. Reflections on the decision process and what made it time.

With lab access limited, simulation becomes even more critical - accelerating our investment in virtual engineering.

When the pandemic hit, our hardware development process had to adapt overnight. What we learned about remote hardware work.

What changes when AR targets enterprise instead of consumer - reliability, integration, and the perception implications.

Systematic testing infrastructure for perception systems - from unit tests to integration tests to field validation.

Reflecting on nearly four years of building mixed reality systems - what I learned, what changed, and what's next.

Techniques for transferring knowledge from synthetic to real data, and from one perception task to another.

Evolving our sensor fusion architecture for V2 - tighter coupling, failure prediction, and graceful degradation.

Enabling multiple devices to share spatial maps - the technical architecture for multi-user AR experiences.

Advanced techniques for neural network optimization on embedded systems - beyond basic quantization and pruning.

Systematic evaluation of depth sensors for V2 - from datasheets to real-world performance characterization.

Lessons from managing perception development across continents - what actually works between Florida and Israel.

Building infrastructure to generate millions of training images - the engineering of a synthetic data factory.

Advances in spatial mapping for V2 - dense reconstruction, mesh quality, and the compute challenge of real-time 3D.

Designing a full-featured hand tracking system for V2 - gesture recognition, extended range, and the training data challenge.

Techniques for maintaining visual SLAM in challenging lighting conditions - from sensor improvements to algorithmic adaptations.

Translating field feedback and technical learnings into requirements for the next-generation device.

Reflections on the year we shipped - what a product launch teaches that no amount of planning can.

Influencing next-generation compute chip architecture for perception - what hardware do SLAM, depth processing, and ML inference actually need?

After shipping under pressure, it's time to address the technical debt we accumulated - deciding what to fix, what to rewrite, and what to accept.

What we're learning from real users using real devices - the telemetry infrastructure and insights that drive product improvement.

We shipped a mixed reality headset. Reflections on what it means, what we got right, what we got wrong, and what comes next.

The final push before device launch - triaging bugs, making hard cuts, and keeping the team functional under extreme pressure.

Using diffractive optical elements to enable eye tracking in waveguide-based AR displays - one optical stack doing two jobs.

Using laser speckle patterns for high-precision eye tracking - the physics, implementation, and path to a patent.

The journey from a working sensor prototype to thousands of units rolling off a production line - yield, calibration, and quality at scale.

Using extreme variation in synthetic data to bridge the sim-to-real gap - theory, practice, and hard-won lessons.

Deploying neural networks on power-constrained hardware - quantization, pruning, architecture search, and the gap between research and production.

The technical pipeline for real-time hand tracking - depth sensing, segmentation, keypoint detection, and skeleton fitting.

Reflecting on my first full year building mixed reality systems - what worked, what didn't, and what I've learned.

How to generate millions of diverse, realistic environments procedurally - the key to scaling synthetic data.

Why rendering for synthetic data differs from gaming or film - physically accurate sensor simulation over visual beauty.

Why I pushed to create a dedicated synthetic data team, and the case for simulation-first perception development.

The optical design challenges of capturing the eye from millimeters away - illumination, imaging, and integration with the display system.

Building robust eye tracking for mixed reality - from pupil detection to gaze estimation to the unique challenges of see-through displays.

The privacy implications of always-on spatial sensors - what data we collect, how we protect it, and the trust we must earn.

Designing map persistence systems that let AR content stay in place across sessions - the foundation of spatial computing.

Deep dive into VIO algorithms - how we combine visual features with inertial measurements for robust 6DoF tracking.

The hidden complexities of calibrating Time-of-Flight depth sensors for accuracy across temperature, distance, and reflectivity.

The gap between textbook calibration and production-scale calibration of perception systems - where precision meets throughput.

Lessons from hiring and structuring a multi-disciplinary perception team spanning computer vision, optics, and embedded systems.

What I learned from our first major hardware spec freeze - the art of committing to specifications when uncertainty remains.

Architecting a complete perception system for mixed reality - from sensor selection to software pipeline to system integration.

Understanding the optical system design choices for depth sensing - from field of view to distortion to manufacturability.

The physics and engineering of infrared illumination systems for ToF and structured light depth cameras.

Creating tools and processes for systematic performance measurement on embedded hardware - the foundation of optimization.

The brutal realities of running computer vision algorithms on wearable hardware - where every milliwatt counts.

Why we need high-fidelity sensor simulation for developing perception algorithms, and the challenges of making simulated data match reality.

Designing a unified sensor fusion pipeline that combines cameras, depth sensors, and IMUs into a coherent spatial understanding system.

Understanding Simultaneous Localization and Mapping from an implementation perspective - the backbone of any spatial computing device.

A deep technical comparison of Time-of-Flight and structured light depth sensing approaches for mixed reality applications.

Reflections on joining one of the most ambitious hardware startups in history, and the technical challenges that await in building a true mixed-reality device.