cd ~/
MappingSLAMPersistenceSpatial ComputingMagic Leap

Persistent Spatial Maps: Remember Where You Were

Designing map persistence systems that let AR content stay in place across sessions - the foundation of spatial computing.

Evyatar Bluzer
3 min read

Put on an AR headset today and it wakes up in a brand new universe: the virtual furniture you placed yesterday is gone, your carefully positioned sticky notes have vanished, and everything has to be set up again. Until maps persist, AR stays a toy. Persistence was already on the MR-specific SLAM list a year ago; this is the design for it.

The Persistence Problem

Persistence breaks down into four requirements. Relocalization: recognize that you're in a previously mapped space. Alignment: register the current tracking session to the stored map. Consistency: virtual content appears in exactly the same physical location it was left in. And evolution: handle environments that change over time.

What do we actually store?

This is the storage question I left unsettled in the architecture post. A sparse map holds keyframes with visual features and 3D landmarks. It's compact, on the order of megabytes per room, and fast to load and match, but it only works where there are good visual features. A dense map, a full 3D mesh or voxel grid, gives you complete geometry for occlusion and physics at the cost of size - hundreds of megabytes per room. The hybrid approach uses sparse for localization and dense for rendering and interaction, and its cost is the complexity of keeping the two synchronized.

We're pursuing hybrid: the sparse map is the "skeleton" for the VIO tracking core, with the dense mesh attached for content interaction.

Hybrid map layersTwo boxes side by side inside a faint container labeled hybrid map, sparse on the left and dense on the right, joined by a highlighted double-headed arrow; an arrow drops from each to a consumer box below, tracking on the left and content interaction on the right. HYBRID MAP Sparse map keyframes, visual features, 3D landmarks megabytes per room Dense map 3D mesh or voxel grid hundreds of megabytes per room kept synchronized skeleton for attached for VIO tracking core and relocalization Occlusion, physics, content interaction
The hybrid map as two layers: a sparse skeleton of megabytes per room that the tracking core localizes against, and a dense mesh of hundreds of megabytes per room for occlusion and physics, with synchronization between them as the cost of having both.

Relocalization Pipeline

When the headset wakes up:

1. Capture initial frames
2. Extract features
3. Query map database (visual vocabulary / learned descriptors)
4. Candidate map retrieval
5. Feature matching against candidates
6. Geometric verification (PnP + RANSAC)
7. Pose refinement
8. Confidence check → Relocalized!

Target: relocalize in under 2 seconds with >95% success rate in mapped areas.

Map Updates

Environments change. Furniture moves, lighting shifts, walls get renovated. You can keep maps immutable and accept drift, re-map from scratch periodically, or merge new observations into the existing map incrementally. Incremental is the ideal and also the hardest: how do you merge conflicting observations, and when is a change permanent rather than temporary?

We're starting with immutable maps plus an explicit "remap" user action. Incremental updates are a future optimization.

Storage and Retrieval

At scale, users will have maps of home, the office, friends' houses, coffee shops - potentially hundreds of maps. The architecture we're sketching keeps local storage for frequently visited places, cloud storage for the full history, and smart prefetch based on location and calendar.

Which brings up the part of this that worries me most. These maps are detailed 3D models of private spaces, so encryption, access control, and user consent have to be designed in from the start. I'm saving the privacy architecture for its own post, because it is not a footnote.

Comments