cd ~/
Map SharingMulti-UserArchitectureSpatial ComputingMagic Leap

Cross-Device Map Sharing: Architecture and Challenges

Enabling multiple devices to share spatial maps - the technical architecture for multi-user AR experiences.

Evyatar Bluzer
4 min read

Every multi-user AR demo rests on a prerequisite that rarely gets mentioned: my device's map of the room has to become your device's map of the room. Map persistence was about a map surviving until tomorrow; this is about it surviving the trip to someone else's headset. That handoff is harder than it sounds, and it's what my team has been working through for V2.

The Problem

Device A maps a room. Device B walks into the same room. Before any shared experience can happen, B has to recognize that it's standing in A's mapped space, align its coordinate frame to A's, and then both devices have to keep tracking in that common frame while updates from either side propagate to the other. Miss any of those and the shared content drifts apart or never appears.

Map sharing handoffA sequence diagram with three lifelines, Device A, Cloud and Device B; a note on A describes the sparse map, arrows carry it to B directly or through the cloud, a highlighted note on B is the alignment step, a second note is tracking in the common frame, and a final two-headed arrow between A and B is the update exchange. Device ACloud (sync, backup)Device Bco-located: local sharingotherwise: cloud sync sparse map, direct sparse map sparse map for this room updates from either side;eventual consistency,vector clocks resolve conflictsMaps the room; keeps a sparse map:3D landmarks plus descriptors,100s of KB, no imagesMatch features to A's landmarks,estimate the rigid transform,verify it makes physical sense,refine as observations arriveTracks in the common frame;named anchors carry persistent content
Custody of a room's map from Device A to Device B: the sparse map travels directly when the devices are co-located or through the cloud otherwise, B aligns itself to A's frame, and both then exchange updates under eventual consistency.

Architecture Options

Option 1: Cloud-Centric

All maps stored centrally, devices query the cloud for local maps.

Device A → Upload map → Cloud → Download map → Device B

You get a single source of truth and easy consistency. You pay for it in latency, connectivity dependency, and privacy exposure.

Option 2: Peer-to-Peer

Devices exchange maps directly.

Device A ←→ Direct connection ←→ Device B

No connectivity required and lower latency, but now you own a discovery problem and some genuinely ugly consistency challenges.

Option 3: Hybrid

Local sharing when devices are co-located, cloud sync otherwise.

      ┌─── Cloud (sync, backup) ───┐
      │                            │
Device A ←── Local when nearby ──→ Device B

This captures the strengths of both, and in exchange you maintain multiple sharing paths forever. We're implementing hybrid for V2.

What Do We Actually Share?

Raw data - keyframes, features, point cloud - is complete but heavy, 10s of MB per room, and it carries privacy risk because images are in there. A sparse map of 3D landmarks plus descriptors runs 100s of KB, which is sufficient for localization and privacy-preserving since no images are involved. Anchor-based sharing, just named spatial anchors, is minimal in size with the easiest privacy model, but it only covers the anchor locations themselves.

We landed on sparse maps for localization and anchors for persistent content.

Coordinate Frame Alignment

Two devices mapping the same space won't share a coordinate origin, so alignment is its own pipeline. Match features to find common 3D points visible to both devices, estimate the rigid transform (rotation plus translation), verify the result makes physical sense, then keep refining as more common observations arrive. The failure mode that keeps us up at night: when the space has changed between mappings, alignment may fail outright or, worse, quietly land on a wrong answer.

Consistency Management

Once both devices are updating the shared map, you inherit the classic distributed-systems questions. Who wins when observations disagree? How do you merge complementary observations? How do you track what each device has seen? We're implementing eventual consistency with vector clocks for conflict resolution, which handles all three without pretending a headset has a reliable connection.

Privacy Architecture

Sharing maps raises real privacy concerns. The 3D structure reveals your room layout, visual features could plausibly be matched against photos, and a location history is implicit in how the maps get used. Our protections carry over the 2017 rule that descriptors, not images, are what we keep: explicit sharing consent per map, feature representations that can't reconstruct images, and sharing scope controls (public, friends, session-only).

Working on patent for the sharing architecture.

[Patent granted 2024: US12066545 "Methods and Systems for 3D Map Sharing Between Heterogeneous Computing Systems"]

Comments