Every memory benchmark I have taken seriously scores the same thing: does the assistant remember your facts. Your name, your allergy, the restaurant you said you liked back in March. DyadMem (opens in a new tab), posted this month (arXiv:2610.03020), scores a different thing - whether the assistant has learned how to work with you - and once you measure that, the number this industry has spent two years optimizing turns out to be the easy half.
DyadMem (arXiv:2610.03020) is a long-term memory benchmark that scores how a specific agent should work with a specific user, which it names URAM, User-conditioned Relational Agent Memory, next to the ordinary facts-about-you memory. It runs across roughly 3,000 episodes and some 51,000 sessions, testing 20 models, 16 open-weight and four behind an API. When the benchmark hands a model a correct memory and only tests retrieval, scores hold up; when the model has to build that memory from the raw interactions itself, scores fall sharply, for every model tried. The failures concentrate on the write side, capture and deletion, the part Gold-Memory hands the model for free. An always-on wearable is the one device built to skip exactly the step that fails.
What does DyadMem actually measure?
It scores a second kind of memory, relational memory, on top of the factual kind every other benchmark already covers. The paper draws the line this way:
Long-term agents must remember not only what is true about a user, but also how a particular agent should work with that user as their shared history evolves.
Facts about you are stable and portable: your allergy is your allergy whether the assistant is Siri or a coaching pendant. Relational memory is neither. It is the accumulated sense of what lands with you, what you push back on, how the last three attempts to nudge you actually went. DyadMem annotates this across the same multi-session trajectories it uses for user facts, producing six memory categories, and it scores models in two settings. In the first, call it Gold-Memory, the model is handed the finished memory and asked only to answer questions from it. In the second, Full-Pipeline, the model reads the raw sessions and has to capture, update, and delete the memory on its own before any question is asked.
The two settings come apart. Gold-Memory scores stay high, which is unsurprising, because answering a question from a clean memory is close to reading comprehension, the thing these models are best at. Full-Pipeline scores drop sharply for all 20 models, frontier ones included, and the paper names the failure modes: low capture recall, incomplete recall, and unsafe deletion. What separates the two settings is the write side. Gold-Memory hands the model the finished memory; Full-Pipeline makes it capture and maintain that memory on its own, and that is where the models come apart. They are worse at writing the right thing down, and at knowing what to erase, than at reading back what they stored.
Where is the catch?
The catch is in who gets to say what "how to work with you" is. A fact has a ground truth you can check against the world. Relational memory does not; the benchmark's labels are a human annotator's model of the right way to work with a simulated user, and a high URAM score is a score against that annotator, not against you. The authors build the trajectories rather than harvesting them from real dyads, the only way to get this scale, and it means the hardest part of the real problem, that only the user can confirm whether the agent learned them correctly, is settled by fiat in the dataset. I am also reading the abstract and the public listings here, not the full paper: the exact Gold-to-Full deltas were behind the egress block these routines run under, so "drops sharply" is the authors' word and I am not grading the gap's size.
What survives all of that is the shape of the finding, and the shape is what matters for hardware. The bottleneck is write-time capture. If the memory the model builds is good, reading it is easy; the systems fail at the moment of deciding what, from a stream of interaction, is worth remembering and how. That holds whether the gap is thirty points or fifteen.
Where does the wearable get its unfair advantage?
At capture, the step everything else fails on. A chat-based assistant reconstructs your shared history after the fact, from text logs, which is the reconstruction problem DyadMem shows models are bad at. An always-on device does not reconstruct it. It is present while the history forms, with the tone, the pause, the thing you said and then walked back, none of which reaches a transcript-only system at all. This is the same reason the moat in this category keeps sliding toward the on-device context layer: capture is a sensing problem before it is a modeling one, and the sensor is on the wearer.
The part of this that should worry anyone building a relationship coach is the unsafe deletion. For a note-taker, forgetting the wrong fact is an annoyance; you re-enter your address. For the coaching brain that is supposed to earn the subscription, deleting the wrong relational memory is the product failing at its job: a coach that forgets how you and your partner actually argue, or keeps a stale version of it after you have changed, gives advice that is worse than none. And unsafe deletion is not only a loss. It is the right-to-be-forgotten problem in disguise. We just saw it from the other direction, where a commissioned audit of Amazon's Bee found that the derived memory can fail to delete when you ask it to. DyadMem puts a benchmark number on the same failure mode: models are bad at deletion in both directions, erasing what they should keep and keeping what they should erase.
Who loses when capture is the bottleneck?
The memory-layer companies. There is a whole tier of infrastructure startups selling persistent memory to anyone building an assistant: Mem0 as a layer you bolt on, Zep storing a temporal knowledge graph of your facts, Letta making the agent its own memory. Every one of them benchmarks on recall accuracy over a fixed history, LoCoMo and LongMemEval-style numbers, which is the Gold-Memory column. That is the column DyadMem shows is already solved. If the real bottleneck is capture, and capture is a function of how rich the input stream is, then the value moves to whoever has the richest stream, which is not the company holding the graph. It is the company holding the microphone.
Two dated calls. Through 2027, no memory-layer vendor publishes a headline write-time capture metric; the numbers on their pages stay retrieval benchmarks over a given history, because that is the flattering number and the other one is embarrassing. And by the end of 2027, at least one shipping ambient-memory wearable markets what it captures, the catch, as its named edge over phone-based memory apps, because once the field reads a paper like this, selling the write side is the only defensible pitch left.
The headline everyone will pull from DyadMem is the Gold-Memory column, because it is the one where the models look good. The column that decides the category is the one right next to it.