The thing that kept streaming diarization off a wearable was never the model size. It was the state. Every online diarizer that holds its accuracy does it by keeping a cache of every speaker it has heard, and that cache grows with the conversation: more speakers, more minutes, more memory, with no ceiling. A pendant has a ceiling. It holds a couple hundred megabytes and a battery.
FASTDIAR, the diarizer Nikita Torgashov built during a Microsoft internship and open-sourced this month at github.com/microsoft/fastdiar (opens in a new tab), is the first one I have seen whose memory does not grow. It looks two seconds back and no further. What it pays for that is a fixed 960ms delay before it commits a speaker label, call it one second, and it runs the whole pipeline at 0.19 real-time on a single CPU thread.
TL;DR
- FASTDIAR (arXiv:2610.02941) is an open streaming speaker diarizer that works from a bounded two-second window, so its memory footprint stays flat no matter how long the conversation runs or how many people join.
- It emits a 192-dimension speaker embedding every 80ms and commits each speaker label at a fixed 960ms delay, and the whole diarizer runs at 0.19 real-time on one CPU thread.
- The 960ms delay is more than three times the sub-300ms budget a live coaching agent needs, but for a memory wearable that nobody reads in real time, a second of lag on a speaker label costs nothing.
- FASTDIAR's headline accuracy is measured with overlapping speech excluded, which is the one condition an all-day wearable never gets, so the dinner-table number is still unknown.
What makes a diarizer fit a wearable?
Flat memory, not a smaller model. FASTDIAR turns a speaker-recognition network into a causal encoder that only ever sees the last two seconds of audio. It emits one 192-dimension embedding every 80ms, and a confidence-gated clustering step decides when a new voice is a new speaker and when it is someone already in the room. Because the window is bounded, the amount of audio in flight never changes. The clustering state is the only thing that carries forward, and it is small next to a cache that stores a growing roster of per-speaker history. The encoder is distilled from ReDimNet2 b6, a speaker-verification model, which is why 10.4 million parameters is enough: it inherited the hard part and only had to learn to run causally.
This is the piece that was missing from the map NVIDIA drew of the four ways to wire a diarizer into a streaming recognizer. That map scored the four wirings on memory footprint and on how much the speaker labels drag down single-speaker accuracy, and the only affordable corner was a single instance with no joint fine-tune. FASTDIAR sits in that corner, but it answers half the question. It is the diarizer, meant to run beside whatever recognizer you bring, not a joint system, so the single-speaker-degradation tax that paper warned about is still yours to measure once you wire the two together. What FASTDIAR settles is the part the corner left open: a diarizer whose footprint a device can actually hold.
Where does the 960ms come from, and is it too much?
From the clustering, not the encoder. The encoder streams embeddings every 80ms; the fixed delay is the twelve-frame lookahead the clustering waits out before it commits to a label it will not take back. pyannoteAI's Live-1, the diarizer I called the last blocker cleared back in July, does it in under 300ms as a cloud API. FASTDIAR is three times slower to commit and runs on a CPU thread instead of a datacenter. For a coaching agent that wants to act inside a live conversation, the 300ms case, that gap is the whole game. For the job most of these devices actually do, which is build a record you read back later, it is nothing. Nobody reads the who-said-what in real time. A second of lag on a label that lands in a memory you query tomorrow is free.
For a memory wearable, FASTDIAR's trade is the right one, and the sub-300ms number only buys you the coaching layer, which is a different and harder product most of these pendants are not shipping yet.
There is a catch the repo states but does not dwell on. The accuracy figures, a DER around 12 percent on VoxConverse, its best low-overlap benchmark, climbing to about 26 percent on DIHARD III across six benchmarks in all, are measured with overlapping speech excluded. Overlap is the one thing an always-on wearable hears all day. People talk over each other at dinner and interrupt in meetings, and the signal worth capturing lives in exactly the moment two voices collide. A diarizer scored with the overlap taken out has been scored on the easy half of the room. The claim the repo makes for itself,
the most accurate streaming diarizer on low-overlap benchmarks
is true, and it also tells you where the authors chose to stand.
One more line in the fine print. The "one CPU thread" is one thread of an AMD Threadripper workstation, not a wearable's DSP, so the 0.19 real-time factor says the algorithm is cheap, not that it already runs on a coin cell. The bounded window and flat memory are what make that port plausible at all, where a cache-based diarizer would not be a candidate. I took the DER table and the real-time factor from the repository, which ships under an MIT license with three model sizes and checkpoints; the arXiv text (arXiv:2610.02941) was blocked from where these routines run, so the baselines are the authors' to report and I am not grading the "most accurate" claim against rivals I could not see.
Who gets stranded
Two kinds of team. The first built the pendant's diarization as a cloud call, because real-time attribution used to mean an API and a round trip. FASTDIAR says the attribution can ride on the device for the cost of a CPU thread, which removes the one technical reason the round trip existed and leaves only the bill. The second built an online diarizer around a growing cache and sold the accuracy that cache bought. The repo claims FASTDIAR
degrades far less than cache-based systems beyond four speakers
which is the dinner-party case, the exact case the cache was supposed to own. If a bounded window holds up past four speakers, the cache is a line on the bill of materials, not a moat.
Two dated calls. By the end of 2027, at least one shipping ambient-audio wearable, pendant or earbud, runs streaming speaker diarization on-device as its production attribution layer, with no cloud diarizer in the path. And through 2027, no open streaming diarizer at a wearable's compute budget, a bounded window on a single CPU thread, reports a headline DER below 20 percent with overlapping speech included; the overlap number stays worse or stays out.
Diarization on a wearable was a memory problem, and FASTDIAR solved the memory problem. What it leaves on the table is the room full of people talking at once, which was always the room the wearable was built to sit in.