Meta has announced the next version of its AI system for non-invasively decoding brain activity into text. Called Brain2Qwerty v2, the company hopes its latest method will help people with neurological injuries or diseases that impair speech.
Meta’s latest brain-computer interface (BCI) builds off last year’s Brain2Qwerty v1, which initially showed that non-invasive brain recordings could be decoded into text with surprisingly high character-level accuracy. It used both electroencephalography (EEG) and magnetoencephalography (MEG)—two non-invasive methods that measure the magnetic and electric fields elicited by neuronal activity—although it was only capable of decoding individual characters.
Now, the company has shown off its v2 model, which is said to improve nearly every aspect of the system by using an end-to-end architecture, large language models (LLMs), real-time decoding, and vastly improved pattern recognition.
Note: The findings was presented in a recent paper, which involved researchers from Meta and a host of universities and institutes, including Université PSL (incl. École Normale Supérieure), University of Lille, Paris Cité University, Université Paris-Saclay, CNRS, Inria, CEA (NeuroSpin), Basque Center on Cognition, Brain and Language (BCBL), and Hospital Foundation Adolphe de Rothschild.
According to the paper, Brain2Qwerty v2 was trained on approximately 22,000 sentences from nine volunteer participants, each of which were recorded for 10 hours wearing an MEG device while actively typing.
Meta says that instead of relying on hand-crafted pipelines to detect neural events, they used end-to-end deep learning to decode directly from raw brain signals—essentially meaning they could not only decode single letters like in v1, but also full words and sentences.
While v2 represents a pretty significant leap forward, it hasn’t approached 100 percent accuracy yet:
“Brain2Qwerty v2 recovers sentences coherently from noisy neural inputs, achieving a word accuracy rate of 61%, significantly improving upon the 8% word accuracy from other non-invasive methods,” Meta says.
Meta says its most performant participant achieved a 78% word accuracy, where “more than half of all sentences are decoded with one word error or less,” the company says.
“We also find that decoding accuracy improves log-linearly with data volume, suggesting that the remaining performance gap with surgical approaches could be further narrowed through data scaling alone.”
The project’s long-term goal is to develop communication technologies for people with neurological injuries or diseases that impair speech, notably without requiring invasive surgery, like Elon Musk’s Neuralink BCI startup, which expanded human clinical trials earlier this year.
Researchers highlight in Nature that while invasive methods are more efficient at thought-to-text, they expose patients to “nonnegligible risks of brain hemorrhage and infection.” Very real challenges in maintaining cortical implant function over extended time periods is also a risk, making invasive methods less scalable overall.
That said, there’s still a long way to go before we see anything approaching consumer-grade MEG devices. Many of the classical MEG devices of today are still very much “helmet in a hospital room” levels of massive, although there are smaller devices now that can operate at room temperatures, like Cerca’s optically-pumped magnetometers (OPMs).
The key limiter holding MEG back though for eventual consumer adoption is background magnetic interference, which requires even these much smaller systems to work in a magnetically-shielded environment; the magnetic fields generated by the brain are much weaker than the Earth’s magnetic field and the host of everyday tech like smartphones, Wi-Fi, and power lines which are all millions of times stronger.
Whatever the case, it’s heartening to see that patients who can’t qualify for invasive BCI could get a significant boost in quality of life someday, hopefully soon.
