On June 29, Meta introduced a system that reconstructs typed text from brain activity recordings without surgical intervention. According to the company, the average word-level accuracy reached 61%, compared to about 8% with previous non-invasive methods.
Brain2Qwerty v2 was trained on approximately 22,000 sentences. The experiment involved nine healthy volunteers, each spending 10 hours in a magnetoencephalography scanner while typing heard phrases on a keyboard.
“Instead of manually created schemes to identify neural events, we use end-to-end deep learning to decode data directly from raw brain signals,” Meta stated.
Brain2Qwerty v2 works with continuous brain activity recordings and reconstructs entire sentences. To refine the results, the system employs a fine-tuned language model that considers semantic context. The best participant achieved 78% word-level accuracy, with more than half of their sentences decoded with one error or none.
The development is not yet ready for clinical or everyday use. Brain2Qwerty v2 was tested on healthy volunteers who actively typed text. This approach needs separate validation for individuals who have lost their speech or mobility.
Additionally, the system does not yet function as a full real-time text input. The model processes entire sentences, so users do not see each word immediately after it is typed.
The project is linked to the Digital Brain Project, an initiative aimed at creating open neuroscience datasets. The program has a budget of $5 million.
Recall that in September 2022, Meta published research on a "brain decoder" that uses artificial intelligence to convert thoughts into speech.