ALLPCB
Overview
Neuroprostheses are devices that can replace lost neural functions and may restore communication for patients who cannot speak or type due to paralysis. It has been unclear whether silently attempting to speak can be used to control a communication neuroprosthesis. Earlier work demonstrated a prosthetic system that could decode up to 50 words, but that approach was limited to a fixed vocabulary and required the participant to attempt to speak those words aloud, which can be extremely effortful for paralyzed individuals.
Study
On November 9, 2022, Edward Chang and colleagues at the University of California, San Francisco published a paper in Nature Communications titled "Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis." The study developed a device that decodes brain activity while a person with aphasia and paralysis silently attempts to spell words to form sentences. The results highlight the potential of a silent, spelling-based speech neuroprosthesis to generate sentences.
Participant and Implantation
The participant experienced severe vocal and limb paralysis after a stroke at age 20 and had been unable to speak for more than 15 years. To decode his language intent, the research team implanted brain-computer interface electrodes in the speech motor cortex in 2019 as part of an arm and speech BCI restoration trial. The electrodes recorded neural activity while the participant attempted to produce different words or letters, and those signals were analyzed to infer intended speech.
Methods and Results
The team designed the system to decode phoneme-related neural activity and thereby extend prior methods to a much larger vocabulary. In tests where the participant silently attempted the spoken sounds of letters, the device decoded neural activity and generated sentences from a 1,152-word vocabulary that covers more than 85% of typical English sentences. The system produced text at 29.4 characters per minute with an average character error rate of 6.13%. Further experiments showed the method generalized to a vocabulary of more than 9,000 words (approximately the number of distinct words most people use in a year), with an average error rate of 8.23%. These findings indicate that a spelling-based, speech-encoded neuroprosthesis controlled silently has substantial promise, though additional work is required to determine reproducibility across more patients.
Related Work
Edward Chang has previously published research on decoding brain activity and human language. In October 2021 he and colleagues reported in Nature Medicine the use of a chronic responsive deep brain sensing and stimulation device that provided therapeutic benefit for a patient with treatment-resistant severe depression.
