Scientists develop new technology to give speech-disabled patients a voice
Recent advancements in technology are offering hope to those who have lost the ability to speak due to neurological damage. (CREDIT: Creative Commons)
Recent advancements in technology are offering hope to those who have lost the ability to speak due to neurological damage. Scientists are developing neural speech prostheses, special devices that translate brain activity into speech.
A team of researchers from New York University (NYU), led by Yao Wang, Professor of Electrical and Computer Engineering and Biomedical Engineering at NYU Tandon, and Adeen Flinker, Associate Professor of Biomedical Engineering at NYU Tandon and Neurology at NYU Grossman School of Medicine, along with Tandon ECE Ph.D. student Xupeng Chen, has made significant progress in this area. Their findings were published in Nature Machine Intelligence.
The team's work focuses on improving the decoding of speech using neural architectures. By capturing signals from the brain and transforming them into audible speech, their research aims to enhance communication for individuals who struggle with speaking.
The proposed neural speech decoding framework. (CREDIT: Nature Machine Intelligence)
One key aspect of their innovation is adapting neural architectures to accommodate diverse speech patterns, a crucial step in developing effective neural speech prostheses.
Gathering data for such prostheses involves using Electrocorticographic (ECoG) recordings from epilepsy surgery patients. These recordings, obtained through implanted electrodes, provide precise cortical data during speech, leading to promising results in speech decoding.
The team's approach has been validated across 48 individuals, significantly larger than previous studies, ensuring a more robust decoding process.
Related Stories:
Two major challenges in decoding speech from neural signals are the limited duration of training data and the variability in speech production. To address these challenges, the NYU team developed a unique speech synthesizer that translates interpretable speech parameters into natural-sounding speech.
Leveraging neural network architectures, their system can decode neural signals into speech parameters, producing speech that closely resembles the participants' voices.
A particularly intriguing finding from the study is the contribution of the right hemisphere to speech decoding. Traditionally, the left hemisphere is associated with language functions, overshadowing the right hemisphere.
Decoding performance comparing the original and decoded spectrograms across non-causal and causal models. (CREDIT: Nature Machine Intelligence)
However, participants with electrodes only on their right hemisphere still provided valuable information for accurate speech decoding. This discovery sheds light on how speech is processed across the brain's hemispheres and opens new possibilities for therapeutic interventions, especially in addressing speech disorders following damage to the left hemisphere.
In addition to scientific findings, the study offers an open-source neuro-decoding pipeline, promoting collaboration and replication of results within the research community. This initiative fosters transparency and accelerates progress in the field of neural decoding.
Comparison of decoding performance under different settings of the 3D ResNet and 3D Swin models. (CREDIT: Nature Machine Intelligence)
The implications of this research extend beyond understanding speech processing to potential therapeutic applications. By unraveling the mysteries of the brain's linguistic capabilities, the NYU team's work paves the way for future breakthroughs in neuroengineering and clinical interventions.
With ongoing support from grants, they continue to explore neural decoding approaches, striving to further advance this promising field.
For more science and technology stories check out our New Discoveries section at The Brighter Side of News.
Note: Materials provided above by The Brighter Side of News. Content may be edited for style and length.
Like these kind of feel good stories? Get the Brighter Side of News' newsletter.
