Scientists at the University of California, San Francisco (UCSF) have demonstrated a brain-computer interface that can decode speech and body gestures simultaneously in people with severe paralysis.

The system uses a single high-density electrocorticography (ECoG) implant placed over the brain’s sensorimotor cortex. Artificial-intelligence models interpret neural activity generated when participants attempt to speak or perform gestures.

In the study, three participants were enrolled, while two took part in the main simultaneous speech-and-gesture experiments. Their brain signals were used to control a personalized full-body virtual avatar. Intended speech could appear as text, while movements such as waving, nodding or other gestures could be represented by the avatar.

The researchers found that training the AI using both isolated and simultaneous speech-and-gesture data improved its ability to distinguish different types of brain activity. In one conversational test, one participant achieved median decoding accuracy of 100% for both speech and gestures across the small number of tested blocks.

The technology could eventually provide a more natural communication method for people affected by conditions such as severe stroke or ALS, for whom conventional communication can be difficult or exhausting.

However, the research is still a proof of concept. The experiments involved a very small number of participants and a limited vocabulary and set of gestures. Researchers say larger studies and broader communication capabilities will be needed before the technology can be used routinely.

The team is also working toward a fully implantable wireless version, which could make future systems more practical for long-term use.