Specializing in neuroscience, Professor Junichi Ushiba (Department of Science and Engineering, Keio University) studies the plasticity of the human brain, exploring how it functions and changes with experience. In this interview, he shares insights into his research at IoB on brainwave measurement using wearable sensors.
Controlling Avatars with Wearable Sensors
My role in the IoB project involves developing a Brain-Machine Interface (BMI) that utilizes wearable sensors to non-invasively read brain activity and translate it into real-time AI-powered control of avatars and robots.
Our newly developed EEG (electroencephalogram) sensor resembles a pair of headphones, equipped with EEG sensors on the inside and amplifiers within the earcups to amplify the signals. The data is transmitted via Bluetooth, allowing for real-time analysis of the wearer’s brain state on a PC.
Caregiving Experience as a Research Catalyst
When I was in high school, my grandfather had a stroke. He lost his ability to speak and move independently. In his mind, he was still the same person, thinking about various things and wanting to talk to his family, but he couldn’t express his thoughts or walk up to them. Witnessing my grandfather struggle with communication filled me with frustration.
I wondered if there was a technology that could help him communicate his thoughts or a way for him to move using his own mind. Perhaps with such tools, both he and the family could have lived richer and more fulfilling lives. These firsthand experiences motivated me to pursue BMI research.
Stylish and Everyday EEG Headsets
Our primary focus has been on developing headset-style EEG devices. Leveraging our expertise in medical device manufacturing, we have created a device that allows for everyday EEG measurement. The design is stylish enough to blend seamlessly into everyday life, resembling a pair of regular headphones that you might find at an electronics store. We have also carefully considered the electrode design and materials to ensure high-quality EEG recording even for individuals with hair.
Another significant achievement is the development of AI-powered middleware that removes noise from EEG signals, enabling accurate measurements even in noisy environments. EEG signals are based on subtle changes in electrical potential, making them susceptible to noise from slight head movements or interference from facial muscle activity. The unpredictable nature of noise is a major challenge in EEG measurement. Our team’s originality lies in developing this middleware technology to enhance signal quality.
Playing “Fortnite” with Brainwaves
As part of our user testing, we designed a racing circuit in the metaverse of the massively popular game “Fortnite,” allowing participants to control their avatars using their brainwaves in a time trial challenge to see who could reach the finish line the fastest.
Leveraging this environment, we organized a “Brainpic” event where participants could playfully compete in controlling their avatars. This event welcomed individuals with various physical disabilities, including cerebral palsy and ALS, as well as middle and high school students.
During the Brainpic, we extracted on/off signals from brainwaves indicating whether the player wanted to move forward or stop, translating these signals into their avatar’s movements. However, this limited control to moving forward or stopping. To address this, we incorporated head tilt sensors to control the avatar’s direction.
The Sense of Self-Control Powered by Brainwaves and AI
While it might seem like the AI is doing most of the work, when you actually use the system, you feel a strong sense of control as if you’re moving it with your own mind. This discovery of the gap between perception and reality is one of the more fascinating aspects of this research. To put it in extreme terms, even if the AI does 99.9% of the work, as long as a small part of your intention is reflected, you feel a sense of agency, as if you’re the one controlling the movement.
This realization brings a sense of hope. For individuals with severe physical disabilities, using AI technology to the fullest extent to enable communication and mobility could help them maintain their self-esteem and stay connected to society.
Enriching Lives Through Communication
As we envision the society we want to create by 2050, I believe it should be a world where everyone can live each day with joy and fulfillment. When I think about what that means, I realize that it’s about living with the people we cherish and connecting with them on a deeper level.
Therefore, I believe that creating a state where we can communicate with the world around us, even down to the devices we use, is essential for building a 2050 that is rich, compassionate, and filled with happiness.
Embrace the Excitement of Science!
What I want to convey to young people is that science is brimming with wonder and excitement. It’s a tool that can make dreams a reality. Science is not just about creating practical things; it’s about enriching the lens through which we see the world. That’s the beauty of science that I want everyone to experience.
Interview, Writing, and Video Editing by Space-Time Inc.