Overview
IoB Middleware aims to develop technology that directly shares information extracted from brain activity, from the brain to a computer or between brains. Specifically, we will develop new AI systems capable of extracting sensory and thought information from brain activity, build mathematical theory to enable the sharing of the extracted information, and test the technology through animal experiments.
To make direct brain-to-brain communication possible, it is essential to have the technology to decipher the neural codes used in individual brains and the technology to transmit the deciphered codes from brain to brain. However, the neural codes used by the brain are likely to be different between individuals, and the task of translating the codes between multiple brains is necessary. In addition, operating devices based on brain activity also requires code translations between the brain and the device. Information translation technology between different systems is one of the core technologies for the success of this project, and the main goal of this research item is to develop the mathematical theory for its construction and achieve its engineering implementation.
Member
-
SASAI Shuntaro,Ph.D.
Chief Research Officer and Director of R&D Department, Araya Inc.
To facilitate information communication between different information systems, such as the brain and AI or between brains, it is required to develop techniques that appropriately translate information between different systems. This research aims to build such translation technology by developing methods to align the structure of information representation across different systems. Additionally, we will use this method on neural activity data obtained from humans and monkeys to examine whether information representations can be standardized across individuals. We aim to further extend this technology to create an information sharing space that can be utilized by multiple AIs and brains.
- IoB Middleware
-
ARULKUMARAN Kai,Ph.D.
Team Leader, Neurotechnology R&D Unit, Araya Inc.
In this research project, we will develop AI systems that can act autonomously to fulfill the goals of the user, which may be specified in forms such as natural language or via BMI. To do so we will focus on goal-based reinforcement learning agents, which learn to perform tasks based on interaction with their environment and reward signals. The agents will also need continual learning capabilities in order to adapt to changes after deployment. Furthermore, we will develop multi-agent systems that can infer the goals of other agents, such as humans, to work on collaborative tasks. The agents can take physical or virtual forms, allowing applications in domains from robotics to virtual avatars.
- IoB Middleware
-
HAYASHI Ryusuke,Ph.D.
Senior Researcher, Human Informatics and Interaction Research Institute, National Institute of Advanced Industrial Science and Technology (AIST)
In order to develop IoB that reads neural information from the brain for avatar-control and brain-to-brain communication, it is necessary to construct a mathematical basis for standardizing the representation of conceptual information that differs among brains and AI models. In this research, we will focus mainly on conceptual information extracted from image data, and aim to develop a mathematical infrastructure to realize the common representation of such conceptual information. Furthermore, we will record neural information data from the visual cortex and experimentally verify the developed technology. Through this research, we will contribute to developing new technologies that will lead to the expansion of human abilities.
- IoB Middleware
-
KOIKE Hideki,Ph.D.
Professor, School of Computing, Tokyo Institute of Technology
In this research, we will develop a method for skill acquisition that takes into account brain information, in addition to physical information, in contrast to conventional skill acquisition research that focuses only on physical information. As the first research item, we will develop a method to predict the posture in the near future by measuring the physical and brain information during a specific movement using deep learning. Secondly, we will develop a spatio-temporally distorted skill acquisition environment using virtual reality. Then, we will identify the optimal spatio-temporal distortion parameters based on the measured brain states.
- IoB Middleware
-
OIZUMI Masafumi,Ph.D.
Associate Professor, Graduate School of Arts and Sciences, The University of Tokyo
In this research project, by utilizing the methods of control theory and nonequilibrium statistical mechanics, we will (1) estimate mental fatigue from brain activity, and (2) estimate optimal input to achieve specific brain state transitions, which is needed for direct brain-to-brain communication. More specifically, we will develop a theoretical framework to quantify the transition cost between brain states in stochastic (non)linear neural systems and work on the estimation of mental fatigue and optimal control input to the brain from the perspective of transition cost and efficiency.
- IoB Middleware
-
REKIMOTO Jun,Ph.D.
Professor / Deputy Director Interfaculty Initiative in Information Studies, The University of Tokyo / Sony Computer Science Laboratories Inc.
In this research, we will develop technology to decode speech intentions from non-invasive, invasive, and non-contact information. We will solve the problems of deciphering the intended content of speech from these signals and inputting the content to the brain of another person. In addition, by using the extraction technology of the mental state to be developed, we aim to convert the sensory understanding of an expert into data and feed it back to the individual, so that it can be used to support the acquisition of skills that are difficult to communicate verbally. This research is expected to enable a new means of interaction between humans and computers.
- IoB Middleware
Related Links
[Research Article] Masafumi Oizumi and Colleagues’ Research Outcomes Featured in a Press Release: “Decomposing Thermodynamic Dissipation of Linear Langevin Systems via Oscillatory Modes and Its Application to Neural Dynamics――Toward a Thermodynamic Understanding of Neural Dynamics――”
[Research Article] Silent Communication and AI will Change our Society: Silent Speech Enables Fusion of AI and Humans
[Event Report] We Exhibited at the Neuro2024 Exhibition Booth
[Research Article] Utilizing Consciousness in Communication: Measurement of Brain Waves during Speech Using an Ultra-High-Density EEG (Electroencephalograph)
[Research Article] Discovery of a Scaling Law in Deep Learning Using EEG: A Path Towards Practical Non-Invasive Speech BMI
[Research Video] ChatGPT-empowered Brain Gmail Interface
[Research Video] Brain-Robot Interface with Imitation Learning