Our goal is to create a world where people with or without disabilities can move their bodies as they wish using brain waves.

At “Brain World 2024 for Everyone: Super Diversity,” visitors can come into contact with cutting-edge research results from various companies, universities, and research institutes that are working to realize a neuro-diverse society.NTT Human Information Laboratories is engaged in research aimed at making your body move freely by brain waves when you imagine moving your body in your mind. Nanako Ishido (Photo 5), Director of B Lab, which is promoting the “Brain World for Everyone” exhibition, interviewed Takashi Isesaki (Photo 1) and Yukio Koike (Photo 1) of NTT Human Information Laboratories.

NTT Human Information Laboratories
Takashi Isesaki, Yukio Koike

Move Your Body the Way You Want with Brain Waves
Movement support by neurocybernetics

Ishido: “This is the second time for NTT Human Information Laboratories to participate in the exhibition, following 2023. Could you describe your latest research, including the contents of your exhibit at “Brain World for Everyone”?”

Mr. Isesaki: “We are researching ‘exercise support through neurocybernetics’ that utilizes brain waves. We are working on the construction of an exercise system that extracts the person’s exercise intentions from brain information and enables the person to move his or her body using this information. Our goal is to realize a world where people can move their bodies as they wish, regardless of whether they have disabilities or not. (▲Photo 2▲)

There are two types of methods used to measure brain activity when actually extracting motor intentions from brain information: one is called invasive, in which electrodes are implanted in the brain. The other is non-invasive, in which a cap-like object is placed over the scalp. The non-invasive method is the most widely used, but we are also conducting research on the invasive method, which requires surgery but can measure brain activity with high precision, in anticipation of its future potential.

In the invasive type, electrodes are placed on the surface of the brain to measure the brain’s electrical signals. The feature of this technology is that it provides a more detailed reading of how a person wants to move his or her body through the electrical signals from the brain. For example, we are conducting research to realize a motor reconstruction technology that will enable people with disabilities, such as spinal cord injuries, who cannot move their bodies as they wish, to move their arms. Specifically, the human body is connected to the brain and muscles by nerves that pass through the spinal cord, and we are working to create a model that simulates these nerve connections using digital technology.

In the non-invasive type, on the other hand, the subject wears a cap-like object and is asked to imagine an exercise such as “squeezing his/her right hand. We are conducting research to read the brain waves and use AI to extract information such as “this person is now squeezing his/her right hand,” which can then be used to control wheelchairs, devices, game characters, avatars, and so on. We are also researching non-invasive EEG to see if there are scenes where it can be used in the real world. Non-invasive brain waves also allow people to experience the ability to control games using brain waves. Recently, there are games that allow users to control characters not only with a controller but also by moving their bodies. We would like people to know the possibilities of such operations by using brain waves. (Photo 3)

The game is still under development, but we are creating a game in which the character on the screen is moved left and right by imagining the movement of the right or left hand, while hitting a ball against a target in front of it to destroy it. (▲Photo 4▲)

People with spinal cord injuries and other conditions that prevent them from freely moving their bodies
In the future, it will be possible to “drink water from a glass” using brain waves.

Ishido: “In this research, when a person thinks right or left, the character moves in that direction; in other words, you can make things move based on what you think in your head.

Mr. Isesaki: “Ideally, we aim to realize such functions.

Ishido: “I know it is difficult to measure, but what is the degree of perfection, or the degree to which the game player feels he or she can move the game as he or she wishes? And what factors, if any, inhibit that?”

Mr. Isesaki: “We are still at the stage of considering this project, but there are still hurdles to overcome before we can make it a reality. In order for users to feel that they can move as they wish, the accuracy must be high, so we are conducting research and development of EEG signal processing technology to increase the accuracy as much as possible. We also think that even if the accuracy is a little rough in the adaptation scene, we need to have a scene that is somewhat viable.”

Ishido: “You say that the game is controlled by brain waves, but I think some people can do it well and others can’t. Do you have any tips on how to do it well? Do you have any tips on how to do it well?

Mr. Isesaki: “At an event we held last year, we had children actually try it out. It is true that there are children who are good at it and children who are not so good. The children who are good at it concentrate well. If they laugh during the process, it will cause noise in their brain waves. Then there are the children who can adapt to the game in their own way, being aware of how it affects the game when they are imagining the right or left movement for themselves, and they seem to do well.”

Ishido: “It is quite difficult. I think that people who are good at games, e.g., e-sports players, play with great concentration. Would such people be able to move the game better?”

Mr. Isesaki: “To explain in more detail, it is necessary to concentrate. In addition, when measuring motor imagery, we look at the distribution of activity in the right and left temporal motor cortices of the brain. It is important that the distribution is clean and reproducible over and over again, and this is related to accuracy.

We are looking into that as a possibility to see if the distribution is cleaner and more reproducible for people who have an exercise habit or have done that kind of exercise in the past.

Ishido: “There were many different elements to think about in the game scene I just described. For example, I was more conscious of other things, such as being more concerned about the color green or pink than being told right, but aren’t many of those other elements noisy?”

Mr. Isesaki: “Information from various environments makes noise, so I think children who can operate well seem to shut out information. They may need to discern information in their own way.”

Ishido: “I was surprised that we have reached the point where we can move and manipulate things with brain waves if we wear something like a cap in a non-invasive manner. Could you tell us about the areas where you can apply this technology in our daily lives at this point, and the areas you are envisioning?”

Mr. Isesaki: “The first is wheelchair operation. One of our goals is to enable people with physical disabilities to go wherever they want to go by operating their wheelchairs at their own will.

Ishido: “On the other hand, what are your goals for the invasive type at this point? Also, with an eye toward 2050, what kind of things are you aiming to be able to do?

Mr. Isesaki: “Currently, the non-invasive type can only measure granularity, such as ‘right or left,’ but the invasive type can measure in more detail. For example, it is now possible to measure the thumb and index finger in the right hand, as well as the degree of force applied. In the future, our goal is to make it possible for people who have lost the ability to move freely due to spinal cord injury to perform actions such as “drinking water from a glass,” for example. We are conducting this kind of research in collaboration with external research institutions.

Researching “future communication technology” that conveys what you think or imagine to others as it is
Research on “communication technology of the future” that allows

Ishido: “In the theme of neurodiversity in the Brain World for Everyone, we are considering how society as a whole can create an environment in which each individual can live comfortably with the characteristics of the brain itself due to its diversity, including, for example, people with developmental disabilities and mental illnesses.

If there are any findings that Mr. Isesaki and Mr. Koike have obtained through their research that can be applied to the realization of neurodiversity, please let us know.

Mr. Isesaki: “When we measure brain waves, we find that there are differences among people even when they imagine the same kind of exercise. I think it is very diverse.

In this context, we are considering “which parts of the EEG can be modeled as common” and “which parts should be customized for each individual. At this point, we are not measuring the brain waves of people with developmental disabilities or mental illnesses, but we believe that we can expect to develop technologies that take into account both commonality and individuality, including these people.

Ishido: “In the field of sports, for example, I believe that top athletes such as Shohei Ohtani and ordinary people use their bodies differently. Is there a possibility that by measuring, comparing, and verifying brain waves, we can provide feedback to the individual so that he/she can move his/her body the way a top athlete does, thereby improving his/her athletic ability?

Mr. Isesaki: “This is an area we have yet to fully enter. It is also an area that I would like to explore in the future. The way you use your body will show up in your brain waves, and I am sure that the way you have a mental attitude, for example, how much you visualize the movement of a play just before a game, will also show up in your brain waves. Based on this premise, we would like to conduct research in the future to enhance the effectiveness of training by feeding back brain information.

Ishido: “Regarding future research directions, please tell us about any other research areas you would like to explore.

Mr. Isesaki: “We are considering the transmission of thoughts directly to the other person, as in the so-called telepathy, as a communication technology. We are working on research in this area, in which we do not explain things in words, but speak directly to the other person’s brain.

Ishido: “Whenever I hold a workshop to think about the future with children, at least one of them always comes up with the idea of telepathy, such as inputting information directly into the brain, or communicating with the person next to you using only the brain. What is the current level of technology to realize such a world?

Isesaki: “First, there are two directions in brain science research: research that measures brain waves and extracts information from the brain, and research that talks to the brain and inputs stimuli to the brain. Both types of research, invasive and non-invasive, are being conducted and various papers have been published.

Ishido: “What kind of things can be done at the cutting edge? For example, it seems that there are technologies that can capture images of what you envision in your mind, such as stave diffusion.

Mr. Isesaki: “Research in the U.S. has shown that electrodes are implanted in people’s brains and they communicate directly with each other through words, playing simple ping pong-like games, and so on.

Ishido: “As such research progresses and new ways of communication emerge, the possibilities for smoother communication increase, which is really wonderful for people who have had difficulty speaking due to some kind of disability.

On the other hand, if what you think, what you do not say, and even what you think but do not intend to say are conveyed, there are ethical considerations that should be taken into account and taken into account. How are such ethical provisions being discussed at the NTT Human Information Laboratories?

Mr. Isesaki: “In the scope of our EEG research, I believe that we are still in the research and development stage before we get to ethics, although I think it will become a problem in the future. I believe that ethical discussions at our institute are very strictly discussed in LLM and other AI-related research and development.

Koike: “I feel that we should consider the ethical aspects on two levels. First, we need to think about the research itself. I think it is important to think about it from the viewpoint of whether or not it is appropriate to conduct such research. Second, we need to consider whether we can deliver the results of our research to society and whether we can have an impact on society. It is important to think in these two stages, and we make ethical decisions by first discussing whether it is appropriate to conduct such research in the first place at the stage of starting the research. We have established an ethics committee within our department and have a system in place to consider whether or not it is appropriate to engage in research from a variety of perspectives before proceeding.

We also believe that our initial focus on this exercise support is also an area that has little impact from an ethical standpoint. We believe that research on voluntary movement, in which people who are unable to move their bodies as they wish due to some kind of disability are able to move freely using brain waves, will be more easily accepted by society. I hope that this will be the first step toward future research.

Ishido: “Thank you for speaking so clearly. Whenever a new technology comes out, there is a lot of discussion about its ethical aspects, so I felt that it is very important that you have organized your thoughts on this issue.

Koike: “We can discuss the first of the two phases within the institute, but when it comes to the second phase, social implementation, we need experts from various fields and actual users to participate and discuss what the use of brain waves should be like. I feel that it is necessary to discuss how brain waves should be used and how to proceed with the implementation of the two phases of society. I hope that events such as “Brain World for Everyone” will serve as a catalyst for such discussions and provide a forum for discussion with a wide range of people.

Mr. Isesaki: “I believe that in the near future, it will be commonplace to measure brain waves in our daily lives and to operate devices such as smartphones and games using brain waves. I would like to conduct research with the aim of creating a world in which people with various disabilities, including physical disabilities, can use various things more conveniently and live more comfortably.