Designing a Society for Human-System Coexistence through Dialogue with AI Agents

The Neurodiversity Project, organized by B Lab, aims to create a society where neurological differences are respected and everyone can reach their full potential. In this installment of our interview series, we feature the work of Professor Daisuke Katagami (▲Picture 1▲) from the Information Technology Course, Department of Engineering at Tokyo Polytechnic University, who debuted his exhibit at “The World in Our Minds” in 2025. At the Katagami Laboratory, the team seeks to design a world where humans and systems coexist through interactive communication between people, robots, and digital systems. Nanako Ishido (▲Picture 2▲), Director of B Lab, sat down with Professor Katagami to discuss his innovative research.

> Interview videos are also available!

Family caregivers of dementia patients.
How to reduce the emotional burden

Ishido: “In “The World in Our Minds” in 2025, you introduced your research with the interesting titles “Living with Grandma” and “Job Interview Simulation”. Could you tell us about the contents of each exhibit and your usual research?”

Dr. Katagami: “First, let me explain about “Living with Grandma”. This allows users to experience scenes of living with an elderly person with dementia through dialogue with an AI agent. By learning how to behave when faced with a troubling situation while living with a person with dementia in a simulated situation, users can acquire the ability to respond, empathy, and knowledge about caregiving.

Behind this is Japan’s aging society. In particular, the number of dementia patients is increasing every year, and data indicate that the number will exceed 7 million by 2025. (▲Picture 3▲)

Under these circumstances, one of the major problems is the burden on family caregivers of dementia patients. In fact, many family caregivers feel a mental burden, with a survey showing that as many as 86.9% of family caregivers feel a burden. (▲Picture 4▲)

Family caregivers’ lack of knowledge about dementia and their inability to understand and empathize with dementia patients are considered a major cause of emotional burden. Among them, many family caregivers are troubled by BPSD (Behavioral and Psychological Symptoms of Dementia) of dementia patients. (▲Picture 5▲)

Therefore, I wanted to support family caregivers to reduce their emotional burden by learning about dementia from a young age and learning how to alleviate BPSD of dementia patients.

Focusing on “listening” which is emphasized in the field of nursing care
and developed a system that enables training.

Specifically, we focused on the caregiver’s “listening. Listening is an act in which a caregiver speaks to a dementia patient to confirm his or her wishes, listens to the patient’s voice, and accepts, empathizes with, and respects the patient’s wishes without denying them. We worked on the development of a training system for family members caring for dementia patients to learn this listening skill. (▲Picture 6▲)

The goal is that by learning to listen, caregivers will learn to understand the patient’s emotional state and deepen their understanding of dementia, thereby improving their attitude toward the patient and enabling caregiving behaviors that alleviate BPSD. In fact, more experienced caregivers are better at listening, and we wanted to help family caregivers develop this skill, so we created a system that would serve as a precursor study. This training system creates scenarios based on several typical cases in caring for dementia patients and reproduces them on the system to learn knowledge while actually dealing with an AI agent. (▲Picture 7▲)

We interviewed caregivers who had actually used the system and found that their thoughts differed depending on their years of experience. The more experienced caregivers were able to provide care based on listening, while the less experienced caregivers were trying to “persuade” the dementia patient. With more experience, they became able to provide care based on listening and found that they were able to successfully deal with the patients. (▲Picture 8▲)

We decided to convey this to family caregivers through their experiences, and exhibited it under the name “Living with Grandma”. (▲Picture 9▲)

I will explain in detail what kind of system it is. In the actual nursing care field, it is desirable to confirm the intention of the patient before providing care, and to talk to the patient. Therefore, we first used the OpenAI API to determine the caregiver’s behavioral sentences. For example, “I’m going to brush my teeth now! May I?” or “Do you want to go to the bathroom? Refracing is also desirable, in which the caregiver expresses empathy by saying, “You want to go to the bathroom,” when the patient says, “I want to go to the bathroom. (▲Picture 10▲)

On the other hand, there are also undesirable voice calls. For example, “What are you doing? or “You can’t do that! or “Don’t do that!” are undesirable (▲Picture 11▲).

Based on this data, in “Living with Grandma”, they can experience the voice calls themselves and get feedback on those calls. For example, this is how it works,” he says.

*Demonstration of the system below

System: You have finished breakfast, so brush your teeth.

System: Grandma, let’s brush our teeth.

Granny:…….

System: You are trying to get your grandmother to brush her teeth, but she won’t open her mouth. Think of a way to talk to your grandmother to get her to brush her teeth.

Caregiver: Please open your mouth so I can brush my teeth.

System: He seems scared. Let’s talk to him again.

Caregiver: why are you scared?

System: It seems I forgot to brush my teeth. Let’s call out to him again.

Caregiver: It feels good to brush your teeth.

SYSTEM: Apparently the grandmother remembered to brush her teeth.

System: apparently he forgot to brush his teeth and was afraid to brush his teeth.

SYSTEM: This is a toothbrush to brush your teeth. Let’s clean your mouth.

System: Granny started brushing her teeth. The grandmother did not seem to recognize breakfast or brushing her teeth. Dementia has a symptom called ‘agnosia,’ in which an object can be seen but not recognized. It seems that the grandmother had trouble recognizing food as something to eat and was afraid to brush her teeth.

System: This concludes the scenario.

We will now review your actions. You may want to try to talk to your grandmother a little more to reassure her. In caregiving trouble situations, dementia patients are often confused.

*This is as far as I’ve gotten.

Dr. Katagami: “In this way, the system allows the user, the caregiver, to experience various scenarios with feedback based on what the caregiver is responding to. The system I showed you had text input, but in reality, the user speaks to the system by voice. The system is designed to determine the humanistic qualities that are important in caregiving, depending on how the user speaks to the system. Our work on this system to date has received the ‘Neurodiversity Award’ from the Neurodiversity Awards.

Through dialogue with an AI interviewer
through a dialogue with an AI interviewer.

Another exhibit, “Job Interview Simulation”, is a system that allows users to experience a job interview through dialogue with an AI agent interviewer (AI interviewer). The system not only allows visitors to practice self-introductions and self-appeals under nervous situations, but also supports the AI interviewer in asking in-depth questions in response to what the interviewee has said. (▲Picture 12▲)

Generally speaking, there are six competencies that you should be aware of during the interview process. These are the power of self-change, the power of relationships, the power of commitment, the power of creativity, the power of problem solving, and the power of teamwork. (▲Picture 13▲)

Base questions are prepared according to these contents, and AI interviewers ask questions based on them, and further in-depth questions are generated in real time using GPT-4o in response to the interviewee’s responses,” he said. (▲Picture 14▲)”

*The following is a demonstration of the system (▲Picture 15▲)

AIInterviewer: Now, please tell us more about the part-time job you focused on during your school days.

Interviewee: Yes. The part-time job I had in college was sorting convenience store products in a warehouse.

AIInterviewer: What is the most valuable skill you gained from that part-time experience?

Interviewee: Yes. At that workplace, you can visit …… per hour.

*This is as far as I’ve gotten.

In this way, the AI interviewer asks in-depth questions in response to the interviewee’s speech, allowing the interviewee to practice for the interview. It is difficult to create the tension of an interview, but it is known to be effective to a certain extent. We think that the system will be even better if we can express the tension better. This is an issue for the future.

“A human-friendly and considerate AI
The day is near when such an AI will appear.

Ishido: “I hope that the two exhibits were experienced not only by specialists but also by general visitors. What kind of reactions and insights did you receive from many of those who actually experienced the program?

Dr. Katagami: “There were many questions about how the system works and how it operates. There may have been students and researchers,” he said.

Ishido: “Dementia and interview simulation may seem like completely different topics at first glance, but I believe that you have a common research axis. What is that?

Dr. Katagami: “The AI agent provides some kind of assistance to the user. That is the axis of the system. The two systems are quite different in terms of the areas in which they are actually used, but they are quite close in terms of how they work. Both systems have a typical scenario and provide feedback based on the user’s reaction to the scenario. This direction is the same. As long as we have the right content, I think we can apply the system to a variety of areas and provide various kinds of support.

Ishido: “For example, I feel that the attitude of ‘listening’ that is emphasized in dementia caregiving is also an extremely important skill in the fields of child-rearing and education. From this perspective, this research is not limited to dementia caregiving, but could be developed as a more versatile study that focuses on the theme of “training to learn listening” itself. From this perspective, what kind of development do you envision for future research and social implementation?

Dr. Katagami: “In fact, through the “Neurodiversity Award,” we have received proposals for collaborative research from people in other fields. In other words, I believe that horizontal development is also possible. However, horizontal development requires information as a base. In fact, it is difficult to collect this information, and it is difficult to develop a system unless you are familiar with the field in the first place. If we can work with members who have specialized knowledge, it will be easier to achieve horizontal development.

Ishido: “For example, the importance of ‘listening’ itself can be conveyed through books or lectures. However, in this exhibition, I felt that the emphasis was placed on the participants’ own awareness of the importance of listening through experience, rather than on one-way presentation of the correct answer, through dialogue with an AI agent. We would like to hear your thoughts on the design philosophy behind this approach of “learning through dialogue with an AI agent.

Mr. Katagami: “As you say, it may be more efficient to read a book just to gain knowledge. However, we believe that learning through one’s own experience is preferable, in other words, “when you realize you have acquired knowledge,” and we emphasize the experience of actually interacting with the AI agent.

Ishido: “I think the “Job Interview Simulation” will be an initiative mainly for students. In the neurodiversity project we are promoting, we believe that individual personalities and characteristics, as well as differences in the way we perceive and recognize things, have a great impact on the interview process. Based on this premise, is it possible to use AI agents to identify trends in the way interviews are conducted and the way people interact with each other that are suited to each individual?

Dr. Katagami: “At the moment, the system is not designed to respond to the user’s personality. However, personalization, interviewing according to individual personalities, etc., are important keywords.

We are also researching driver assistance for automobiles. This is a driver assistance agent placed next to the driver’s seat that talks to the driver in various ways while the driver is driving, but actually changes the way it speaks to the user, or driver, depending on his or her personality.

Driving is an activity that makes a big difference in one’s personality, and you have probably heard people say that ‘your personality changes when you get behind the wheel. For example, whether or not a person can accept advice (instructional speech) such as ‘look to the right or it will be dangerous’ (receptivity) varies greatly from person to person in terms of age, gender, and personality. In addition, acceptability also depends on who is speaking to you, so I think a system personalized according to personality would be very effective in situations such as driving.

“Job Interview Simulation” aims to ‘bring you up to a level where you meet the criteria for being rated as good by companies. We have not yet reached the stage of providing individualized support.

Ishido: “This means that individualized support can be fully realized in the future.

Dr. Katagami: “I think that more advanced interviews can be expressed, and perhaps this could be an interesting system for corporate HR. It could serve as a simulation for interviewers, such as ‘what questions to ask students with different personalities,’ or ‘how to talk to students with different personalities.

Ishido: “It seems that the characteristics of each student can be visualized through interviews, which may have a positive impact on post-hiring personnel assignments and team building. Recently, more than a majority of students feel more comfortable talking to an AI than to their parents or friends. On the other hand, what do you think of this trend? Especially in the context of interview practice, what are the positive aspects of working with AI, and what should we pay attention to?

Dr. Katagami: “You are right. There is an advantage that ‘it is easy to talk because it is AI,’ but there is also a disadvantage that ‘AI does not make me nervous’ (even though I want to practice tense situations). In this case, we are targeting students who have difficulty with basic things like self-presentation, so the concept is to use it casually and practice without making them too nervous. In universities, students practice interviews with counselors, but this is positioned as a preliminary step for them to casually talk about what appeals to them to an AI partner,” he says.

Ishido: “Dr. Katagami, you are also involved in research and development of AI that compliments and cuddles people.

Dr. Katagami: “In AI development, I think we must be concerned about both positive and negative aspects. For example, we may be happy to be pampered and praised by AI, but it would not be good if our social life goes wrong as a result. For example, in the driver assistance scene, speech from a human instructor is the most acceptable, and speech from a robot is also said to be highly acceptable. On the other hand, many people find it difficult to accept when someone close to them, for example, their partner, says things to them from the passenger seat. Since there are various influences by the passenger (passenger effect), we would like to extract the elements that have a more positive impact and successfully provide things such as praise and guidance to people.

It is not enough to praise well, nor is it enough to instruct well; if we can create an AI with a high level of social skills and consideration that can be tailored to provide for others as appropriate, we think we are probably headed in the right direction.”

Ishido: “How many years do you think AI capable of such considerations will be available?

Dr. Katagami: “I wouldn’t be surprised to see it implemented in the not-too-distant future, in five to ten years. In fact, research is being done on statements based on advanced consideration. Japanese people have strengths in areas such as social consideration and hospitality, so if we can successfully incorporate this into the system, creative development may accelerate. That is interesting. Recently, with the development of LLM (Large Language Model) technology, implementation has become easier. If we make good use of the technology while checking its effectiveness, I expect that it will be incorporated into the system sooner than expected.

AI from “Tool” to “Partner
Advancement and diffusion of technology will change the “norm” of society.

Ishido: “That is a very interesting story. As you mentioned earlier, Japanese society has a cultural quality of valuing consideration for others, but there are many people who find it difficult to communicate with others based on tacit understanding and reading between the lines. For those who have such characteristics, it is difficult to communicate with others in a way that is intended for them. For people with such characteristics, I feel that “Considerate AI,” which assists them in understanding the intentions of others and the atmosphere of the place, could be an effective means of communication support.

In this regard, what do you think is the role and potential that “Caring AI” can play in the implementation of neurodiversity?

Dr. Katagami: “Even if you are not a good communicator, I think it is possible for you to acquire skills through experience. This may be off your question, but I am currently doing research with Chinese students. We are creating a system to help them understand cultural differences, as it is difficult for Chinese people to learn about Japanese culture. This is to train social communication skills, but I believe that by learning this, they can learn cultural differences and behavior”.

Ishido: “It is very interesting to see the perspective of using it as a tool to help people from different cultures and backgrounds understand each other. While there are cases, such as those of international students, where social skills training makes life easier for the individual and those around them, I think that an important aspect of a neuro-diverse society is not to forcibly fill in differences, but to establish a society based on the premise that there are differences. In this light, AI can not only be used as a tool for social skills training, but can also be used as a “translator” between people, assisting and complementing their intentions and contexts.

What are your thoughts on this point from the perspective of Dr. Katagami’s agent research?”

Dr. Katagami: “Our research on a system for family caregivers of dementia is exactly in line with that idea. Our goal is not to try to do something about dementia patients, but to change the awareness of people around them, so I think our approach is similar to what Ms. Ishido mentioned about changing the way people around them think and perceive the disease.”

Ishido: “I felt that this hands-on exhibit was designed because the difficulties faced by people with dementia are not found on their own side, but rather in their relationships with those around them and the way their environment is designed.

From this perspective, I believe that in order to realize a neuro-diverse society in the future, it will be more important to establish a society in which differences are respected as they are than to enable people to do what they cannot do. In order to realize such a society, what roles and possibilities do you expect AI and other technologies to play?

Dr. Katagami: “This is a difficult question, but I hope that we can change common sense. It may be similar to changing the surroundings I mentioned earlier, but when the fundamental way of thinking in the world in general changes, it can bring about many significant changes. I used to do research called ‘atmosphere engineering. It is an engineering study of the results that can be brought about by changing the atmosphere. For example, when we think about peer pressure, if it works in the right direction, everyone can move toward a single goal, but if it works in the wrong direction, it can lead to bullying. In our research, we found that when there is peer pressure, if even one person says, ‘That’s not true,’ the atmosphere softens. In other words, by placing a few androids in the classroom and moving them around intentionally, it is possible to alleviate certain peer pressure and change the overall atmosphere of the class.

The same is true of this system, and technology can make great strides as people’s general perception of it changes. For example, AI agents are still seen by humans only as a tool, but as they gradually become an existence that we cannot let go of, they will turn into partners. As people develop a sense of trust in AI agents, they may change what they consult with, how they use them, and how they talk to them. As trust is created, what they can do will change, and the content of their services will develop in various directions. In this sense, I feel that changing general awareness is an interesting research goal.

Ishido: “What we value in the Neurodiversity Project is to “create a new normal. Rather than piling something on top of the existing assumptions, we question the common sense itself, and in fact, we may be able to present other options and a variety of alternatives. We emphasize this attitude. In terms of changing this common sense and creating a new common sense, I felt that the concept of the professor is very compatible.

Also, you mentioned earlier that the presence of an android in the classroom changes the atmosphere of the place. In my own activities in the field of edtech, I had the experience of introducing a “transfer student” to an elementary school class when the use of generative AI was just beginning to spread. I was very impressed by the change in the children’s comments and discussions, and the very atmosphere of the place itself. We would like to hear your thoughts on what other examples and possibilities you can think of for using technology to “change the atmosphere” in this way.

Dr. Katagami: “Basically, humans have a tendency to anthropomorphize various objects. It is called “Media Equation,” and systems and AI also tend to be treated anthropomorphically. You mentioned earlier that you brought AI into the classroom as a transfer student, and I think the discussion changed because you treated AI statements as the statements of a transfer student, a human being.

We are studying HAI (Human Agent Interaction), and we believe that agents can interact and work in a way that has some influence on humans. However, whether the influence of agents on humans leads to the maintenance of diversity is not technically possible at this point. Agents themselves can be easily replicated, so it is not technically difficult to create agents with various personalities. However, research has not yet progressed on how this will affect people, so I think this will be a theme for the future.

Ishido: “Finally, what are your aspirations for future research and what message do you have for the realization of a neuro-diverse society?

Mr. Katagami: “I had a very valuable experience at this exhibition and received many positive encounters and comments. I was able to make connections with people in different fields, and I would definitely like to participate again in the next and subsequent years.

As for my future research, I would like to help people in need while successfully harnessing the power of AI and agents. To realize a neuro-diverse society, it is important to change human consciousness. I would like to promote research that can work well from that perspective as well.

Ishido: “I think that Dr. Katagami’s technology is a very warm technology. Through this exhibition, I felt that technology can be like a mirror that reflects us. Through this mirror, each of us can question the common sense we have taken for granted and seize the opportunity for change. I realized once again that as the number of such people gradually increases, the common sense of society as a whole will eventually be updated.

Thank you very much for your valuable talk today.”