NTT Communication Science Laboratories Innovative Communication Laboratory08




Agents that Encourage Emotions and Intelligence

Realization of agents that interact with people from one situation to another

Personal contact with people and nature can produce tranquility and a sense of warmth. Occasionally, quite by chance, we experience the pleasure of intellectual insight or learn something surprising and extraordinary. During calm reflection, we realize that the accumulation of such slight events enhances our lives. Yet faced with such social developments as urbanization, computerization, smaller nuclear families, and aging societies, people tend to feel more stress and unease in their lives. To extricate ourselves from the unilateral devotion to the values of efficiency, convenience, and rationality, we must conceive of completely new perspectives that will contribute to richer lives. Our research focuses on agents that interact with people from one situation to another to realize such perspectives. In this study, the agents understand people and environments from such information as images, speech, sounds, and language and control their lines of vision, face direction, and speech based on those people and environments. By razing the barriers surrounding existing research disciplines and strategically developing transdisciplinary areas to pursue this objective, we envision a new research field that involves everything from signals, media, language, and knowledge processing to developmental psychology and perceptual neuroscience.

This project aims to stimulate conversation environments under multi-modal and multi-party communication to achieve free and natural conversation among people and agents. We define free and natural conversations as dialogs that are liberated from fixed scenarios and emerge based on different conversational situations, consequently generating freely asynchronous communication among people and agents. These communication functions will be available to applications called robot collaborative classrooms where agents support the learning of children by directly and indirectly encouraging thought. The agents support the establishment of a new social system for teaching children at home while addressing declining birthrates.


■Action Control for Changes of Human Emotions and Actions in Real Time

media_9_2e.jpgAgent acquiring human niceties Beyond the basic emotions of anger, surprise, disgust, sadness, happiness, and fear, people also express a great range of feelings including relief, foreboding, skepticism, uncertainty, confusion, and impatience. Just as it is important in everyday life to casually give thoughtful consideration to others, technology that treats emotions is indispensable to endow agents with the ability to reduce anxiety and promote well-being. In addition to collecting a diverse range of expressions and vocalizations that are produced under various conditions, we have also extracted reliable features that express emotions from face movements and such non-verbal utterances as laughter, sighs, intonation, and stress. We are creating novel models that respond appropriately to the changes of emotional or psychological states. Understanding human and environmental situations and acting with appropriate timing simultaneously require capturing, analyzing, and integrating multi-modal information. In addition, robustness is required to deal with the information and the calculations in real time. Toward this objective, we are researching a multi-modal action control system by taking clues from sounds, speech, facial expressions, language, and other human actions in terms of multi-modal distributed and multi-modal integrated processing.

■Generating Agent Actions that Influence Human Emotions and Thought

media_9_3e.jpgthought-evoking multi-party dialogue For conversational agents to mediate the interaction between people and their information environments, the agents must effect relevant changes in human emotions, feelings, and thought. We are mainly concerned with such agent action generation and are now focusing on human communication stimulation in thought-evoking multi-party dialogues. In such dialogues, multiple human participants and agents engage in quiz-style interactions. By presenting hints in an appropriate order and expressing relevant emotions, agents can stimulate human thought and communication. By exploiting research works on human emotion recognition and emotional speech synthesis, we hope to create a computational model for generating agent actions that can influence human emotions, feelings, and thought with speech, gestures, and other modalities.

■Human Science for Developing Telecommunications Environments

As such social issues as the burgeoning digital divide become more evident, one objective of ambient intelligence is to create an information communication environment that everyone can readily enjoy. Such consideration is especially crucial for senior citizens and children who tend to be information have-nots in the information society. Aiding these members of society requires a deep understanding of their nature. Building on previous research results regarding the language development of children, we are researching how agents learn actions as entities that mediate among people and their environments.