About MM+Space


Kazuhiro Otsuka, Shiro Kumano, Ryo Ishii, Maja Zbogar, Junji Yamato, “MM+Space: n x 4 Degree-of-Freedom Kinetic Display for Recreating Multiparty Conversation Spaces”, In Proceedings of the 15th ACM on International conference on multimodal interaction (ICMI2013), Pages 389-396 , Sydney, Dec., 2013


A novel system, called MM+Space, is presented for recreating multiparty face-to-face conversation scenes in the real world. It aims to display and playback pre-recorded conversations as if the people were talking in front of the viewer(s). This system consists of multiple projectors and transparent screens, which display the life-size faces of people. The key idea is the physical augmentation of human head motions, i.e. the screen pose is dynamically controlled to emulate the head motions, for boosting the viewers' perception of nonverbal behaviors and interactions. In particular, MM+Space newly introduces 2-Degree-of-Freedom (DoF) translations, in forward-backward and right-left directions, in addition to 2-DoF head rotations (nodding and shaking), which were proposed in our former MM-Space system. The full 4-DoF kinetic display is expected to enhance the expressibility of head and body motions, and to create more realistic representation of interacting people. Experiments showed that the proposed system with 4-DoF motions outperformed the rotation-only system in the increased perception of people's presence and in expressing their postures. In addition, it was reported that the proposed system allowed the viewers to experience rich emotional expressibility, immersion in conversations, and potential behavioral/emotional contagion.

Demo Movie (2013 version)

Increased expressibility produced by translational motions

Supplement Movies (Fig.5)


Copyright (c) 2013, NTT Communication Science Labs., All rights reserved

overview movie

Fig.5(b) back-channel response; nodding

Fig.5(c) back-channel response; nodding

(c)ACM 2013. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 15th ACM on International Conference on Multimodal Interaction




author = {Otsuka, Kazuhiro and Kumano, Shiro and Ishii, Ryo and Zbogar, Maja and Yamato, Junji},

title = {MM+Space: N x 4 Degree-of-freedom Kinetic Display for Recreating Multiparty Conversation Spaces},

booktitle = {Proceedings of the 15th ACM on International Conference on Multimodal Interaction},

series = {ICMI '13},

year = {2013},

isbn = {978-1-4503-2129-7},

pages = {389--396},

url = {http://doi.acm.org/10.1145/2522848.2522854},

doi = {10.1145/2522848.2522854},

acmid = {2522854},

publisher = {ACM},

address = {New York, NY, USA},

keywords = {computer-mediated communication, face-to-face conversation, multimodal interaction, projection mapping, telepresence},