| 21 |
Understanding the Mind from Eyes and Brain in the WildScaling mind reading beyond the lab 
|
|---|
Humans act in rich and complex everyday environments, but it is still difficult to understand how the brain works in such situations and whether a person’s mental state can be estimated from natural behavior. This exhibit introduces a series of studies that extend mind reading beyond the laboratory by using eye and brain signals measured during daily activities such as driving, cooperative gameplay, and video viewing. Based on knowledge from controlled laboratory experiments, we show that it is possible to estimate higher-level mental information in more realistic settings. Examples include detecting attentional state and concentration during natural driving, finding cooperative states from eye-related signals during team play, and decoding perceived or imagined video content from brain activity. In the future, this approach may support better communication and adaptive assistance by adjusting information presentation and interaction to each person’s state. In driving, for example, it may help driver monitoring systems detect inattentive states.
[1] J. Yamashita, Y. Suzuki, F. Nakanishi, M. Yamataka, H. Terashima, K. Maruya, “Finding Workload in the Wild: Recovering Latent Cognitive Dynamics Jointly from Eye Metrics and Driving Telemetry,” in Proc. The 2026 ACM Symposium on Eye Tracking Research & Applications (ETRA2026), accepted.
[2] H.-I. Liao, M. Montemayor, K. Haly, M. Kashino, S. Shimojo, “Interpersonal pupil synchronization during high-engagement team plays,” in Proc. The 47th European Conference on Visual Perception (ECVP2025), 2025.
[3] T. Horikawa, “Mind captioning: evolving descriptive text of mental content from human brain activity,” Science Advances, Vol. 11, No. 45, eadw1464, 2025.
Hiroki Terashima, Sensory Representation Research Group, Human Information Laboratory