The brain learns a relation between sounds and visual motion within a short period of time
- Contributing to the developments of multimedia technologies with high sense of presence and sensory substitution systems -

[Points]
The brain learns a relationship between visual and auditory signals without noticing, such as someone’s face and voice, and a musical instrument and its sounds. This mechanism is found to be useful in daily lives. However, it has been unknown how and how fast the brain connects and learns the relationship between visual and auditory signals. This study presented the pioneering evidence showing that strong association between sound sequence and visual motion is easily formed within a short period and that, after forming the association, sounds are able to trigger visual motion perception for a static visual object. This effect continued at least for a couple of days. The findings will dramatically accelerate the research regarding the multisensory integration mechanisms in the brain and the development of multimedia technologies with high sense of presence.

[Summary]
Two white circles placed side by side were presented in alternation. The onset of the two circles was synchronized to a tone burst of high and low frequency, respectively. After exposure to the visual apparent motion with tone bursts for 3 min, a circle blinking at a fixed location was perceived as lateral motion in the same direction as the previously exposed apparent motion, when the flash onset was synchronized to the tones. Furthermore, the effect lasted for a considerably long time, at least for a few days. The study was conducted by Wataru Teramoto (Postdoctoral Researcher) at Research institute for Electrical Communication and Graduate School of Arts and letters, Tohoku University, Souta Hidaka (Assistant Professor) at Department of Psychology, Rikkyo University and Yoichi Sugita (Senior Researcher) at National Institute of Advanced Industrial Science and Technology (AIST). The findings were published on line as a paper in Public Library of Science (PLoS) ONE on August 20, 2010.  The title of this paper is: Sounds move a static visual object.

More Information (Japanese)

[Contact]
Wataru Teramoto
Graduate School of Arts and Letters, Tohoku University
Address: 2-1-1 Katahira Aoba-ku Sendai, Miyagi, 980-8577 Japan
TEL: +81 22 217 5461
Address: 27-1 Kawauchi Aoba-ku Sendai, Miyagi, 980-8576 Japan
TEL: +81 22 795 6048
E-mail: teraw*ais.riec.tohoku.ac.jp (Replace * with @)

Souta Hidaka, Assistant Professor
Department of Psychology, Rikkyo University
Address: 1-2-26 Kitano Niiza-shi, Saitama, 352-8558, Japan
TEL: +81 48 471 7041
E-mail: Hidaka*rikkyo.ac.jp (Replace * with @)

Yoichi Sugita
Systems Neuroscience Research Group, Human Technology Research
Institute, National Institute of Advanced Industrial Science and
Technology (AIST)
Address: 1497-1 Kashiwayama Teragu Tsukuba, Ibaraki, 300-4201 Japan
TEL: +81 29 869 1921, FAX：+81 29 869 1904
E-mail: y.sugita*aist.go.jp (Replace * with @)