This project is focused on exploring body motion such as bowing, plucking, stroking, hitting, shaking, as well as various motion of the vocal apparatus, that go into producing musical sound. The project is motivated by the idea that sound-producing body motion is crucial not only for generating sound that we hear, but also for how we perceive and interpret musical sound. For instance, when we hear ferocious drumming, we may tend to also imagine energetic motion of mallets, hands, arms, and even of the whole body of a drummer, and when we hear soft string music, we may tend to imagine slow, protracted bowing motion. The project has been much influenced by so-called motor theory, a theory that claims perception of sound is closely linked with images of body motion producing the sound. In our research over the last decade, we have in fact come to see music as a fusion of sound and body motion.
In this project, we are making close-up studies of musicians' sound-producing body motion and correlating features of such motion with the acoustic features of the resultant musical sound. To this end, we use various methods for recording musicians' motions such as infrared motion capture, video, and various sensors, and methods for processing the recorded data and correlating such data with sound features using various statistical methods.
A workshop on sound-producing body motion was held in November 2015.