Vivaldi? Blue grass? The Face Makes the Music

by ScienceDave | May 14, 2007 at 10:37 am | 732 views | 4 comments | 2 recommendations

A Rensselaer polytech (NY) student, Zane van Dusen, has developed a way to produce music with facial expressions.

The team designed and implemented a computer interface that tracks the movement of a user’s head to allow them to produce electronic sounds and compose music on a virtual keyboard in both solo and ensemble settings.

The device provides a much-needed outlet for creative expression for people with extremely limited mobility, particularly individuals with severe cerebral palsy (CP) — a neurological disorder that permanently affects body movement and muscle coordination and has the capacity to render people unable to speak or move. It also has therapeutic benefits, according to Van Dusen.

“We recently tested the adaptive use musical instrument in a clinic and noticed that many of the children were more focused on their movements because they were motivated by the sounds they were creating,” he said. “One child played the instrument for almost an hour, even though it took a lot of effort for him to keep his head up that long.”

recommend Add a comment
Brian A Kennedy
Brian A Kennedy
flagged this story as Good Stuff

at 04:24 on May 15th, 2007

Great stuff! Thanks for pointing this out.

0
thezane

Thanks for posting my story...very cool! One correction though: the instrument doesn't use facial expressions, it tracks small head movements (left/right, up/down). The software is meant to be used with basic webcams, which are super cheap, but lack the resolution necessary to monitor small facial movements.

0
jordan

Thanks for setting the record straight, Zane. It looks like a fascinating project.

0
kate

This is an interesting project. THe other person I know who is working in this area is Steve Dipaola. He's done a number of amazing projects that map music and audio information to faces in one way or another, although this might actually be an inverse project to yours Zane, as "MusicFace" anyway "creates facial choreography driven by musical input"? Check it out.

Add a comment

The content of this field is kept private and will not be shown publicly.

May 14, 2007 at 10:37 am by ScienceDave, 732 views, 4 comments

Vote for us at the Mashable Open Web Awards 2008

Crowd Power

Brian A Kennedy
First Flagged at 4:24 AM, May 15, 2007 by Brian A Kennedy
These members have powered this story:
 

closeSign in to NowPublic

is reporting from