"[Robyn Taylor] sings the blues ... and the reds and the yellows"

-- St. Albert Gazette

I am studying how virtual reality technologies can be applied to live musical performance in order to create an immersive performance environment. Read an article the St. Albert Gazette wrote about my research. Listen to a radio interview I did with CBC.

I became interested in working in the Advanced Man-Machine Laboratory after observing the novel and creative projects the Laboratory encourages. I began my participation in lab activities as a research assistant in early 2002, then enrolled full time in the Master's program in order to begin working on my own project under the supervision of Dr. Pierre Boulanger.

My research is focused upon finding ways to visually illustrate aspects of musical performance. I have spent much of my life studying and performing as a singer, so I am especially interested in ways to visualize the human singing voice. I would like to be able to use the extraordinarily immersive technologies that we have available to us in this lab to create a multimedia performance space that can provide visual accompaniment to live musical performance.

My music visualization research focuses on two types of concerns:

• What features are present in a piece of sung music?
• What notes are being sung?
• How are they being sung? Loudly? Quickly? Slowly?
• What kind of voice is singing them?
• How can we illustrate these features using visual imagery?
• Should we map each sung feature directly to a visual feature?
• Should we create a more abstract mapping?

I have worked for the past year on addressing these questions.

To isolate and identify musical features in live performance, I have developed a program which extracts musical features from sung vocal input. I am able to determine what notes the singer is singing, how loudly she is singing them, and how they harmonically relate to the key signature of the piece. I am also able to generate a description of the singer's 'vocal timbre' or 'vocal colour'.

I have then attempted to illustrate these extracted features using computer-generated imagery, mapping musical input to visual feedback. I combine live music with responsive graphics in order to create multimedia performance pieces.

Acknowledgements

We gratefully acknowledge Alias Systems for providing us with the Maya software used to create our 3D models.

Publications

Visualizing Emotion in Musical Performance Using a Virtual Character
Robyn Taylor, Pierre Boulanger, and Daniel Torres
To be presented at Smart Graphics 2005
Munich, Germany.

Using Music to Interact with a Virtual Character
Robyn Taylor, Daniel Torres, and Pierre Boulanger.
Presented (in poster format) at New Interfaces for Musical Expression 2005
Vancouver, Canada.