Skip to the content of the web site.

Karen Collins

Sound, Semiotics and Software: Canada Research Chair Karen Collins investigates both the theory and practice of interactive audio

Having just arrived in town, Karen Collins is already at work getting her lab set up and her pencils sharpened in readiness for the work ahead.  Joining the Canadian Centre of Arts and Technology (CCAT) as its new Tier II Canada Research Chair in Communication and Technology, Collins’ research program is ambitious:  not only will she be conducting the theoretical work of developing a semiotics of sound that better accounts for the increasing interactivity of new media, but she will be using this theory as a basis for developing a software engine capable of composing real-time audio for interactive multimedia applications (e.g. for educational software, video games, interactive museums and the like).  Collins’ agenda, thus, stands as a model of both basic and applied research in the Arts.

Collins is no newcomer to the theoretical side of sound, and she comes to Waterloo well versed in the field of music semiotics (the study of music as a symbolic language).   In her doctoral studies at the University of Liverpool, Collins sought to better understand the function of sound and music in film; however, she found the existing semiotic models to be wanting.  “With a few exceptions, the vast majority of work in music semiotics has focussed on music as an entity separate from other forms of expression,” she explains.  “The semiotics of sound in media is a relatively new and as yet largely unexplored field.  It is a field that was opened up in 1979 with the publication of Philip Tagg’s groundbreaking study, Kojak: 50 Seconds of Television Music, a book that updated traditional musicology to account for the impacts of popular music and mass media.” 

Working with Tagg, and with a research focus on the role of music and sound in science fiction cinema, Collins succeeded in expanding existing models and developing them so that they would  better account for and accommodate the additional layers of meaning and connotation generated by mixing visual and auditory cues.  In her post-doctoral studies at Carleton University, Collins took the next step by moving from linear cinema to interactive media.  As Canada Research Chair in Communication and Technology, Collins will continue this research, further refining her models to account for the impact of sound in media environments where audiences play an active role in co-creating the audio-visual experience.   

This is no simple task.  Interactive media’s non-linearity (and hence unpredictability) is a challenge not only to semioticians but to designers as well.  On top of this, there is the fact that sound and music have a deep connection to human emotion and can produce profound, if sometimes unrecognized, affective impacts on media users. Collins hopes to develop a system that will begin to codify some of these affective dimensions of sound.  By doing this, she will not only pave the way to better understanding their function in interactive media, but she will be laying the groundwork for media developers to more seamlessly, and more effectively, incorporate sound elements into their products.  Eventually, the plan is to use these findings to create a sophisticated software engine capable not only of selecting appropriate segments of audio in real time, but of composing, on the spot and in response to user cues, affectively appropriate and musically coherent sequences of audio for interactive applications.  To do this, Collins will work with technical experts but also draw upon her own extensive industry experience as a web developer and digital designer.

Collins’ research couldn’t come at a better time.  As well as playing a central role in the multi-billion dollar video games industry—which has been surprisingly slow in incorporating research’s latest findings on sound-- interactive audio is increasingly being adopted in a wide range of other applications, including computers, cars, mobile phones, web sites, theme parks, museum exhibits, education centres, and shopping checkout systems.  “It’s crucial that humanities and social sciences researchers participate in and help shape this burgeoning new field of media and communications,” says Collins.  “My research will help bridge the gap between academia and the new cultural industries.” 

 Karen Collins is an assistant professor in UW’s Department of Drama and Speech Communication and CRC in Communication and Technology at the Canadian Centre of Arts and Technology (CCAT).  Funding for the new interactive audio lab will be provided through a CFI infrastructure grant of over $100,000. Karen Collins can be reached at collinsk @uwaterloo.ca                         

Angela Roorda
Summer 2007