An Audiovisual Experience Combining Sound Synthesis and Music Production

Department of Computer Science and Engineering

Project Abstract

The goal of our project was to create a virtual environment where users could synthesize new sounding instruments, learn about music theory concepts such as key signatures, common chords, and how to play music.  In our vision, the environment would be highly responsive to the qualities of the sound the custom instrument produces, as well as the user’s musical production.  The user could play the instrument manually, using on-screen buttons, or add notes to a “sequencer”, a common interface seen in computer-generated music software.  The programming language we have used to create the sounds and play notes is ChucK.  ChucK (the capital ‘K’ is intentional) is a strongly-timed language, an eloquent term used to describe the manner in which the virtual machine, the language runs on, plays sounds precisely when the program specifies, and for how long it specifies.  The language makes it easy to generate complex waveforms by stacking various frequencies on top of each other. 3ds Max, which is a 3d modeling software, was used to create 3d components for our virtual environment. 3ds Max is user friendly, and versatile making it the ideal software for creating a visually appealing environment. Unity is the interface that we used to combine, and create our interactive environment from the components we created in ChucK, and 3ds Max. Using Unity’s logical interface we applied the research we did in music concepts to create the virtual environment that helps the audience learn more about music.

Project presentation