If visual cues can help musicians create music for film, they can also be used by non-composers, students of film and film critics to interpret the relationship between the music and moving image. I saw this as an opportunity to revisit the original interface and to apply color and iconographic shape as meaningful cues in film study.

Films are broken down into scenes. This is an ideal place to compare films or between places in a single film. There are four separate scenes that deal with the death scene in Twelve Monkeys, so I started by putting all four scenes in the four quadrants of the computer screen. I then placed a typical playback mechanism with a timeline below each of the films and assigned each of the scenes titles. I could then review each of the films separately with their own playback devices or by using a master playback section found in the upper left hand corner.

On the timeline itself, I created a system in which I could click on the timeline and create a cue tag. A cue tag is a duration of time on the timeline with an associated color and name. A cue tag could have been placed just about anywhere and for any purpose, but I chose to use music and sound as my guide.

If I placed the same named cue tag on other scene, the same color would appear in that scene’s timeline. When I placed all of the tags, I could play each film separately and watch as the corresponding-colored streamer came across, or click on the colored cue tag on the timeline. When that happened, each film that contained that cue tag would snap to the beginning of the same section. All other scenes would disappear.

When I clicked on the master play button with all four airport scenes of Twelve Monkeys on the “gunshot” cue, I was surprised to hear (and see) that both the beginning and the end scene music were almost the same arrangement with the same tempo. I also noticed that in all four scenes, the death sequences were played out of time with one another (intentionally, I gather, by the director Terry Gilliam).

I learned about using color and moving shape to show associations by making the tool and by studying the films’ content through the interface. However, I didn’t get at how to use the tool as a way of understanding the direct relationships between sound and image.

I wanted to explore more with the possibilities of using space as a more natural environment with image and sound, the way I had intended to with the panorama scroll interface, but with more user control, the way I had done with the A.S. Mixer.

contents ©2009 colin owens