LSO music 'motion captured'

The transformation of Sir Rattle's performance to visual art

London Symphony Orchestra has more digital recordings than any other orchestra in the world. Since opening in 1904, they have gone from gramophone recordings to digitised film scores. Now, after 113 years of classical music, the orchestra is moving into new territory: forming a visual identity with motion capture technology.

At a Barbican rehearsal in June 2016, Sir Simon Rattle, music director of LSO, donned a motion capture suit and gloves to conduct a performance of Elgar’s Variations on an Original Theme, ‘Enigma’. Using a specially modified conductor’s baton with reflective markers, a motion capture team from the University of Portsmouth was able to track Rattle's upper body, arm, hand and finger movements from the beginning to the end of his performance, over a 45 minute period.

Over the course of Rattle's performance, 12 motion capture cameras were used to record 120 frames per second. Using these recordings, it was then possible for digital artist, Tobias Gremmler, to transform this data into a series of animated films, bringing to life the sheer power of music at full force and in startling colour. The result is a vortex of wood, brass, smoke and strings, all derived from Rattle's sweeping gestures.

Alex Counsell, principal technician at the University of Portsmouth, told WIRED about the process of transforming a conductor's music into visual images: "The art of the conductor is to convey the score, making every gesture as natural as possible. Using motion capture data we're able to visualise Sir Simon Rattle's conducting movements in great detail and learn more about his interpretation of the piece. Additionally, it provides new information on the relationship between the conductor and the instrumentalists. Learning more about how instructions relating to tempo, volume and much more are communicated to the musicians."

The transformation of Sir Rattle's performance is an example of how data visualisation can be used to project visual emotion onto a piece of music. As Rattle's arms sway to the lull of string instruments, so the spirals he creates grow looser, calmer in colour, almost relaxing upon the air. As the tone reaches a frantic climax, we move to minutia, a pulsing set of squares, like dominos collapsing without end.

The art produced by LSO's motion capture project, and using Vicon's technology, is largely impressionistic in nature, with a digital twist. However, Rattle's performance has also been used to develop two custom typefaces - with the conductor himself having cut them into being. Each letter has a particular swipe through one point, where Rattle's baton cut through air.

LSO's experiments with motion capture technology can be used to reflect a performer's physicality during extremes of musical expression - how a body changes to reflect certain pitches in sound, or scores that necessitate rapid motion.

The animated films derived from this project will form a "visual language" that will introduce the LSO’s 2017/18 season, demonstrating the colourful, passionate motion of the orchestra's performers.

source: www.wired.co.uk