In TRANSLATION OF THREE IMPROVISERS (2016), I was given data recordings (from accelerometer, gyroscope, EMG sensors) of three dancers collective improvisation. All I knew about the improvisation was that it was performed by three separate dancers, done in one recording. I was then asked to make a musical composition derived from this data.
In my work, I am interested in the dynamics of machine/human collaboration, considering all agents involved as modular contributors to a comprehensive system.From this perspective, I created a generative synthesizer that would model the three performers’ improvisations, allowing me to create my own musical forms based on metaphoric choreography. To build this synth, I used the factored restricted conditional Boltzmann machine (FCRBM) algorithm, which allowed me to map and generate material across modes, in this case, from movement data to sonic consequence.
For the initial model, I derived everything from the dancers’ movement data. Source waveforms were built from aggregate statistics (i.e. mean and variance) of the different movement features. I used frequency modulation to generate higher order representations of the dancers from a generalized statistic perspective (i.e.modulated a dancer’s mean waveform with the same dancer’s variance waveform). Once these higher-order representations were created, I modeled the raw movement data using the FCRBM, creating three distinct classes, one for each dancer. After training, this allowed for me to “play” each performer by turning on the respective label for each dancer. Furthermore, I mapped each dancer to a granular synthesizer, using the FCRBM to transition from movement features to synth parameters.
The resulting compositional tool allows the composer to write a sonic choreography of improvisations drawn directly from the initial dancers’ expressions. While any choreography can be designed, I wrote three etudes highlighting the dynamic capabilities of the system.