Kinect V2 - Depth Tracking Piano Installation

The Kinect experiment was undertaken to explore a more advanced method of input interaction through the use of depth and motion tracking. The coding sketchbook software known as Processing is used, and the library used to run the Kinect is written by Daniel Shiffman and can be found at this URL:

https://github.com/shiffman/OpenKinect-for-Processing

The inclusion of depth offers a more immersive experience for the user within sound art installations. A participant within this installation wants to know how they can take part in the music, therefore, they can walk into the zone of play and easily realise their input by moving their arms around. Slowly learning to play this instrument by correlating specific movements with desired sound with no previous knowledge of music.

As this experiment is being used alongside the facial tracker, both experiments work to coincide with each other. With one user being able to play a melodically structure and the other being able to manipulate the ambient background has allowed further transformation of the composition as a whole. This allows for extra-musical variation within the piece and both players have more of a realisation of the part they are playing, which is an important aspect of a collaborative installation.

The inclusion of this experiment within sound art installations presents a more flexible method of interaction. The Kinect is a device that can work with many other variables rather than depth, therefore there are many possibilities for it within sound art installations. Furthermore being able to use the Kinect alongside Wekinator and Reaktor has allowed a second person to collaborate in the composition within these experiments. Therefore, this set up, if perfected could work as a small travelling sound art installation, contacting businesses for the approval to situate within public places such as galleries, museums on public streets. The project’s sound can be manipulated to reflect the space it is within, for example, within a museum the sounds can be made to fit into a theme such as dinosaurs or the industrial revolution.

This collaborative project between the facial tracker and using the Kinect has explored how more than one person can participate within the composition. Demonstrating the importance for participants to realise their role within installations and feeling that they have added a meaningful contribution to the sound through the means of manipulating, transforming and feeling the illusion of space through reverb.