Beyond Sonification - making audible sense to data
The project examines the interaction between movement, sound and colour in real time in the production of new media and performances. A web app (Synestizer) is being used for this which converts colours into sounds as well as other software which supports performative practices. Three modules are planned: (1) technical development, (2) development of teaching units and (3) a festival with students.
With a web app which converts webcam data into sounds, you hear a moving image as sounds, what is called sonification. The sounds gain a deeper meaning if the technical sonification is applied to sound design. Because both a visual and acoustic level of composition is made possible by manipulating views for the camera, where the image can be explored as sound and vice versa. This research is being carried out with the help of open-source software which we have developed and which can be used and explored for different performance ideas in the real-time conversion of moving images. This also frequently involves integrating the instrumentalist's local environment into the music.