The integration of 3D-Audio in all central activities is an essential feature of the Immersive Arts Space. Spatial audio concepts are currently applied in a highly differentiated way through artistic research components of The Umbrella Project.
This ongoing research project explores the use of 3D audio and projection mapping as a means to create immersive experiences without isolating participants from the real world, essentially enabling an imaginary fantasy world to come to life in our own. We employ multiple levels of 3D audio and projection mapping (both directly within and on the umbrella, as well as throughout the room itself) in order to transport the participant into this virtual world.
As such, we can consider the umbrellas, and the overall system of which they are a part, to be an instrument of sorts in that they literally allow us to compose and explore reactive sonic environments in 6 degrees of freedom (6 DoF). Additionally, the umbrellas can also function as a measurement instrument, much like a stethoscope is a sonic instrument for medical examination; however, in this case, through the immersive experiences they purvey, the umbrellas enable us to examine the nature of our own perception of reality.
Ultimately, this research seeks to answer the question:
In order to address this goal, we consider 3 related research sub-questions:
-
How can we use Augmented Objects to avoid isolating participants from the real world (due to obvious and/or intrusive technology such as VR head-mounted displays and headphones) by embedding a new (extended) reality into the real world?
-
On a more practical level, how can we combine multiple levels of spatial audio (near and far field) to create 6 DoF immersive experiences?
-
How can we use sensor data (and technology in general) to create interactive environments that support a sense of presence, and in turn, enable immersive experiences?