Mainnavigation

      • DE
      • EN
    • Watchlist
    • Menu Menu
    You are here:
    1. Research Projects
    More: Research Projects

    Audio-Haptic modalities in Musical Interfaces

    Institute for Computer Music and Sound Technology (ICST)

    The sense of touch is of great importance when playing traditional musical instruments. Rich tactile response is however lacking in the current generation of digital musical interfaces. The project defines and pursues a novel approach to the design and evaluation of future digital musical interfaces providing active tactile feedback.

    The sense of touch is of great importance when playing traditional musical instruments. Rich tactile response is however lacking in the current generation of digital musical interfaces. The project defines and pursues a novel approach to the design and evaluation of future digital musical interfaces providing active tactile feedback.

    Context and Motivation
    Recent studies support the idea that tactile interaction in instrumental practice is key to achieving top performance levels, and supports expressivity and self-monitoring. This is however a critical aspect of digital musical instruments, since they generally do not offer rich tactile feedback. In order to recover such lost physicality and provide enhanced user experience and performance, we hypothesize that future digital musical instruments will implement active tactile feedback. Currently however, research on actuated digital musical instruments is often grounded on practice and intuition only, leading to the production of one-of-a-kind devices, and usually leaving out proper evaluation methodologies, thus preventing the achievement of general results.

    Methodology and Objectives
    This interdisciplinary project – which deals with human-computer interaction, musical practice, engineering and applied psychology – aims at setting a novel systematic and scientifically founded methodology to design and evaluate digital musical interfaces that provide audio-haptic feedback. The project will proceed iteratively through 1) experiments for investigating the role of the auditory and haptic modalities in instrumental practice, 2) measurement of vibrations and forces in musical instruments, and 3) development of prototype interfaces.

    Details

    • Research Focus
      • FSP Technologie und Musikalische Praxis
    • Applicant
      • Stefano Papetti (ICST)
      • Gerhard Tröster (ETH, Wearable Computing Lab, IfE)
    • Team
      • Martin Fröhlich (ICST)
      • Hanna Järveläinen (ICST)
      • Thomas Peter (ICST)
      • Sébastien Schiesser (ICST)
      • Matthias Flückiger (ETH, Wearable Computing Lab, IfE)
      • Tobias Grosshauser (ETH, Wearable Computing Lab, IfE)
    • Cooperations
      • HCI research group, Department of Psychology, Universität Basel
      • Federico Avanzini, Sound and Music Computing group, Università degli Studi di Padova, Italy
      • Federico Fontana, Università degli Studi di Udine, Dipartimento di Matematica e Informatica
      • Bruno Giordano, University of Glasgow, Department of Psychology
    • Duration

      01.03.2014 – 31.03.2016

    • Financing
      • Schweizerischer Nationalfonds SNF (01.03.2014 – 31.03.2016)
    • Research Approaches
      • Basic research
      • Scientific research
    • Disciplines

      Design, Music, Transdisciplinary

    • Keywords

      Evaluation, expressivity, Measurement, Material Discrimination, tangible interaction, Human-Computer Interaction, Audio-Haptic Feedback, Musical Interfaces, Performance, Interaction

    • More Links
      • D. Rocchesso, S. Delle Monache, and S. Papetti. Multisensory Texture Exploration at the Tip of the Pen. Int. J. of Human-Computer Studies, vol. 85, January 2016.
      • S. Papetti, H. Järveläinen, B. L. Giordano, S. Schiesser, and M. Fröhlich. Vibrotactile sensitivity in active touch: effect of pressing force. IEEE Trans. Haptics. vol.PP, no.99, 2016.
      • S. Papetti, H. Järveläinen, S. Schiesser and T. Grosshauser. Effects of audio-tactile feedback on force accuracy in a finger pressing task. In proc. int. workshop on Haptic and Audio Interaction Design (HAID), Daejeong, Korea, 2013.
      • F. Fontana, F. Avanzini, H. Järveläinen, S. Papetti, F. Zanini, and V. Zanini. Perception of Interactive Vibrotactile Cues on the Acoustic Grand and Upright Piano. In proc. Joint ICMC and SMC conf., Athens, Greece, 2014.
      • S. Papetti, S. Schiesser and M. Fröhlich. Multi-point vibrotactile feedback for an expressive musical interface. In proc. conf. on New Interfaces for Musical Expression (NIME), Baton Rouge, USA, 2015.
      • S. Papetti, H. Järveläinen, G-M. Schmid, Vibrotactile sensitivity in active finger pressing, in IEEE World Haptics Conference, Chicago, IL, USA, 2015.
      • F. Fontana, F. Avanzini, H. Järveläinen, S. Papetti, G. Klauer, & L. Malavolta. Rendering & subjective evaluation of real vs. synthetic vibrotactile cues on a digital piano keyboard. In proc. Sound & Music Computing conf. (SMC), Maynooth, Ireland, 2015.
      • G-M. Schmid, A. N. Tuch, S. Papetti, and K. Opwis. Three Facets for the Evaluation of Musical Instruments from the Perspective of the Musician. In proc. Music and HCI workshop at ACM CHI, San Jose, USA, 2016.
    • Downloads
      • AHMI_poster_AHMI.pdf

    Output

    • Aufsätze in Sammelbänden, Ausstellungskatalogen, Buchbeiträge

      Papetti, Stefano & Saitis, Charalampos (2018): «Musical Haptics. Introduction». In: Papetti, Stefano & Saitis, Charalampos (Hg.): Musical Haptics. Springer Series on Touch and Haptic Systems. Cham: Springer Open, 1–7. Online unter: https://doi.org/10.5281/zenodo.3628490.

    • Aufsätze in Sammelbänden, Ausstellungskatalogen, Buchbeiträge

      Papetti, Stefano / Fröhlich, Martin / Fontana, Federico / Schiesser, Sébastien & Avanzini, Federico (2018): «Implementation and Characterization of Vibrotactile Interfaces». In: Papetti, Stefano & Saitis, Charalampos (Hg.): Musical Haptics. Springer Series on Touch and Haptic Systems. Cham: Springer Open, 257–282. Online unter: https://doi.org/10.5281/zenodo.3628523.

    • Aufsätze in Fachzeitschriften

      Papetti, Stefano / Järveläinen, Hanna & Fontana, Federico (2023): «Design and Assessment of Digital Musical Devices Yielding Vibrotactile Feedback». Arts. 12, 4. 1–25. Online unter: https://doi.org/10.3390/arts12040143.

    • Aufsätze in Fachzeitschriften

      Fontana, Federico / Papetti, Stefano / Järveläinen, Hanna & Avanzini, Federico (2017): «Detection of keyboard vibrations and effects on perceived piano quality». The Journal of the Acoustical Society of America. 142, 5. 2953–2967. Online unter: https://doi.org/10.5281/zenodo.3676535.

    • Aufsätze in Fachzeitschriften

      Papetti, Stefano / Järveläinen, Hanna / Giordano, Bruno / Schiesser, Sébastien & Fröhlich, Martin (2017): «Vibrotactile sensitivity in active touch. Effect of pressing force». IEEE Transactions on Haptics. 10, 1. 113–122. Online unter: https://doi.org/10.1109/TOH.2016.2582485.

    • Aufsätze in Fachzeitschriften

      Rocchesso, Davide / Delle Monache, Stefano & Papetti, Stefano (2016): «Multisensory texture exploration at the tip of the pen». International Journal of Human-Computer Studies. 85. 47–56. Online unter: https://doi.org/10.5281/zenodo.3648183.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Fontana, Federico / Avanzini, Federico / Järveläinen, Hanna / Papetti, Stefano / Klauer, Giorgio & Malavolta, Lorenzo (2016): «Un prototipo di pianoforte digitale con feedback vibrotattile». In: Terzaroli, Anna & Valle, Andrea (Hg.): Atti del XXI CIM - Colloquio di Informatica Musicale (CIM). Cagliari: 151–157. Online unter: https://doi.org/10.5281/zenodo.4071390.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Del Piccolo, Alan / Delle Monache, Stefano / Rocchesso, Davide / Papetti, Stefano & Mauro, Davide Andrea (2015): «To "Sketch-a-Scratch"». Proceedings of the Sound and Music Computing Conference. Maynooth, Ireland: 309–316. Online unter: https://doi.org/10.5281/zenodo.3694976.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Fontana, Federico / Avanzini, Federico / Järveläinen, Hanna / Papetti, Stefano / Klauer, Giorgio & Malavolta, Lorenzo (2015): «Rendering and Subjective Evaluation of Real vs. Synthetic Vibrotactile Cues on a Digital Piano Keyboard». Proceedings of the Sound and Music Computing Conference. Maynooth, Ireland: 161–167. Online unter: https://doi.org/10.5281/zenodo.3694947.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano / Järveläinen, Hanna & Schmid, Gian-Marco (2015): «Vibrotactile sensitivity in active finger pressing». IEEE World Haptics Conference (WHC). Evanston, IL: 457–462. Online unter: https://doi.org/10.5281/zenodo.4314903.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano / Schiesser, Sébastien & Fröhlich, Martin (2015): «Multi-point vibrotactile feedback for an expressive musical interface». Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Baton Rouge, LA: 235–240. Online unter: https://doi.org/10.5281/zenodo.1179151.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Fontana, Federico / Avanzini, Federico / Järveläinen, Hanna / Papetti, Stefano / Zanini, Francesco & Zanini, Valerio (2014): «Perception of Interactive Vibrotactile Cues on the Acoustic Grand and Upright Piano». Proceedings of the Joint International Computer Music Conference and Sound and Music Computing Conference. Athens, Greece: 948–953. Online unter: https://doi.org/10.5281/zenodo.3694956.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Fontana, Federico / Järveläinen, Hanna / Papetti, Stefano / Avanzini, Federico / Zanini, Francesco & Zanini, Valerio (2014): «Un esperimento sulla percezione di vibrazioni sulla tastiera del pianoforte nella performance musicale». In: Licitra, Gaetano & Fidecaro, Francesco (Hg.): Atti del 41° Convegno Nazionale Associazione Italiana di Acustica, Pisa, 17-19 giugno 2014. Roma: Associazione Italiana di Acustica, 1–8. Online unter: https://doi.org/10.5281/zenodo.4071392.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano / Järveläinen, Hanna / Schiesser, Sébastien & Grosshauser, Tobias (2013): «Effects of audio-tactile feedback on force accuracy in a finger pressing task: a pilot study». International workshop on Haptic and Audio Interaction Design (HAID). 1–3.

    • Herausgeberschaft

      Papetti, Stefano & Saitis, Charalampos (Hg.) (2018): Musical Haptics. Springer Series on Touch and Haptic Systems. Cham: Springer Open. Online unter: https://doi.org/10.5281/zenodo.3628467.

    • Audio-Datei

      Papetti, Stefano / Avanzini, Federico & Fontana, Federico (2018): «BiVib. Audio-Tactile Piano Sample Library». Online unter: https://doi.org/10.5281/zenodo.1213210.

    • Öffentliche Präsentationen

      Papetti, Stefano (2018): «Auditory-tactile modalities in everyday and musical interaction». Genova, Italy.

    • Medienresonanz

      Fischer, Roland (2017): «Vibrierendes Feedback». Horizonte – Das Schweizer Forschungsmagazin, 113.32–33. Besprechung von: Audio-Haptic modalities in Musical Interfaces. Papetti, Stefano / Fröhlich, Martin / Järveläinen, Hanna & Schiesser, Sébastien. Online unter: http://www.snf.ch/SiteCollectionDocuments/horizonte/Horizonte_gesamt/SNF_horizonte_113_de.pdf.