Mainnavigation

      • DE
      • EN
    • Watchlist
    • Menu Menu
    You are here:
    1. Research Projects
    More: Research Projects

    Haptic Technology and Evaluation for Digital Musical Interfaces

    Research on hardware technology, haptic perception and human-computer interaction for the advancement of haptic interaction with digital musical devices

    Institute for Computer Music and Sound Technology (ICST)

    Music performance with traditional instruments usually involves the establishment of a complex haptic exchange between players and instruments. However digital musical devices currently do not offer such physical richness.

    Advancing haptic interaction with such devices will require input from research on hardware technology and on haptic perception, as well as of various aspects of human-computer interaction (user experience, performance assessment, etc.).

    The recent interest in haptics - especially by the HCI and engineering communities and the consumer electronics industry - relies on the assumption that properly designed haptic interfaces result in better user experience, improved performance and usability. Indeed, while performing on traditional instruments, the players' haptic channel is involved in a complex action-perception loop. Accordingly, we expect that digital musical instruments (DMIs) will especially gain from haptic technology.The former interdisciplinary project AHMI (SNSF grant 150107) initiated an original, exploratory investigation on the role of auditory and haptic modalities in musical practice. The HAPTEEV project will continue that research focusing on the development of advanced haptic technology suitable for musical interfaces, while parallel subjective evaluation and performance assessment will provide scientific foundation to technological and design solutions.Motivation and Goals: Human-computer interfaces are usually operated by touch, however information feedback to the user mostly relies on the visual and, to a lesser extent, auditory modalities. Similarly, current DMIs generally lack haptic feedback, and we argue that re-establishing a rich haptic exchange between musicians and digital interfaces would improve user experience, and support performance control and expressivity. In this framework, project HAPTEEV will pursue the design and implementation of novel technology for rendering rich vibrotactile feedback.Evaluation experiments, addressing both subjective and objective interaction characteristics, will provide scientific foundations to the design of haptic DMIs.Broader impact: In addition to offering improved user experience, performance and expressivity, haptic musical interfaces have a high potential for providing guidance in musical tuition, large ensembles and remote performance, and for granting access to musical practice for persons with somatosensory loss, the visually and even the hearing impaired. Technological and experimental results of the project are expected to set a reference for research in HCI and haptic engineering at large, fostering the development of next generation haptic interfaces.

    Details

    • Research Focus
      • FSP Technologie und Musikalische Praxis
    • Applicant
      • Stefano Papetti (ICST)
      • Yves Perriard (Ecole polytechnique fédérale de Lausanne, EPFL)
    • Team
      • Martin Fröhlich (ICST)
      • Hanna Järveläinen (ICST)
      • Thomas Peter (ICST)
      • Sébastien Schiesser (ICST)
      • Jonathan Cavanne (Ecole polytechnique fédérale de Lausanne, EPFL)
      • Florian Copt (Ecole polytechnique fédérale de Lausanne, EPFL)
      • Camilo Hernandez Mejia (Ecole polytechnique fédérale de Lausanne, EPFL)
    • Cooperations
      • Federico Avanzini, Università degli Studi di Milano
      • Federico Fontana, Università degli Studi di Udine, Dipartimento di Matematica e Informatica
    • Duration

      01.08.2018 – 31.07.2022

    • Financing
      • Schweizerischer Nationalfonds SNF (01.08.2018 – 31.07.2022)
    • Research Approaches
      • Applied research
      • Scientific research
    • Disciplines

      Design, Music, Transdisciplinary, Other

    • Keywords

      Art & Technology, Evaluation, Human-Computer Interaction, Audio-Haptic Feedback, Musical Interfaces, Touch-based Interaction, Haptics, Vibration

    • More Links
      • Projektseite des SNF

    Output

    • Aufsätze in Sammelbänden, Ausstellungskatalogen, Buchbeiträge

      Järveläinen, Hanna / Papetti, Stefano & Larrieux, Eric (2024): «Exploring the Effects of Additional Vibration on the Perceived Quality of an Electric Cello». Vibration. 7, 2. 407–418. Online unter: https://doi.org/10.3390/vibration7020021.

    • Aufsätze in Sammelbänden, Ausstellungskatalogen, Buchbeiträge

      Papetti, Stefano & Saitis, Charalampos (2018): «Musical Haptics. Introduction». In: Papetti, Stefano & Saitis, Charalampos (Hg.): Musical Haptics. Springer Series on Touch and Haptic Systems. Cham: Springer Open, 1–7. Online unter: https://doi.org/10.5281/zenodo.3628490.

    • Aufsätze in Sammelbänden, Ausstellungskatalogen, Buchbeiträge

      Papetti, Stefano / Fröhlich, Martin / Fontana, Federico / Schiesser, Sébastien & Avanzini, Federico (2018): «Implementation and Characterization of Vibrotactile Interfaces». In: Papetti, Stefano & Saitis, Charalampos (Hg.): Musical Haptics. Springer Series on Touch and Haptic Systems. Cham: Springer Open, 257–282. Online unter: https://doi.org/10.5281/zenodo.3628523.

    • Aufsätze in Fachzeitschriften

      De Pra, Yuri / Papetti, Stefano / Järveläinen, Hanna / Bianchi, Matteo & Fontana, Federico (2023): «Effects of vibration direction and pressing force on finger vibrotactile perception and force control». IEEE Transactions on Haptics. 1–11. Online unter: https://doi.org/10.5281/zenodo.7794732.

    • Aufsätze in Fachzeitschriften

      Papetti, Stefano / Järveläinen, Hanna & Fontana, Federico (2023): «Design and Assessment of Digital Musical Devices Yielding Vibrotactile Feedback». Arts. 12, 4. 1–25. Online unter: https://doi.org/10.3390/arts12040143.

    • Aufsätze in Fachzeitschriften

      De Pra, Yuri / Papetti, Stefano / Fontana, Federico & Tiberi, Emidio (2021): «An Open-Source Robotic Tool for the Simulation of Quasi-Static Finger Pressing on Stationary and Vibrating Surfaces». IEEE Transactions on Haptics. 14, 2. 273–278. Online unter: https://doi.org/10.5281/zenodo.6606251.

    • Aufsätze in Fachzeitschriften

      Papetti, Stefano / Järveläinen, Hanna & Schiesser, Sébastien (2021): «Interactive Vibrotactile Feedback Enhances the Perceived Quality of a Surface for Musical Expression and the Playing Experience». IEEE Transactions on Haptics. 14, 3. 635–645. Online unter: https://doi.org/10.5281/zenodo.6619050.

    • Aufsätze in Fachzeitschriften

      De Pra, Yuri / Fontana, Federico / Järveläinen, Hanna / Papetti, Stefano & Simonato, Michele (2020): «Does It Ping or Pong? Auditory and Tactile Classification of Materials by Bouncing Events». ACM Transactions on Applied Perception. 17, 2. 1–17. Online unter: https://doi.org/10.5281/zenodo.4555721.

    • Aufsätze in Fachzeitschriften

      De Pra, Yuri / Papetti, Stefano / Fontana, Federico / Järveläinen, Hanna & Simonato, Michele (2020): «Tactile discrimination of material properties: application to virtual buttons for professional appliances». Journal on Multimodal User Interfaces. 14, 3. 255–269. Online unter: https://doi.org/10.5281/zenodo.4012301.

    • Aufsätze in Fachzeitschriften

      Papetti, Stefano / Avanzini, Federico & Fontana, Federico (2019): «Design and Application of the BiVib Audio-Tactile Piano Sample Library». Applied Sciences. 9, 914. 1–15. Online unter: https://doi.org/10.5281/zenodo.3564615.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano / Larrieux, Eric & Fröhlich, Martin (2023): «A Versatile Finger-Interaction Device with Audio-Tactile Feedback». ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal Interaction. 107–108. Online unter: https://doi.org/10.1145/3610661.3620661.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano / Larrieux, Eric & Fröhlich, Martin (2023): «The TouchBox MK3: An Open-Source Device for Finger-Based Interaction with Advanced Auditory and Vibrotactile Feedback». ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal Interaction. 302–305. Online unter: https://doi.org/10.1145/3610661.3616181.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      De Pra, Yuri / Fontana, Federico / Järveläinen, Hanna / Papetti, Stefano / Bianchi, Matteo & Sonego, Mauro (2022): «Evaluation of rotation gestures in rotary vs. motionless knobs». 2022 IEEE Haptics Symposium (HAPTICS). Santa Barbara, CA, USA: 1–6. Online unter: https://doi.org/10.1109/HAPTICS52432.2022.9765608.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Järveläinen, Hanna / Papetti, Stefano & Larrieux, Eric (2022): «Accuracy of Musical Pitch Control Through Finger Pushing and Pulling». In: Saitis, Charalampos / Farkhatdinov, Ildar & Papetti, Stefano (Hg.): Haptic and Audio Interaction Design. HAID 2022. Lecture Notes in Computer Science, 13417. Cham: Springer, 125–134. Online unter: https://doi.org/10.5281/zenodo.7798516.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano / Järveläinen, Hanna & Larrieux, Eric (2022): «Uno studio sul controllo di intonazione di un sintetizzatore tramite gesti di spinta e trazione». Proceedings of the XXIII Colloquio di Musica Informatica / Colloquium of Musical Informatics. Ancona: CIM (Colloquio di Informatica Musicale / Colloquium of Musical Informatics), 1–6. Online unter: https://doi.org/10.5281/zenodo.7373268.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      De Pra, Yuri / Fontana, Federico & Papetti, Stefano (2021): «Endless Knob with Programmable Resistive Force Feedback». In: Ardito, Carmelo / Lanzilotti, Rosa / Malizia, Alessio / Petrie, Helen / Piccinno, Antonio / Desolda, Giuseppe & Inkpen, Kori (Hg.): Human-Computer Interaction – INTERACT 2021. Lecture Notes in Computer Science, 12935. Cham: Springer, 580–589. Online unter: https://doi.org/10.5281/zenodo.6627968.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      De Pra, Yuri / Fontana, Federico & Papetti, Stefano (2021): «Interacting with digital audio effects through a haptic knob with programmable resistance». Proceedings of the 24th International Conference on Digital Audio Effects (DAFx20in21). DAFx20's Vienna Series. 113–120. Online unter: https://doi.org/10.5281/zenodo.6619151.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      De Pra, Yuri / Fontana, Federico / Papetti, Stefano & Simonato, Michele (2020): «A low-cost endless knob controller with programmable resistive force feedback for multimedia production». In: Spagnol, Simone & Valle, Andrea (Hg.): Proceedings of the 17th Sound and Music Computing Conference. Torino: 242–246. Online unter: https://doi.org/10.5281/zenodo.4012221.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      De Pra, Yuri / Fontana, Federico / Järveläinen, Hanna / Papetti, Stefano / Simonato, Michele & Furlanetto, Riccardo (2019): «Auditory and tactile recognition of resonant material vibrations in a passive task of bouncing perception». International Workshop of Haptic and Audio Interaction Design - HAID 2019. Lille: 1–5. Online unter: https://doi.org/10.5281/zenodo.4604253.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano / Fröhlich, Martin & Schiesser, Sébastien (2019): «The TouchBox: an open-source audio-haptic device for finger-based interaction». IEEE World Haptics Conference. Tokyo, Japan: 491–496. Online unter: https://doi.org/10.5281/zenodo.3929455.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano (2018): «Aptica e interazione musicale». XXII Colloquio di Informatica Musicale / 22nd Colloquium on Music Informatics - Atti della Conferenza / Conference Proceedings. 27–34. Online unter: https://doi.org/10.5281/zenodo.3925264.

    • Aufsätze in Tagungsakten (Vortrag, Keynote)

      Papetti, Stefano / Avanzini, Federico & Fontana, Federico (2018): «BIVIB: A Multimodal Piano Sample Library of Binaural Sounds and Keyboard Vibrations». In: Davies, Matthew / Ferreira, Aníbal / Campos, Guilherme & Fonseca, Nuno (Hg.): Proceedings of the 21st International Conference on Digital Audio Effects (DAFx-18). Aveiro: 237–243. Online unter: https://doi.org/10.5281/zenodo.3982954.

    • Herausgeberschaft

      Papetti, Stefano & Saitis, Charalampos (Hg.) (2018): Musical Haptics. Springer Series on Touch and Haptic Systems. Cham: Springer Open. Online unter: https://doi.org/10.5281/zenodo.3628467.

    • Audio-Datei

      Papetti, Stefano / Avanzini, Federico & Fontana, Federico (2018): «BiVib. Audio-Tactile Piano Sample Library». Online unter: https://doi.org/10.5281/zenodo.1213210.

    • Öffentliche Präsentationen

      Papetti, Stefano (2018): «Auditory-tactile modalities in everyday and musical interaction». Genova, Italy.

    • Andere aktive Teilnahmen (Podiumsteilnahme, Posterpräsentation etc.)

      Papetti, Stefano / Fröhlich, Martin (2018): «The TouchBox: A DIY haptic interface for finger-based interaction». Eurohaptics. 13.06.2018–16.06.2018. Pisa, Italy.

    • Teilnahmen auf Einladung

      Papetti, Stefano / Linn, Roger / Tolan, Claire / Kirn, Peter (2018): «The Future of Listening by Amplifon @ Sonar+D». 14.06.2018–16.06.2018. Barcelona. Online unter: https://sonarplusd.com/en/programs/barcelona-2018/areas/talks/the-future-of-listening-by-amplifon.

    • Teilnahmen auf Einladung

      Papetti, Stefano / Saitis, Charalampos (2018): «Touch the Music @ Most Wanted Music Berlin 2018». 07.11.2018–08.11.2018. Berlin. Online unter: https://soundcloud.com/berlinmusiccommission/touch-the-music?in=berlinmusiccommission/sets/most-wanted-music-2018.

    • Medienresonanz

      Schnydrig, Stephanie (2019): «Digital touch». horizons The Swiss Research Magazine, 06.06.2019. Zeitungsbericht. Besprechung von: Hapteev. Forschung. De Pra, Yuri / Fontana, Federico / Järveläinen, Hanna & Papetti, Stefano. Online unter: https://www.horizons-mag.ch/2019/06/06/digital-touch/.

    • Medienresonanz

      Möller, Christoph (2018): «Elektrische Impulse im Konzertsaal. Musik mit allen Sinnen erleben». Deutschlandfunk Kultur, 13.11.2018. Tonart.Radiobeitrag, Interview anlässlich Most Wanted Music 2018. Besprechung von: Musical Haptics. Buch. Papetti, Stefano & Charalampos, Saitis. Online unter: https://www.deutschlandfunkkultur.de/elektrische-impulse-im-konzertsaal-musik-mit-allen-sinnen.2177.de.html?dram:article_id=433099.