Conversations with Puppets

Description

Conversations with Puppets offers an innovative and interactive experience in which the visitors become an integral part of Fred Schneckenburger’s puppet cabaret. After World War II, the versatile artist Fred Schneckenburger founded the “Puppencabaret Fred Schneckenburger” in Zurich and enjoyed success throughout Europe with his sketches. His expressionistic, surreal stick puppets were characterized by their idiosyncratic and unique design. In terms of content, his pieces criticized the hypocrisy and contradictions of the conservative bourgeoisie, with stereotypical Swiss characters in the form of puppets. 

Three of the original stick puppets – Kaspar, der Polizist and die Moral/das Fräulein – from the collection of the Museum für Gestaltung Zürich have been photogrammetrically captured and digitized. 3D artists worked out the details of the textures and cloths to recreate and animate the physical appearance of the puppets surfaces. In addition, Motion Capture recordings were used to add more puppet-like movement to the avatars. 

Large Language Models (LLMs) and generative AI form the basis of the interaction and conversations. In order to create a character prompt for each puppet, the original audio recordings of the pieces in which the specific puppets appeared, were transcribed and analyzed. The result of the analysis were distillations of the most characteristic words and/or expressions of the puppets, which were then inserted in the prompt. In addition, to the character references the prompts were further developed to fit the role of each puppets within the exhibition and its dramaturgy.

Further, the interaction is supported by different systems, aiming at a flawless interaction and conversation. SAR motion, a system internally developed by Florian Bruggisser, detects via a camera the movement and body of a visitor. This triggers the interaction and the puppet starts to talk.

Exhibitions

The Museum of the Future – 17 Digital Experiments
29th August 2025 untill 1st February 2026, Museum für Gestaltung Zürich
The exhibition ‘The Museum of the Future – 17 Digital Experiments’ examined the potential, areas of application, forms of content expansion and implementation options relating to digitality in forward-looking museum and educational work.

Team

  • Florian Bruggisser: Technical Lead and Software Development 
  • Oliver Sahli: Idea, Interaction and System Integration
  • Chris Elvis Leisi: Interaction and Experience Design  Stella Speziali: 3D Artist, Animation
  • Johannes Köberle: 3D Artist
  • Corinne Soland: Motion Capture 
  • Pascal Lund-Jensen: Audio Engineering     
  • Sébastien Schiesser: Technical support
  • Kristina Jungić: Project coordination
  • Prof. Christopher Salter: Initiative / Director IAS