pub.mdw | CLOUD HANDS: THREE EMANATIONS OF A TOUCHLESS MUSICAL ASSEMBLAGE

CLOUD HANDS: THREE EMANATIONS OF A TOUCHLESS MUSICAL ASSEMBLAGE

Full item page

Download

Abstract
- Touchless control of human-computer interfaces in science-fiction has inspired several musical applications. This paper presents Cloud Hands, a touchless musical assemblage that controls sample-based granular synthesis via hand motion. The paper discusses how the assemblage manifests in three emanations: as a sound interface, a digital music instrument and an interactive multimedia composition. A dedicated Python script extracts the hand landmark positions and distances from a camera frame, which are mapped to control parameters of granular synthesis of a sample in Pure Data. A Wekinator patch may extract symbolic hand poses to change samples and playback speed. The emanations use varying modes of visual feedback, from no feedback at all to wire-frame models of the hand to sound material-related segmentation of video material as a reference map. Performance recollections from the first author and other users of Cloud Hands are analyzed in order to identify conveniences and constraints of the sensing technology, mapping, and sound production. Finally, the paper discusses the influence of conveniences and constraints on aesthetics, form and performance behavior.
Date Issued 2025-07
Authors
Grund, Tim-Tarek | University of Music and Performing Arts Vienna

Hofmann, Alex | University of Music and Performing Arts Vienna
DOI 10.5281/zenodo.15840925
Language en
License CC-BY-4.0
ÖFOS mit Kunstzweigen
30% MUSIK - Computermusik

15% NATURWISSENSCHAFTEN - Informatik

10% MUSIK - Interpretation – instrumental

10% GEISTESWISSENSCHAFTEN - Kunstwissenschaften

15% MUSIK - Pädagogik / Vermittlung

10% NATURWISSENSCHAFTEN - Physik, Astronomie