research project

Mit geschärfter Klinge durch das digitale Auge: Machine Learning und Pose-Estimation bewegungsanalytisch durchleuchtet

Bildtafel XLIII. In: Sechste Mappe. Courses de Testes et de Baque faites par le Roy – Denkmäler des Theaters. Inszenierung/Dekoration Kostüm des Theaters und der grossen Feste aller Zeiten. Wien/München: R. Piper & Co. Verlag 1925.

info
research project combining movement analysis and machine learning algorithms

in collaboration with Nicole Haitzinger and the Derra de Moroda Dance Archives

supported by

LIMBO Arts Studios, Förderverein der Universität Salzburg

about

In the research project “Mit geschärfter Klinge durch das digitale Auge: Machine Learning und Pose-Estimation bewegungsanalytisch durchleuchtet,” Nicole Haitzinger and I attempted to implement dance-specific body knowledge into machine learning systems. By combining movement analysis and AI methods, we developed an alternative labeling model based on IVB (Inventarisierung von Bewegung) to train a pose estimation system (automated estimation of body posture) for violence detection.
The applications of machine learning systems for pose estimation range from seemingly harmless fitness apps to violence detection in public spaces and automated warfare. The datasets used for this purpose, consisting of photos and video recordings, must undergo a classification process—known as labeling—in order to be usable. 

„The circuit between image, label, and referent is flexible and can be reconstructed in any number of ways to do different kinds of work. What’s more, those circuits can change over time as the cultural context of an image shifts, and can mean different things depending on who looks, and where they are located. Images are open to interpretation and reinterpretation.“[1]

Following the approach of Crawford and Paglen, we developed a labeling method that assigns classifications based on parameters informed by dance studies and media criticism. The aim was to challenge current classification standards, in which the categorization of visual data is often based on the subjective judgments, clichés, and prejudices of the individuals doing the labeling. Our labeling model seeks to counteract these increasingly subjective and stereotypical assessments and instead establish bodily movements and positions as the basis for classification.
The development of the new model took place within a defined historical framework. The visual material under consideration consisted of war-related and violent imagery from the Baroque period drawn from the Derra de Moroda Dance Archives. The choice of this era is rooted in the political dimension of Baroque performance practice.

„Die Reiterfeste [sowie andere theatrale Ausdrucksformen des Barock] sind an einer Schnittstelle zwischen theatraler Kunstform, Veranschaulichung höfischer Hierarchien und paramilitärischem Training anzusiedeln.“[2]

For more information visit the Derra de Moroda Dance Archives Website

 [1] Crawford, Kate; Paglen, Trevor: Excavating AI: the politics of images in machine learning training sets. In: AI & Society, 36 (2021), S. 1107. https://doi.org/10.1007/s00146-021-01162-8, zuletzt aufgerufen am 01.04.2026

[2] Papiro, Martina: Choreographie der Herrschaft. Stefano della Bellas Radierungen zu den Reiterfesten am Florentiner Hof 1637-1661. Paderborn: Wilhelm Fink 2016, S. 28.

Bildtafel XV. In: Sechste Mappe. Courses de Testes et de Baque faites par le Roy – Denkmäler des Theaters. Inszenierung/Dekoration Kostüm des Theaters und der grossen Feste aller Zeiten. Wien/München: R. Piper & Co. Verlag 1925. 

gallery