Ventriloquist Ontology
The continuous implementation of AI and ML systems in all areas of technological artifacts, including art, is challenging the ways in which we understand the world around us and urge us to consider other-than-human entities and ‘objects’ as equally important as human beings. In an exploration of such philosophical ideas that stem from the realms of Posthumanism, Actor Network Theory, and Object-Oriented Ontology, I propose the creation of a modular wearable, trained using Natural Language Processing to create its own personality that manifests in the form of speech and movement actuation.
Performer: Tingyi Jiang
Videographers: Tanja Busking and Francesco Enriquez
The project ‘Ventriloquist Ontology’ explores the limits of control and points of hybridization between the human and the machine through the relationship of a performer and a wearable entity. This ventriloquist modular soft entity speaks through text generated using a GPT-2 language model, trained on a dataset of texts around biopolitics, algo-governance, the surveillanced body, and queer theory.
Performer Tingyi Jiang at the Instrument Inventors Initiative (iii) in The Hague, September 2021.
Inspired by Alejandro Jodorowsky's dystopian theatrical play "The School of Ventriloquists" brought to life by Belgian theater company Point Zéro in 2008, in this project I propose the creation of a wearable that takes over the wearer's body and converts them into a puppet who' s movement is dictated by what they wear.
The conceptualization of the work began during the Algocracy panel at the 4S conference in August 2020 where I discussed with other artists and STS scholars what type of data would be used to train the language the model and speculate about the “conscious” aspect that could be produced. Some of the initial ideas included a dataset of quantified self data and their value, gender and body politics and bioethics (p.e. genetic modification laws etc). The speculative prototype that I presented at 4S pertains to the approximation of the concept of algorithmic governance of the body, but also to experiment with policies around “bodies” as a means of creating sentience.
During Summer 2021 I worked towards creating the final dataset for training the language model and ended up including: excerpts of the US legal code that discusses body politics, factory manuals about body movement and optimization of production, academic articles around surveillance using wearable devices, queer theory texts, posthuman theory texts about cybergization of the body, and transhumanist texts about the technological extension of the body. After compiliing the data I trained a GPT-2 model using Google Collabs, and asked it to generate text using the follwoing prompts: “body”, “bodies”, “gender”, “cyborg", "politics", "queer", "binary", "movement”. I want to acknowledge here that a lot of this work was done with the invaluable help of my research assistant Sadaf Sadri, and consultation from HCDE PhD student Gabrielle Benabdallah. Additional custom Python scripts were developed by James Wenlock to help us scrape through PDFs and web sources for content for the dataset.
On September 2021 I had the opportunity to spend a full month at the Instrument Inventors Initiative in The Hague, Netherlands where I developed the wearable prototype for this system and collaborated with media performer Tingyi Jiang to present the work publically.
To construct the wearable interface I used white neoprene fabric, an Arduino BLE Sense microcontroller, and five Actuonix linear actuators. The wearable consists of five modular pieces with an actuator attached on each one and they are connected together using black fasteners and straps.
For the electrical connections I used soft silicone wires and metallic snaps in order to be able to connect and disconnect the modules separately, allowing for the modules to be adjustable to different body types. Due to the current draw of the linear actuators I had to use three 2500mAh lipo batteries that would allow me flexibility in terms of being able to do long rehearsals and not suffer power cuts.
On the software side of things, I collaborated with James Wenlock in the creation of a custom Python bridge script to communicate over OSC with SuperCollider, and a SuperCollider GUI that allows me to play back the generated GPT-2 text using a computerized voice, as well as to control manually the actuators on the performer’s body.
For a complete technical documentation of the project go to:
The performance was also presented on Wednesday, February 23, 2022 at 7:30pm PDT at Meany Hall—Katharyn Alvord Gerlich Theater in Seattle as part of the DXARTS Winter Concert 'Movement Actuation' with performer Carolina Marín.
In September 2022 a video version of the piece was developed which formed part of PCAI's (Polygreen Contemporary Art Initiative) "Sheltered Gardens" exhibition in Athens, Greece and virtually, curated by Kika Kyriakakou. The video version is a limited edition of 5 copies, and now forms part of PCAI’s video art collection. You can see the virtual exhibition at:
https://360orasis.gr/tour/Sheltered%20Gardens/
Algo-bodies: Identity, language and body actuation
In the context of the iii residency in The Hague, I instructed a public-facing workshop where participants could experiment hands-on with the idea of body control. In this 12-hour long workshop participants explored the use of wearable technology and speech-to-text application programming interfaces such as the Google AIY kit, while discussing ideas around the construction of identity around our bodies, and how algorithms dictate our movement through space. The participants worked together in groups and created performative experiments by actuating each other’s bodies. The last day of the workshop culminated in an experimental collective performance which formed part of the ‘Soft Control’ iii showcase.
Support
This project has been developed with support from DXARTS, University of Washington.
iii’s residency program is made possible through the support of Creative Industries Fund NL and the Creative Europe program of the European Union.
Credits
Research assistants: Sadaf Sadri (GPT-2), James Wenlock (Python, BLE, SuperCollider)
Performer: Tingyi Jiang
Videographers: Tanja Busking and Francesco Enriquez
Exhibitions and Performances
2023
Intelligence Unbound by Foreign Objekt, The Space gallery, San Francisco, CA. Curated by Sepideh Majidi and Maure Coise. (upcoming)
2022
Sheltered Gardens, PCAI (Polygreen Culture and Art Initiative), Athens, Greece. Curated by Kika Kyriakakou.
Possibles, ISEA 2022 (International Symposium of Electronic Art), Recinte Modernista Hospital de Sant Pau, Barcelona, Spain.
CTM Festival, Athens Epidavrous Festival, with Tingyi Jiang, Athens, Greece. Curated by Akis Sinos.
DXARTS Winter Concert: Movement Actuation, Meany Hall—Katharyn Alvord Gerlich Theater with Carolina Marin, UW, Seattle, WA, USA.
2021
Soft Control, iii Workspace, The Hague, Netherlands.
Conferences
2020
4S/EASST Prague 2020 Locating and Timing Matters: Significance and Agency of STS in Emerging Worlds, Ventroloquist Ontology artist talk as part of the "Workshop on Experiments with Algo-governance and Future-Making: STS Scholars as Designers".
Publications
2022
Psarra, Afroditi, and Jiang, Tingyi. 2022. Ventriloquist Ontology. In Artworks Catalogue of the 27th International Symposium of Electronic Art (ISEA2022 Possibles). Barcelona, Spain, pp. 216–217. https://doi.org/10.7238/ISEA2022.Catalog
Psarra, Afroditi. 2022. Ventriloquist Ontology. Published in “Prologoi”, edited by Miss Dialectic (Klea Charitou, Eleanna Papathanasiadi, and Kelly Tsipni-Kolaza), Athens, Greece, 126-131. https://drive.google.com/drive/folders/1FwuOZ8UuGSRpOOfRh1ENGSGOyuv66J0V
Residencies
2021
iii research residency, iii Workspace, The Hague, Netherlands.
Workshops
2021
Algo-bodies: Identity, language and body actuation, iii Workspace, The Hague, Netherlands.
Press
Sabatini, Benedetta. “Ventriloquist Ontology, the Fabric Is More Alive than the Skin.” Neural Magazine, 13 Jan. 2023, https://neural.it/2023/01/ventriloquist-ontology-the-fabric-is-more-alive-than-the-skin/