Time of the Flies

22.07.2022

Installation at Haus der Kunst München
One of two video screens showing group behavior of fruit flies

Time of the flies is a further development of Drosophila Karaoke Bar.

It was first shown at the Science Gallery Melbourne as a part of the SWARM exhibition.

During the time of the pandemic I was experiencing our biological condition in a new way: Pandemics are teaching us about spheres, proximities, our distance behavior. We are asking ourselves, if we have been paying enough attention to the safety of our sphere of life so far and who we let come close. I called this period „time of flies“, mainly as I was living in my one room apartment in Berlin alone with fruitflies – cultures of fruit flies in my home. 

As soon as you work with a biological organism (animals or cells), something very fundamental is changing: your day is determined by the needs of your culture, the critters you let into your habitat are dominating your daily routines.  You have to assure that they are comfortable, reproduce, even if the preferred living condition would be different.

To make flies my housemates was an idea that came to me because i was looking for a living being to serve as an object of observation. It should exhibit interesting collective behavior, and have a significance in science and our civilized way of life.

That’s how I ended up with flies. Neuroscientists have chosen flies because their brains are more simple than ours. If one looks at the behavior of the flies, one can perceive that their activities e.g. during their ‘party hour’ are already very complex.

One sees flying fruit flies, sitting, chasing and sniffing each other, visiting friends, courting flies, mating flies, fighting flies (as reported by Ralph Greenspan[1]).

Fruit flies sing. Beyond the well-known buzzing during their flight, they are also singing with their wings to communicate with each other. Of these songs, sinus tone and pulse tone are known, the latter of those being semantically encoded But I noticed sonic patterns, besides the scientifically described pulse song by Birgit Brüggemeier[2] and sine song like the buzzing of flying flies, but also other patterns that sound rather aggressive or like strange signals to make themselves heard.

To engage in an exchange with the flies, I draw on these behavioral patterns. With this work I offer you a possibility to talk directly to fruit flies through singing. When we humans sing, our songs often develop along already known melodies. But here, we don’t know which song to sing. We first have to listen.

To facilitate the contact, we have used a software with which we adapt the sound of human song to the song of flies. The software is based on audio mosaicing provided by the Fraunhofer Institut IIS[3]. Audio mosaicing is more advanced than the vocoder software we used for Drosophila Karaoke Bar. A vocoder adapts only the envelope, i.e. the characteristic curve of the amplitude of a sound that shapes the timbre, to a target sound. Audio mosaicing also adapts the frequency spectrum in its activation patterns to a pre-selected template.

How has the audio mosaicing changed the interaction with the flies? While with the vocoder I still had the feeling of speaking through an instrument, the audio mosaicing immediately induced me to sing with the usual pitches and timbres of fly song. This fundamentally changed the experience with the flies. The vocoder allowed to immediately stimulate the flies. Very soon it was apparent that this tool is good to arouse the flies. After an initial enthusiasm to be accepted into the world of flies, however, the question quickly arose: what happens next? Finally, you can only disappoint the flies as soon as they realize that you are not a suitable partner. And here’s where audio mosaicing became interesting. It allowed me to enter a sound sphere that was that of the flies.

Impressions from the opening

Installation in my flat in Berlin, experiments with a different type of petri dish and inverse microphones.
Schematic set-up of the installation

I have often and repeatedly tried to develop an understanding of the flies by singing with the audio mosaicing. I had to understand that the flies are lazy and quirky. If they don’t want something, they simply don’t care. When they are having fun, they don’t bother with us humans. They don’t need us –  except maybe our food. But that was it. When I was in the sound spectrum of the fly, I got into complaining, moaning, groaning, and every utterance was physical, immediate, not very sophisticated.

I ofteh wished that my voice was trained, that of a singer. And I could demand more artistry from it. But – and this seems to me the bottom line – in these times the obvious is the essential.

But what does all this work mean now: sharing one’s habitat with a so-called nuisance? And to sing laments or songs of suffering, of perseverance, of waiting, with this nuisance? Sartre (“Les mouches”) would give an unambiguous answer: Overcome the guilt. Step out of the shadows of the past…

I, however, wanted just to create an open situation, in which we observe and interact without bias before ‘knowing’ or making decisions about an opponent. And any artfulness would be out of place.

And I focused on fruit flies because they are part of that what we eliminate from daily life, what we eradicate with pest control, what does not belong to a culture of modernity. The confrontation with our own shortcomings seems to me essential for overcoming it.

Interaction Screen of the installation
set-up instruction

The work has been produced with the support of a grant from the Fraunhofer Artists in Lab Program.

But it would not have been possible without the technological design, programming, support and endurance of Felix Bonowski.


[1] Hans Dierick, Ralph Greenspan; Molecular analysis of flies selected for aggressive behavior. Nat Genet. 2006 Sep;38(9):1023-31 https://doi.org/10.1038/ng1864

[2] Birgit Brüggemeier, Mason A. Porter, Jim O. Vigoreaux, Stephen F. Goodwin; Female Drosophila melanogaster respond to song-amplitude modulations. Biol Open 15 June 2018; 7 (6): bio032003. doi: https://doi.org/10.1242/bio.032003

[3] Patricio López-Serrano, Christian Dittmar, Yiğitcan Özer, and Meinard Müller
NMF Toolbox: Music Processing Applications of Nonnegative Matrix Factorization
In Proceedings of the International Conference on Digital Audio Effects (DAFx), 2019.

 


Drosophila Karaoke Bar

01.12.2018

Installation setting at ars electronica 2019
Documentation of the inauguration at MoMuseum Vilnius 2019
A perfect feedback when ER plays on a Bach B50 Basstrombone!

Project Description in German
With our Drosophila Karaoke Bar, we want to look at one of the most widely used model organisms in medicine and brain research: the fruit fly, Drosophila. While humans in their everyday life keep away from flies, science uses these creatures for experiments. Drosophila are cheap, they reproduce quickly, have enough genetic resemblance to humans to study genetic diseases and a brain small enough for us to study.

One striking and little-known behaviour of flies are their mating songs. Fly males sing to females by vibrating their wings in rhythmical patterns. With our karaoke bar we want to offer a possibility to sing with flies, to experience their nature and culture in a shared sensual experience.

Look into the sound isolation box

Can our karaoke bar bring items from our high-tech culture back to our environment? Does it allow the audience to immerse themselves into science? Our attempt to invite people to sing with flies offers a performance to experience a holistic approach to scientific investigations. The setup discusses ecological questions: to which degree do we need to separate the habitats of humans and flies to feel comfortable? Which measures are necessary to make their faint songs audible to humans? How does a laboratory environment affect the behaviour of flies? Under which conditions are we able to enjoy their presence?

The installation invites visitors to establish a direct exchange with fruit flies through a technical interface. A software is transforming human speech into signal that can be perceived by flies. It allows auditory feedback between people and animals. For blending human and fly songs we will use a special signal processing vocoder provided by Berd Edler of the Fraunhofer Institute for Integrated Circuits.

Visitors are requested to talk and sing with flies. Birgit Brüggemeier, Neuroscientist and fly researcher explains in a video the meaning of the separate constituents of fly song. She informs listeners about the syntax and semantics of Drosophila songs, in order to give the visitors a better understanding of fly communication. The video encourages the visitors to sing and speak to flies.

the interior of the sound isolation box

A sound visualization in 2D enhances the auditory perception of the sphere of the flies with a visual monitoring of fly songs on a screen: the location, amplitude and pattern of the sound sources shall help the performer to identify its influence on the fly behaviour.

A large pile of sand covers with its weight the habitat of flies to isolate their buzzing against the noise of humans shows. The massiv sand pile repesents the sensual and the semantic gap between a fly and a human.

Inside the box and under the sand pile
Karaoke Bar @ MOMuseum Vilnius

In a future version, another set of headphones offer an ‘anthropocentric’ view point on flies: we track the occurring frequencies of our fly community, constituting in courtship songs and flight sounds (differing from about 1 octave). A specially designed software enhances the real-time sounds by modulating them with previously found chords. This software raises the question: are there more hidden patterns of communication within fly songs than described by science yet?

Fly song – a rendering of a generative software

With Karaoke Bar we learn to become silent and careful to be able to hear the voice of Drosophila. Our setting offers a possibility to communicate with Drosophila at eye (ear) level. By concentrating on Drosophilas proper way of expression (what kind of signals are they sending to their surroundings? How are they communicating? How does it sound if they are approaching their comrades? What do they want to negotiate? What are our common windows of perception?), we want to circumvent an anthropocentric world view. The installation not only translates the signals of drosophila (sonifying and visualizing, as it was done earlier), but allows a shared practice in a direct feedback situation, offering a novel sensual experience.

Ursula Damm (Artist, Project Lead)

Birgit Brüggemeier (Neuroscientist, former Fly Researcher)

Felix Bonowski (Programming, Interface)