The CONTACT ZONE responds in a multidimensional way to the different situations with the components light, vegetation, and encounter.
A rhythmic illumination in the cycles of the wind is the leitmotiv of our design. Wind-harvesting along the track captures and visualizes the airstream of passing trains. Animated, pulsating light strips and sound installations accompany the urban spaces along the track. Linear light elements are erected in front of the noise barriers and light up in the rhythm of the airflow, which is picked up by small microphones. Light intensity, orientation (to the green? To the people?) and relation to the surrounding urban space and react to different light conditions. For shaded places, as well as under bridges or in tunnels, we propose another, animated form of lighting, which also absorbs the energy of the Fahrtwind, but let it live on in the urban space as a source of light for people and plants animated. For this purpose, we propose compact, small-scale lighting installations that function according to the principle of cellular automata.
In this way, the lamps can carry the energy of the airstream profitably for people and plants into the urban space.
The lighting elements and installations should not only illuminate the urban space for people, but at the same time also enable and promote cryptogamous growth – algae, lichens, mosses – on the noise barriers, accompanying retaining walls, and the luminaires themselves. These bind huge amounts of carbon dioxide and nitrogen from the atmosphere and thus also positively influence the urban climate. The amount of nitrogen fixed by cryptogamic layers and thus supplied to the soil and other living organisms is equal to half of the nitrogen naturally fixed on land, which is particularly important for the development of ecosystems, as nitrogen is often the limiting nutrient component. They are therefore extremely effective in the smallest area of urban space.
The proposal was made by a team Green Noise, Berlin (Ursula Damm), Dr. Klaus Fritze, Rudolph Langner – Station C23 – Architecture and Landscape Architecture, Andreas Hecker (Bauhaus University Weimar), Felix Bonowski.
During the time of the pandemic I was experiencing our biological condition in a new way: Pandemics are teaching us about spheres, proximities, our distance behavior. We are asking ourselves, if we have been paying enough attention to the safety of our sphere of life so far and who we let come close. I called this period „time of flies“, mainly as I was living in my one room apartment in Berlin alone with fruitflies – cultures of fruit flies in my home.
As soon as you work with a biological organism (animals or cells), something very fundamental is changing: your day is determined by the needs of your culture, the critters you let into your habitat are dominating your daily routines. You have to assure that they are comfortable, reproduce, even if the preferred living condition would be different.
To make flies my housemates was an idea that came to me because i was looking for a living being to serve as an object of observation. It should exhibit interesting collective behavior, and have a significance in science and our civilized way of life.
That’s how I ended up with flies. Neuroscientists have chosen flies because their brains are more simple than ours. If one looks at the behavior of the flies, one can perceive that their activities e.g. during their ‘party hour’ are already very complex.
One sees flying fruit flies, sitting, chasing and sniffing each other, visiting friends, courting flies, mating flies, fighting flies (as reported by Ralph Greenspan[1]).
Fruit flies sing. Beyond the well-known buzzing during their flight, they are also singing with their wings to communicate with each other. Of these songs, sinus tone and pulse tone are known, the latter of those being semantically encoded But I noticed sonic patterns, besides the scientifically described pulse song by Birgit Brüggemeier[2] and sine song like the buzzing of flying flies, but also other patterns that sound rather aggressive or like strange signals to make themselves heard.
To engage in an exchange with the flies, I draw on these behavioral patterns. With this work I offer you a possibility to talk directly to fruit flies through singing. When we humans sing, our songs often develop along already known melodies. But here, we don’t know which song to sing. We first have to listen.
To facilitate the contact, we have used a software with which we adapt the sound of human song to the song of flies. The software is based on audio mosaicing provided by the Fraunhofer Institut IIS[3]. Audio mosaicing is more advanced than the vocoder software we used for Drosophila Karaoke Bar. A vocoder adapts only the envelope, i.e. the characteristic curve of the amplitude of a sound that shapes the timbre, to a target sound. Audio mosaicing also adapts the frequency spectrum in its activation patterns to a pre-selected template.
How has the audio mosaicing changed the interaction with the flies? While with the vocoder I still had the feeling of speaking through an instrument, the audio mosaicing immediately induced me to sing with the usual pitches and timbres of fly song. This fundamentally changed the experience with the flies. The vocoder allowed to immediately stimulate the flies. Very soon it was apparent that this tool is good to arouse the flies. After an initial enthusiasm to be accepted into the world of flies, however, the question quickly arose: what happens next? Finally, you can only disappoint the flies as soon as they realize that you are not a suitable partner. And here’s where audio mosaicing became interesting. It allowed me to enter a sound sphere that was that of the flies.
Impressions from the opening
I have often and repeatedly tried to develop an understanding of the flies by singing with the audio mosaicing. I had to understand that the flies are lazy and quirky. If they don’t want something, they simply don’t care. When they are having fun, they don’t bother with us humans. They don’t need us – except maybe our food. But that was it. When I was in the sound spectrum of the fly, I got into complaining, moaning, groaning, and every utterance was physical, immediate, not very sophisticated.
I ofteh wished that my voice was trained, that of a singer. And I could demand more artistry from it. But – and this seems to me the bottom line – in these times the obvious is the essential.
But what does all this work mean now: sharing one’s habitat with a so-called nuisance? And to sing laments or songs of suffering, of perseverance, of waiting, with this nuisance? Sartre (“Les mouches”) would give an unambiguous answer: Overcome the guilt. Step out of the shadows of the past…
I, however, wanted just to create an open situation, in which we observe and interact without bias before ‘knowing’ or making decisions about an opponent. And any artfulness would be out of place.
And I focused on fruit flies because they are part of that what we eliminate from daily life, what we eradicate with pest control, what does not belong to a culture of modernity. The confrontation with our own shortcomings seems to me essential for overcoming it.
The work has been produced with the support of a grant from the Fraunhofer Artists in Lab Program.
But it would not have been possible without the technological design, programming, support and endurance of Felix Bonowski.
[1] Hans Dierick, Ralph Greenspan; Molecular analysis of flies selected for aggressive behavior. Nat Genet. 2006 Sep;38(9):1023-31 https://doi.org/10.1038/ng1864
[2] Birgit Brüggemeier, Mason A. Porter, Jim O. Vigoreaux, Stephen F. Goodwin; Female Drosophila melanogaster respond to song-amplitude modulations. Biol Open 15 June 2018; 7 (6): bio032003. doi: https://doi.org/10.1242/bio.032003
[3] Patricio López-Serrano, Christian Dittmar, Yiğitcan Özer, and Meinard Müller NMF Toolbox: Music Processing Applications of Nonnegative Matrix Factorization In Proceedings of the International Conference on Digital Audio Effects (DAFx), 2019.
Art installation by Ursula Damm and Felix Bonowski, 2021
Curator: Yvonne Volkart
Commissioned for the Flux building by Eawag: Swiss Federal Institute of Aquatic Science and Technology
Technique: Two channel projection; one based on live camera footage and neural network learning rules; simulation based on Perlin noise, Navier Stokes-Solver, reaction diffusion kinetics parameterized with live measurements of Oxygen content, temperature, turbidity.
Kontinuum is a generative 2 channel projection based on live data of the Chriesbach, a rivulet flowing alongside the Institute of Water Research. The two projections represent a certain mode of “reality” of the Chriesbach and its flux throughout the year. Both translate data of seasonal variations, color patterns and physical principles of the stream into sensual images that associate impressionist and Japanese painting. By bringing the outside into the inside, the water into the Flux building, the object of observation to the site of its investigation, the installation reflects the Kontinuum of the stream and the function of the house in an aesthetic way.
The colored projection collects images of the Chriesbach stream and its inhabitants. Real time images of three cameras are passed through a graphics shader which is based on classical neural network learning rules that “remember” colors in areas of high activity. The resulting video is a collage of combined aspects of the streams’ visual appearance from different times and viewing angles. With their daily changes they serve as a kind of aesthetic weather report.
The black and white projection is a live simulation of a fluid meandering through a rock-strewn valley. Based on an ecosystem of nutrients, primary producers, and grazing microorganisms of the stream, it manifests digitally “how the world would look like, if nature followed these rules”. The formulas that govern the shape of the landscape, the dynamics of flow, and the evolution of life in the simulation are parameterized with values derived from actual real-time measurements of physical water properties. The measurements are performed by a station operated by the research institute just a few meters from where the cameras look onto the stream. Correspondences between measurements and model parameters are chosen so that seasonal changes (in temperature), daily rhythms (in oxygen saturation from photosynthesis) and occasional events (turbidity caused by thunderstorms and construction work) leave their traces in the graphics. Transforming from a valley with a few large boulders to a (virtual) riverbed with many small pebbles, from one emergent biological pattern into another, from a slowly meandering flow into a violent gusher, the simulation reveals itself as a being in permanent flux.
At the right border of each projection, the image logic of the other projection intervenes, so that the color-data of the live stream and the patterns of the black and white simulation intersect: contrasty movements in the colored projection (e.g. reflections of light or swimming leaves) become lines, scratches and holes in the black and white one. They appear as forces, which wipe out organic life and destroy the image. Thus, it becomes clear, that no image and no “reality” stand for themselves, rather they can be questioned, disrupted, or interpreted by manifold approaches.
Membrane is an art installation which was produced as the main work of a similarly named exhibition at the Kunstverein Tiergarten in Berlin early 2019. It builds on a series of generative video installations with real time video input. Membrane allows the viewer to interact directly with the generation of the image by a neural network, here the so-called TGAN algorithm. An interface allows to experience the ‘imagination’ of the computer, guiding the visitor according to curiosity and personal preferences.
The images of Membrane are derived from a static video camera observing a street scene in Berlin. A second camera is positioned in the exhibition space and can be moved around at will. Two screens are showning both scenes in realtime.
In my earlier artistic experiments within this context we considered each pixel of a video data stream as an operational unit. One pixel learns from colour fragments during the running time of the programme and delivers a colour which can be considered as the sum of all colours during the running time of the camera. This simple method of memory creates something fundamentally new: a recording of patterns of movement at a certain location.
On a technical level, Membrane not only controls pixels or clear cut details of an image, but image ‘features’ which are learnt, remembered and reassembled. With regards to the example of colour: we choose features but their characteristics are delegated to an algorithm. TGANs (Temporal Generative Adversarial Nets) implement ‘unsupervised learning’ through the opposing feedback effect of two subnetworks: a generator produces short sequences of images and a discriminator evaluates the artificially produced footage. The algorithm has been specifically designed to produce representations of uncategorised video data and – with the help of it – to produce new image sequences. (Temporal Generative Adversarial Nets).
We extend the TGAN algorithm by adding a wavelet analysis which allows us to interact with image features as opposed to only pixels from the start. Thus, our algorithm allows us to ‘invent’ images in a more radical manner than classical machine learning would allow.
In practical terms, the algorithm speculates on the the basis of its learning and develops its own, self organised temporality. However, this does not happen without an element of control: feature classes from a selected data set of videos are chosen as target values. In our case, the dataset consists of footage from street views of other cities from around the world, taken while travelling.
The concept behind this strategy is not to adapt our visual experience in Berlin to global urban aesthetics but rather to fathom the specificity and to invent by associations. These associations can be localised, varied and manipulated within the reference dataset. Furthermore, our modified TGAN algorithm will generate numerous possibilities to perform dynamic learning on both short and long timescales and ultimately to be controlled by the user/ visitor. The installation itself allows the manipulation of video footage from a unchanged street view to purely abstract images, based on the found features of the footage. The artwork wants to answer the question of how we want to alter realistic depictions. What are the distortions of ‘reality’ we are drawn to? Which fictions are lying behind these ‘aberrations’? Which aspects of the seen do we neglect? Where do we go with such shifts in image content and what will be the perceived experience at the centre of artistic expression?
From an artistic point of view, the question now arises, how can something original and new be created with algorithms? This is the question behind the software design of Membrane. Unlike other AI-Artworks we don’t want to identify something specific within the video footage, but rather we are interested in how people perceive the scenes. That is why our machines look at smaller, formal image elements and features which intrinsic values we want to reveal and strengthen. We want to expose the visitors to intentionally vague features: edges, lines, colours, geometrical primitives, movement. Here, instead of imitating a human way of seeing and understanding, we reveal the machine’s way to capture, interpret and manipulate visual input. Interestingly, the resulting images resemble pictorial developments of classical modernism (progressing abstraction on the basis of formal aspects) and repeat artistic styles like Pointilism, Cubism and Tachism in a uniquely unintentional way. These styles fragmented the perceived as part of the pictorial transformation into individual sensory impressions. Motifs are now becoming features of previously processed items and are successively losing their relation to reality. At the same time, we question whether these fragmentations of cognition are proceeding in an arbitrary way or whether there may be other concepts of abstraction and imagery ahead of us?
From a cultural perspective, there are two questions remaining: – How can one take decisions within those aesthetic areas of action (parameter spaces)? – Can the shift of the perspective from analysis to fiction help to asses our analytical procedures in a different way – understanding them as normative examples of our societal fictions serving predominantly as a self-reinforcement of present structures?
Thus unbiased artistic navigation within the excess/surplus of normative options of actions might become a warrantor for novelty and the unseen.
We are using Max/MSP) to indentify the pitch of natural fly songs and modulates them with their own tones. Naturally occuring harmonics are shown in the real-time illustration. First shown at the Mo Museum in Vilnius, May 2019.
Project Description in German With our Drosophila Karaoke Bar, we want to look at one of the most widely used model organisms in medicine and brain research: the fruit fly, Drosophila. While humans in their everyday life keep away from flies, science uses these creatures for experiments. Drosophila are cheap, they reproduce quickly, have enough genetic resemblance to humans to study genetic diseases and a brain small enough for us to study.
One striking and little-known behaviour of flies are their mating songs. Fly males sing to females by vibrating their wings in rhythmical patterns. With our karaoke bar we want to offer a possibility to sing with flies, to experience their nature and culture in a shared sensual experience.
Can our karaoke bar bring items from our high-tech culture back to our environment? Does it allow the audience to immerse themselves into science? Our attempt to invite people to sing with flies offers a performance to experience a holistic approach to scientific investigations. The setup discusses ecological questions: to which degree do we need to separate the habitats of humans and flies to feel comfortable? Which measures are necessary to make their faint songs audible to humans? How does a laboratory environment affect the behaviour of flies? Under which conditions are we able to enjoy their presence?
The installation invites visitors to establish a direct exchange with fruit flies through a technical interface. A software is transforming human speech into signal that can be perceived by flies. It allows auditory feedback between people and animals. For blending human and fly songs we will use a special signal processing vocoder provided by Berd Edler of the Fraunhofer Institute for Integrated Circuits.
Visitors are requested to talk and sing with flies. Birgit Brüggemeier, Neuroscientist and fly researcher explains in a video the meaning of the separate constituents of fly song. She informs listeners about the syntax and semantics of Drosophila songs, in order to give the visitors a better understanding of fly communication. The video encourages the visitors to sing and speak to flies.
A sound visualization in 2D enhances the auditory perception of the sphere of the flies with a visual monitoring of fly songs on a screen: the location, amplitude and pattern of the sound sources shall help the performer to identify its influence on the fly behaviour.
A large pile of sand covers with its weight the habitat of flies to isolate their buzzing against the noise of humans shows. The massiv sand pile repesents the sensual and the semantic gap between a fly and a human.
In a future version, another set of headphones offer an ‘anthropocentric’ view point on flies: we track the occurring frequencies of our fly community, constituting in courtship songs and flight sounds (differing from about 1 octave). A specially designed software enhances the real-time sounds by modulating them with previously found chords. This software raises the question: are there more hidden patterns of communication within fly songs than described by science yet?
With Karaoke Bar we learn to become silent and careful to be able to hear the voice of Drosophila. Our setting offers a possibility to communicate with Drosophila at eye (ear) level. By concentrating on Drosophilas proper way of expression (what kind of signals are they sending to their surroundings? How are they communicating? How does it sound if they are approaching their comrades? What do they want to negotiate? What are our common windows of perception?), we want to circumvent an anthropocentric world view. The installation not only translates the signals of drosophila (sonifying and visualizing, as it was done earlier), but allows a shared practice in a direct feedback situation, offering a novel sensual experience.
Ursula Damm (Artist, Project Lead)
Birgit Brüggemeier (Neuroscientist, former Fly Researcher)
When I left the county side and moved to a city I begun to miss the sound of the fields and the forrest. And when I later returned to the small village in the middle of vineyards, called Diedesfeld, something was gone. I took me a while to figure out that I missed the sounds of insects. And that this sound was like a confirmation of a strong, ecological balance. Science proofed only years later that insecticides diminished insects up to 80 % of their former presence.
Christina Meissner, Teresa Carrasco and me gathered in Weimar to experience the singing of Chironomid midges (Chironomus riparius, commonly used in ecotoxicology) and their aibility to react to our music. In a direct feedback situation between humans and animals technology should be used only to adapt our senses and make it easier to understand the message of the other. In a first performace we noticed that Christina Meissner with her Cello was able to stimulate lazy midges (stimulation – Soundexample from our first session ) to start swarming intensively (swarming in dialogue). We were thrilled to notice that it was so easy and obvious that humans and midges interfere. In our second concert, it was no longer necessary / voluntary to force midges into swarming, but instead to develop a kind of q/a, to listen and to respond to the phrases of the mosquitoes. Our first performance ready for publishing (in full length) is here.
Our concerts can be seen as a call for the subtle atmosphere which allows insects to stay in our neighbourhood. And our readiness to listen to them.
If you appreciate midges, you might also look at the following video, showing synchronized swarms at Taubensuhl, Palatinat, Germany. They can be observed only some days per year – I was very happy to record them. When I came back with my sound equipment, they were gone.
On the front wall of the Schadowstrasse underground station, an LED wall displays a generative video. In front of the wall, a light shaft extends to the surface of the plaza where a video camera is set up. The camera continuously films passing pedestrians on the plaza and streams the feed to a specially developed generative software application (coded by Felix Bonowski) which derives proposed geometries for structures based on the movement patterns of the pedestrians. These interpretations of the real-time video generate new geometries for the location and propose axes and parcels.
Two elevators, to the left and right of the large video image, lead from the plaza to the rail platform.
Pattern drawings on aerial photos of Düsseldorf, Schadowstrasse
Turnstile (Drehkreuz) from resoutionable on Vimeo. On the platform, the geometric structures can be heard as a sound interpretation (by Yunchul Kim). At the centre of the artistic intervention is the video image and its artistic concept. The concept is reflected in the design of the entrance areas. Plates are inserted in the blue glass of the underground station at 21 locations, which display geometries over districts of Düsseldorf.
In the east concourse is the aerial image of the city of Düsseldorf that was analysed according to the geometric concept.
As excerpts from this aerial picture, 16 locations in Düsseldorf were interpreted at the level of a local aerial image. These urban areas were described with regular polygons as energy centres which fitted themselves together through the development of the city architecture (see the text on the concept of the generated patterns).
The fine structure of the patterns juxtaposes both the sensibility of nature and the human, formative gestures against the massive edifice, calling to mind a mode of formation that creates sweeping interconnections through the symbiotic organisation of a multitude of individual elements. In doing so, this formative process completes the social principle through which individuals experience their effect on the whole.
The pattern drawings are generated in slow steps: First a line drawing is created over the image of the city. As this progresses, important motion axes of traffic and pedestrians are emphasised. The areas these axes enclose become polygons. At this point, the angles of the lines and axes are examined in the search for whole-number fractions of regular polygons.
The smallest polygon integrating all of the symmetries at the location (for instance, five-angled and four-angled fragments would be assembled into a 20-sided polygon) is then used to describe an intersection.
A subsequent step is the search for connections (network) between large neighbouring polygons.
Work with the aerial images revealed that the city centre has very small polygons, while outer areas have a significantly more expansive structure. Thus, density is indicated by the presence of small polygons and complex symmetries. Often, the transition from non-rectangles to rectangles can indicate historical breaks in the urban landscape. In this way, the interpretations represent a study of the settlement and planning history of the city.
Drawing Nr. 24, Düsseldorf Flehe
The sound installation
The generative video installation interprets traces of movement created by geometric “agents.” The activity of these agents is translated into sounds which track the visual artefacts. As such, the sounds form the noise that the virtual artefacts generate in their world, and thus represent and extended artistic “level of reality” of the installation.
PROGRAM
Select a location (origin)
Determine the movement axes of people and traffic
Look to see if these axes are at angles to one another, which when mirrored and rotated can form a polygon, the sides of which all extend outward equally
Draw this polygon to approximate the natural geometry of the location
Look to see if, starting from this, the intrinsic geometries of the location can form a surface structure, (tessellation) that periodically repeats the original geometries
Determine whether and how, in the aerial image of the location, the areas fit together in the revealed geometry of the place
Enhance existing structures by developing their geometries
Connect existing structures into the logic of the original geometry
Konzept: Ursula Damm Programmierung: Felix Bonowski Sound: Yunchul Kim
The Urban Development Kit provides tools to ameliorate the atmosphere in contemporary cities. The Urban Development Kit is a collection of tools. Over the time the website aims to become a resource of ideas, concepts and tools for a citizen-driven urban design.
Visualization for the workshop in Osaka
One of our kits supports watchful citizens and plants to compete with pavement, concrete and asphalt. It helps plants to interact with modern cities, to prevail against soil sealing. A website and an interactive map enables the people to collect photos of “asphalt flowers” in Helsinki and other cities and to monitor the progress of the “cultivation”. With respect to urban environmental research, the urban development kit is a statement about the importance to counteract the sealing of the surfaces in the city. Accordingly, in the exhibition can be seen design for urban surfaces which are based on the geometry the plants themselves.
Patterns on polygonium avicularepattern drawing on polygonium aviculare
The work has been developed for the Art&HENVI project, organized by the finish Bioart Society.
counteracting soil sealing
In 2014, a new version of the urban development kit was presented at a creative cloud workshop, organized from ars electronica (see the fotos from the workshop)
Installation view at the Hybris Exhibition at ACC WeimarDetail – culture items for breeding chironomid midges
Non-biting midges (chironomidae) are bred in an aquarium. Inside the aquarium mosquito eggs and larvae swim in sand and water. They are ventilated and supplied with abundant artificial daylight. The choice of midges (Chironomus Riparius, a laboratory strain) allows for captive breeding.
Aquauarium, Ipad with video from military airshows flying in loopsLooping airplanes behind swarming midges
Pink elephants for Hollywood
The scientific paper
the text we highlighted
aggression and genes
about swarms and the few we know about them
swarming midges
aggression
field observations - scientific practices beyond the laboratory
scientific practices beyond the laboratory
Ingrid Bergmann about trained flies in Hollywood
contributors
For a performance I invited Christina Meissner to improvise on the theme of the wingbeat sound. We could experience that the tracks of the midges were visibly influenced especially by dark plucking sounds.
"the outline of paradise" - view into the soundbox
computer control (traces) of sound input
video camera, loudspeaker
Christina Meissner playing
corresponding vibrations
playing with the swarm
the midges, loudspeakers
swarm
Christina Meissner with control monitor
traces of midges on monitor
"the outline of paradise" - the soundbox
The work at Hybris Exhibition Halle 14 Leipzig, Photo: Claus BachThe work at Hybris Exhibition Halle 14 Leipzig, Photo: Claus Bach
This setting allows to find out how swarms develop and how they can be influenced. This installation follows “sustaibable luminosity” and explores the possibilities to train migdes and to pass this behaviour to the next generations