Kontinuum

02.01.2020

Art installation by Ursula Damm and Felix Bonowski, 2021

Curator: Yvonne Volkart

Commissioned for the Flux building by Eawag: Swiss Federal Institute of Aquatic Science and Technology

Technique: Two channel projection; one based on live camera footage and neural network learning rules; simulation based on Perlin noise, Navier Stokes-Solver, reaction diffusion kinetics parameterized with live measurements of Oxygen content, temperature, turbidity.

Kontinuum auf der Website der EAWAG

Felix Bonowski controling the “bachcam” screen, EAWAG Flux Building, Zürich
View of the projection – simulation and generative video

Kontinuum is a generative 2 channel projection based on live data of the Chriesbach, a rivulet flowing alongside the Institute of Water Research. The two projections represent a certain mode of “reality” of the Chriesbach and its flux throughout the year. Both translate data of seasonal variations, color patterns and physical principles of the stream into sensual images that associate impressionist and Japanese painting. By bringing the outside into the inside, the water into the Flux building, the object of observation to the site of its investigation, the installation reflects the Kontinuum of the stream and the function of the house in an aesthetic way.

The colored projection collects images of the Chriesbach stream and its inhabitants. Real time images of three cameras are passed through a graphics shader which is based on classical neural network learning rules that “remember” colors in areas of high activity. The resulting video is a collage of combined aspects of the streams’ visual appearance from different times and viewing angles. With their daily changes they serve as a kind of aesthetic weather report.

The black and white projection is a live simulation of a fluid meandering through a rock-strewn valley. Based on an ecosystem of nutrients, primary producers, and grazing microorganisms of the stream, it manifests digitally “how the world would look like, if nature followed these rules”. The formulas that govern the shape of the landscape, the dynamics of flow, and the evolution of life in the simulation are parameterized with values derived from actual real-time measurements of physical water properties. The measurements are performed by a station operated by the research institute just a few meters from where the cameras look onto the stream. Correspondences between measurements and model parameters are chosen so that seasonal changes (in temperature), daily rhythms (in oxygen saturation from photosynthesis) and occasional events (turbidity caused by thunderstorms and construction work) leave their traces in the graphics. Transforming from a valley with a few large boulders to a (virtual) riverbed with many small pebbles, from one emergent biological pattern into another, from a slowly meandering flow into a violent gusher, the simulation reveals itself as a being in permanent flux.

At the right border of each projection, the image logic of the other projection intervenes, so that the color-data of the live stream and the patterns of the black and white simulation intersect: contrasty movements in the colored projection (e.g. reflections of light or swimming leaves) become lines, scratches and holes in the black and white one. They appear as forces, which wipe out organic life and destroy the image. Thus, it becomes clear, that no image and no “reality” stand for themselves, rather they can be questioned, disrupted, or interpreted by manifold approaches.

Video documentation of the installation
Real time play-out of the so-called ‘bachcam’
Cameras observing the Chriesbach stream
Positioning of the cameras
Real time data from Chriesbach for the simulation screen
Felix Bonowski installing the cameras
Video auf der Website der Eawag
Schematic drawing of data flow
This is the concept video with samples of different interesting moments

U-Bahnhof Schadowstrasse

29.01.2016

‘Turnstile’ – an interactive installation

Turnstile, generative Installation von Ursula Damm (Foto Thomas Mayer)

On the front wall of the Schadowstrasse underground station, an LED wall displays a generative video. In front of the wall, a light shaft extends to the surface of the plaza where a video camera is set up. The camera continuously films passing pedestrians on the plaza and streams the feed to a specially developed generative software application (coded by Felix Bonowski) which derives proposed geometries for structures based on the movement patterns of the pedestrians. These interpretations of the real-time video generate new geometries for the location and propose axes and parcels.

Turnstile, Ursula Damm, Schadowstrasse Düsseldorf 2016 (Foto by Thomas Mayer)

Two elevators, to the left and right of the large video image, lead from the plaza to the rail platform.

Pattern drawings on aerial photos of Düsseldorf, Schadowstrasse

Turnstile (Drehkreuz) from resoutionable on Vimeo.
On the platform, the geometric structures can be heard as a sound interpretation (by Yunchul Kim). At the centre of the artistic intervention is the video image and its artistic concept. The concept is reflected in the design of the entrance areas. Plates are inserted in the blue glass of the underground station at 21 locations, which display geometries over districts of Düsseldorf.

Foto: Ursula Damm und Felix Bonowski, Credits: netzwerkarchitekten, Darmstadt, © 2015 by Jörg Hempel; www.joerg-hempel-com
From the stairs to the screen, Foto: Achim Kukulies
Pattern drawing on aerial photo of Düsseldorf, photo by Achim Kukulies

In the east concourse is the aerial image of the city of Düsseldorf that was analysed according to the geometric concept.

Geometric pattern generation on the spacial structure of Duesseldorf

As excerpts from this aerial picture, 16 locations in Düsseldorf were interpreted at the level of a local aerial image. These urban areas were described with regular polygons as energy centres which fitted themselves together through the development of the city architecture (see the text on the concept of the generated patterns).

Schadowstrasse, pedestrian aerea, location of camera; Photo Achim Kukulies

The fine structure of the patterns juxtaposes both the sensibility of nature and the human, formative gestures against the massive edifice, calling to mind a mode of formation that creates sweeping interconnections through the symbiotic organisation of a multitude of individual elements. In doing so, this formative process completes the social principle through which individuals experience their effect on the whole.

The pattern drawings are generated in slow steps: First a line drawing is created over the image of the city. As this progresses, important motion axes of traffic and pedestrians are emphasised. The areas these axes enclose become polygons. At this point, the angles of the lines and axes are examined in the search for whole-number fractions of regular polygons.

The smallest polygon integrating all of the symmetries at the location (for instance, five-angled and four-angled fragments would be assembled into a 20-sided polygon) is then used to describe an intersection.

A subsequent step is the search for connections (network) between large neighbouring polygons.

Work with the aerial images revealed that the city centre has very small polygons, while outer areas have a significantly more expansive structure. Thus, density is indicated by the presence of small polygons and complex symmetries. Often, the transition from non-rectangles to rectangles can indicate historical breaks in the urban landscape. In this way, the interpretations represent a study of the settlement and planning history of the city.

Drawing Nr. 24, Düsseldorf Flehe

The sound installation

The generative video installation interprets traces of movement created by geometric “agents.” The activity of these agents is translated into sounds which track the visual artefacts. As such, the sounds form the noise that the virtual artefacts generate in their world, and thus represent and extended artistic “level of reality” of the installation.

PROGRAM

  • Select a location (origin)
  • Determine the movement axes of people and traffic
  • Look to see if these axes are at angles to one another, which when mirrored and rotated can form a polygon, the sides of which all extend outward equally
  • Draw this polygon to approximate the natural geometry of the location
  • Look to see if, starting from this, the intrinsic geometries of the location can form a surface structure, (tessellation) that periodically repeats the original geometries
  • Determine whether and how, in the aerial image of the location, the areas fit together in the revealed geometry of  the place
  • Enhance existing structures by developing their geometries
  • Connect existing structures into the logic of the original geometry

Konzept: Ursula Damm
Programmierung: Felix Bonowski
Sound: Yunchul Kim

Article on the production of the artwork in german

Turnstile article english translation

Text von Georg Trogemann über den Besuch der Eröffnung

official website Stadt Düsseldorf

English text on the production of the artwork

 

Urban development kit

29.11.2014

The Urban Development Kit  provides tools to ameliorate the atmosphere in contemporary cities. The Urban Development Kit is a collection of tools. Over the time the website aims to become a resource of ideas, concepts and tools for a citizen-driven urban design.

Visualization for the workshop in Osaka

One of our kits supports watchful citizens and plants to compete with pavement, concrete and asphalt. It helps plants to interact with modern cities, to prevail against soil sealing. A website and an interactive map enables the people to collect photos of “asphalt flowers” in Helsinki and other cities and to monitor the progress of the “cultivation”. With respect to urban environmental research, the urban development kit is a statement about the importance to counteract the sealing of the surfaces in the city. Accordingly, in the exhibition can be seen design for urban surfaces which are based on the geometry the plants themselves.

Patterns on polygonium aviculare
pattern drawing on polygonium aviculare

The work has been developed for the Art&HENVI project, organized by the finish Bioart Society.

counteracting soil sealing

In 2014, a new version of the urban development kit was presented at a creative cloud workshop, organized from ars electronica (see the fotos from the workshop)

Plese look at the presentation of the workshop

cosmical sperm osaka

Chromatographic Ballads

16.12.2013

Test situation in my studio with brain device, operator Lisa and Martin Schneider

The installation received a honorary mention at VIDA 15.0

The Artwork

Martin explaining the Neurovision software

Chromatographic Orchestra is an artistic installation which allows a visitor to direct a software framework with an EEG device. In an exhibition environment with semi-transparent video screens a visitor is sitting in an armchair and learns to navigate unconsciously – with his/her brain waves the parameter space of our software – Neurovision.

Neurovision interacts with live video footage of the location of the exhibition and its surroundings. By navigating with his/her own brain waves the visitor can define and navigate the degree of abstraction of a generative (machine learning) algorithm,  performed on the footage of different, nearby video cameras.

Vizualisation for ideal set-up
ideal set-up for the installation
Our Operator Lisa sitting in front of the screen with a EEG device

The installation refers back to painting techniques in the late 19th and early 20th century, when painting became more an analysis of the perception of a setting then a mere representation of the latter. Impressionism and Cubism were fragmenting the items of observation while the way of representation was given by the nature of the human sensory system.

The installation “chromatographic orchestra” does not apply arbitrary algorithms to the live footage: we developed a software – the Neurovision framework – which mimics the visual system of the human brain. Thus we question whether our algorithms meet the well being of the spectator by anticipating processing steps of our brain.

Artistic Motivation

How much complexity can our senses endure, or rather how could we make endurable what we see and hear? Many communication tools have been developed, to adjust human capabilities to the requirements of the ever more complex city.

Our installation poses the opposite question: How can information emerging from the city be adjusted to the capabilities of the human brain, so processing them is a pleasure to the eye and the mind?

At the core of our installation is the NeuroVision Sandbox, a custom made framework for generative video processing in the browser based on WebGL shaders.

Inside this Sandbox we developed several sketches, culminating in the
“Chromatographic Neural Network”, where both optical flow and color information of the scene are processed, inspired by information processing in the human visual system.

We critically assess the effect of our installation on the human sensory system:

  • Does it enhance our perception of the city in a meaningful way?
  • Can it and if so – how will it affect the semantic level of visual experience?
  • Will it create a symbiotic feedback loop with the visitor’s personal way to interpret a scene?
  • Will it enable alternate states of consciousness? Could it even allow visitors to experience the site in a sub-conscious state of “computer augmented clairvoyance”

Installation

In a location close to the site a single visitor directs a video-presentation on a large screen with a setup we like to call “the Neural Chromatographic Orchestra” (NCO).
Our installation uses an EEG-Device (Emotiv NeuroHeadset) that lets visitors interact with a custom neural network. The setup allows visitors to navigate through various levels of abstraction by altering the parameters of the artificial neural net.

With the NCO device, a visitor can select and explore real-time views provided by three cameras located in public space with different perspectives on the passer-byes (birds-eye view and close-ups)

The installation is based on the NeuroVision Sandbox used in the development of “transits”.
Other than transits, chromatographic ballads uses multi-channel real-time video-input and enables a visitor to interact with irectly via biofeedback with the neural network.

The Neural Chromatographic Orchestra investigates how human perception reacts to the multifaceted visual impressions of public space via an artistic setting. Using an EEG-Device visitors can interact with a self-organizing neural network and explore real-time views of an adjacent hall from several perspectives and at various levels of abstraction.

Biological Motivation

The Chromatographic Neural Network is a GPU-based video processing tool. It was inspired by parallel information processing in the visual system of the human brain. Visual information processing inside the brain is a complex process involving various processing stages.The visual pathway includes the retina, the Lateral Geniculate Nucleus (LGN) and the visual cortex

Scheme of the optical tract with the image being processed (simplified): http://en.wikipedia.org/wiki/File:Lisa_analysis.png

Low-level visual processing is already active at the various layers of the retina. The Interconnection of neurons between retina layers, and the ability to retain information using storage or delayed feedback, allows for filtering the visual image in the space and time domain.

Both image filters and motion detection can easily be achieved by accumulating input from neurons in a local neighborhood, in a massively parallel way.

Our Chromatographic Neural Network uses this approach to cluster colors and to compute the visual flow (or retina flow ) from a video source. The resulting attraction-vectors and flow-vectors are used to transform the memory retained in the memory layer.

The visual output of the system directly corresponds to the state of the output layer of the neural network. The neural layers of the Chromatographic Neural Network, are connected to form a feedback loop. This giving rise to a kind of homeostatic-system that is structurally coupled to the visual input but develops its own dynamics over time.

The set-up

A visitor enters the site – a highly frequented passage, a spacious hall or a public place. Two videocameras, mounted on a tripod, can be moved around at will.

Another camera observes the passer-byes – their transits and gatherings – from an elevated location. The video footage from this site is streamed into a neighboring room – the orchestra chamber of the Neural Chromatographic Orchestra.

Here one can see – in front of a a large video wall a monitor displaying the videos from the adjacent room and the “orchestra pit” – an armchair equipped with a touch device and a neuro-headset. The video wall, showing abstract interpretations of the site itsself, should ideally be visible both from the orchestra pit and from the large hall.

The Orchestra Chamber

Inside the chamber the visitor is seated in a comfortable armchair and an assistant helps her put on and adjust the neuro-headset.

The orchestra chamber should be isolated from the public area as much as possible. A sense of deprivation from outside stimuli allows the visitor to gain control over her own perception and achieve a state of mind similar to meditation or clairvoyance.

The Orchestral Performance

Training Cognitive Control

A performance with the Neural Chromatographic Orchestra starts with a training of up to six mental actions, corresponding to the “push/pull”, “left/right“ and “up/down” mental motions provided by the Emotiv Cognitiv suite. The training typically lasts 10 to 30 minutes.

Playing the Sandbox

After successful training the visitor is asked to sit in front of the NeuroVision Sandbox:

The visitor in the orchestra chamber has three modes of conducting the neural network

  • either the birds-eye view or one of the cameras that take a pedestrian’s perspective
  • A graphical user interface lets her switch between different neural networks and control their parameters
  • A menu lets her choose any of the three cameras as a video source:
  • the NeuroHeadset allows to navigate the parameter space of the selected neural network

Conducting the Orchestra

Once the visitor feels comfortable conducting the NCO on the small screen, she can perform on the large screen, that is also visible from the outside.

On the public screen sliders are not shown, but the conductor may still use a tablet device to access the graphical user interface.

The current position in parameter spaces is represented by a 3d-cursor or wire-frame box, which is very helpful for making the transition from voluntary conduction moves, to a style of conducting that is more directly informed by immersion and interaction with the output of the Chromatographic Neural Network.

The Chromatographic Neural Network

The flow of information is arranged into several processing layers. To realize memory, each processing layer is in turn implemented as stack of one or more memory layers.This allows us to access the state of a neuron at a previous point in time.

Example

The video layer is made up of two layers, so the system can access the state of any input neuron at the current point in time, and its state in the previous cycle.

Processing Layers

The Video layer

The Video layer contains the input neurons. Each neuron corresponds to a pixel of the video source. The Video layer provides the input for the Flow layer.

The Ghost Layer

The Ghost layer represents a haunting image from the past. It implements the long term memory, that interferes and interacts with the current visual input. It does not change over time, and is provided as additional input to the Flow layer

The Flow layer

The Flow layer accumulates the input from the Video layer and the Ghost layer. Each Neuron aggregates input from its neighborhood in the Video Layer at times (t) and (t-1). The computed 2d vector is directly encoded into the the state of the neuron, creating a flow map.

The Blur layers

The Blur layers are used to blur the flow map. While the computation of visual flow is restricted to a very small neighborhood, the blur layer is needed to spread the flow information to a larger region, since flow can only be detected on the edge of motion.

For efficiency reasons the blur function is split into two layers, performing a vertical and a horizontal blur respectively.

Neuron Processing

The state of each neuron corresponds to an RGB color triplet. Every neuron of the Flow layer gets input from corresponding neurons inside a local neighborhood of the input layers. Each of those input samples corresponds to a single synapse. The vector from the center of the neuron towards the input neuron is referred to as the synapse vector.

Color Attraction

To achieve some kind of color dynamics, colors that are close in color space are supposed to attract each other.

The distance between synapse input and the neuron state in RGB color-space, serves as a weight, which is used to scale the synapse vector. The sum of scaled synapse vectors results in a single color attraction vector.

Color Flow

While color attraction is the result of color similarities or differences in space, color flow is the result of a color changes over time. Rather than calculating the distance of the neuron state to a single synapse input, its temporal derivative is calculated by using input from a neuron and its corresponding memory neuron. This time the sum of scaled synapse vectors results in a flow vector.

Encoding

Both color flow and color attraction vectors are added up and their components are encoded in the flow layer.

Parameters

here are various parameters in each layer controlling the amount and direction of color attraction, color flow, the metrics used for calculating color distances, the neuron neighborhood, etc …

Implementation

All neural computation is performed on the GPU using OpenGL and GLSL shaders. This is the mapping from neural metaphors to OpenGL implementation:

Memory layers → Texture-Buffers
Processing Layers → GLSL shaders
Parameters → GLSL uniforms

Outlook

In our implementation both color flow and attraction are integrated into a single level flow map. While this generates interesting local interactions, there is little organization on a global level. The work on Multilevel Turing Patterns as popularized by Jonathan McCabe shows that it is possible to obtain complex and visually interesting self organizing patterns without any kind of video input.

Our future research will combine several layers of flow maps, each operating on a different level of detail. Additional directions include alternate color spaces and distance metrics.
In the current model input values are mixed and blurred, resulting in a loss of information over time. We have also been experimenting with entropy-conserving models and are planning to further investigate this direction.

This project is based on two recent artworks, “transits” and “IseethereforeIam”

Conceppt: Ursula Damm
Programming: Martin Schneider

documentation

08.09.2009

won competition

  • Uni Cottbus (1999)
  • metro station Wehrhahnlinie Düsseldorf (2002)

competitions

  • Campus der Fachhochschule Regensburg (2004)
  • Landesvertretung Rheinland-Pfalz, Berlin (2003)
  • Vorraum der Uni-Mensa der Uni-Koblenz (2000)
  • Gestaltung des Ortszentrums Wörgl/Österreich (1999/2000)

documentation downloaden (PDF, 1 MB)

Fernfuehler

01.04.2007

Interactive furniture for public places


idea | Installation | concept | stools | game | authors | links


fernfuehler

enliven public spaces bringing form and structure into the consciousness of the general public. As they are connected with each other, “Fernfuehler” can also play, and can influence the behaviour of these other “Fernfuehler” (or of the people sitting on them). The town-planning interest lies in enlivening urban spaces for passers-by and making these spaces able to be changed. Instead of providing seating in public spaces as permanently fixed architecture, mobile groups of seats are provided which communicate with each other, thereby discovering, through experimentation, the optimal arrangement of elements in the space. Planning from the bottom-up is brought to bear here, instead of planning from on high, so involving the user in the process of shaping public space.

top

installation

all possible states of fernfuehlers
sensory system of fernfuehlers

“Fernfuehler” are seating options that can be moved around at will. The seats are modular. They can be brought together to form ensembles, or they can stand alone. By pulling out their backrests they can be transformed into spatial elements, or, with the backrest pushed in, they can just be seats.
“Fernfuehler” detect what other “Fernfuehler” (or the people sitting on them) are doing. And they can react to what the other ones are doing.
They are tough and unpretentious.
They like people’s company as they always move in their direction.
They can hear. When you call them, they come.

Everything that “Fernfuehler” do can be observed in a small computer game. A worm’s eye view displays the area where the “Fernfuehler” are located as a network of nodes.

The birds-eye view of the setting can be made publicly visible for anyone on a hand held computer by passers by who are in the area. The network structure’s nodes, which represent the local arrangement of “Fernfuehler”,
can be manipulated by people playing with the “Fernfuehler” on the handheld computers’ displays or on the projected video screen. In this way people can control the paths that the seats follow in the area where they are located.


top

concept

Is software art a further stage of conceptual art? Works by Dan Graham (“Poem Schema”, 1966 – 1969) or Sol Lewitts Wall drawings, together with his ‚sentences on conceptual Art‘ support this understanding. Tilman Baumgaertel draws a line in his paper ‚EXPERIMENTAL SOFTWARE’ from the instruction of lewitts concepts (which are meant to be machines) to the computers from today. But the software we are writing today is not looking for a crafts man executing our will, but for users who have more degrees of freedom in their behavior. Software art today is more something inbetween the programmer and the user.


The installation „Fernfuehler“ refers with its aesthetics to Sol Lewitts “Serial Project #1″ oder “Serial Project ABCD“. A programmer today still needs formal systems to allow the computer to make comparisons, differentiations, decisions. As the world of the computer is much smaller than our every day life, we have to offer the computer a smaller version of the latter. The pedestrians, visitors and passengers will break up the initially ordered setting of the „Fernfuehler“. The visitors can move the stools around and extend their backrests. The position of the stools react to the neural network and organize themselves in a bottom-up process, according to the presence and usage of the visitors.

A moderate number of “Fernfuehler” occupy the area. “Fernfuehler” are intelligent. They are Items of furniture with rollers and a motor. They can therefore move on their own. As soon as people arrive in the area, they will move towards them, as they have microphones which listen for their voices.


Now people can take their places on the seats, they can form groups or remain alone. Because “Fernfuehler” make first for wherever people are, the arrangement of furniture elements in the area corresponds to the structure of the area, thereby strengthening it. Now you could just find a spot in the area and watch how the seats move around and how other people react to them. Anyone who finds just watching the seats operating automatically too boring, can get out a handheld computer, load the game over a wireless network and use it to activate the “Fernfuehler”. On the screen you see a network structure with dots at each node. Each “Fernfuehler” in the area represents one of the nodes on this network.

The network connects each “Fernfuehler” while at the same time acting as a skin lying over the area. At this point there will be several options for determining the behaviour of the “Fernfuehler” in the area by manipulating the graphical interface. The purpose of the installation is to make public space more attractive, especially to young people. By providing networked seating, they experience the area as a place that changes, one that has moved beyond stable architecture. In addition they can themselves try the role of director, either on the hand held computers or, if they prefer, on the big screen, as they can influence the behaviour of passers-by by re-arranging the positions of the items of furniture. They experience what it is like for computer games to have an effect directly on the surrounding physical space and on the other people there.

top

the stools

move on rollers. When you sit on them, they will be on their frames, which settle down onto the ground on springs. Each seat has two side pieces which can be pulled out and used as backrests or, with both extended, transform the seat into an item that divides physical space.


Each seat is at the same time a node in a virtual network, linking every seat together. The nodes in the network are “neurones”, they learn from the signals which the seats, as it were, receive.
The sounds in the public space, and the use made of the seats for sitting on, are the signals feeding the neuronal network. LEDs inside the seat display the seat’s state of activity within the neuronal network, (with a colour or white light).


Each seat has a controller to which a microphone and a pressure sensor are connected. The pressure sensor can detect whether anyone is sitting on the seat, while the microphone picks up surrounding sounds, filtering human voices. If these sensors detect activity, then the seat “learns” that position as “positive”.

led_quer
Prototype of a stool with LED’s to indicate the inner state of the neuronal network
“fernfühler”, an interactive installation for the public space

the game

A game is available over a wireless LAN, representing the spatial arrangement
of the seats and making it possible to integrate them. In this way it is possible
to use the computer game to instantly intervene not only on the screen, but
also into the immediate surroundings and the situation of other players.

Variations of the spacial order:

Autoren:

Ursula Damm, Matthias Weber (dipl. information science)

Links:

Earth sculptures outdoor

08.07.1982

  • Outdoor 1982