Energy harvesting (wind), lichen, illuminationsSimulation of the lighting
The CONTACT ZONE responds in a multidimensional way to the different situations with the components light, vegetation, and encounter.
A rhythmic illumination in the cycles of the wind is the leitmotiv of our design. Wind-harvesting along the track captures and visualizes the airstream of passing trains. Animated, pulsating light strips and sound installations accompany the urban spaces along the track. Linear light elements are erected in front of the noise barriers and light up in the rhythm of the airflow, which is picked up by small microphones. Light intensity, orientation (to the green? To the people?) and relation to the surrounding urban space and react to different light conditions. For shaded places, as well as under bridges or in tunnels, we propose another, animated form of lighting, which also absorbs the energy of the Fahrtwind, but let it live on in the urban space as a source of light for people and plants animated. For this purpose, we propose compact, small-scale lighting installations that function according to the principle of cellular automata.
In this way, the lamps can carry the energy of the airstream profitably for people and plants into the urban space.
The lighting elements and installations should not only illuminate the urban space for people, but at the same time also enable and promote cryptogamous growth – algae, lichens, mosses – on the noise barriers, accompanying retaining walls, and the luminaires themselves. These bind huge amounts of carbon dioxide and nitrogen from the atmosphere and thus also positively influence the urban climate. The amount of nitrogen fixed by cryptogamic layers and thus supplied to the soil and other living organisms is equal to half of the nitrogen naturally fixed on land, which is particularly important for the development of ecosystems, as nitrogen is often the limiting nutrient component. They are therefore extremely effective in the smallest area of urban space.
Demovideo of the project
The proposal was made by a team Green Noise, Berlin (Ursula Damm), Dr. Klaus Fritze, Rudolph Langner – Station C23 – Architecture and Landscape Architecture, Andreas Hecker (Bauhaus University Weimar), Felix Bonowski.
Collection of lichen present in Düsseldorf, lightingThe Urban Laboratory, planned by Klaus Fritze
Art installation by Ursula Damm and Felix Bonowski, 2021
Curator: Yvonne Volkart
Commissioned for the Flux building by Eawag: Swiss Federal Institute of Aquatic Science and Technology
Technique: Two channel projection; one based on live camera footage and neural network learning rules; simulation based on Perlin noise, Navier Stokes-Solver, reaction diffusion kinetics parameterized with live measurements of Oxygen content, temperature, turbidity.
Felix Bonowski controling the “bachcam” screen, EAWAG Flux Building, ZürichView of the projection – simulation and generative video
Kontinuum is a generative 2 channel projection based on live data of the Chriesbach, a rivulet flowing alongside the Institute of Water Research. The two projections represent a certain mode of “reality” of the Chriesbach and its flux throughout the year. Both translate data of seasonal variations, color patterns and physical principles of the stream into sensual images that associate impressionist and Japanese painting. By bringing the outside into the inside, the water into the Flux building, the object of observation to the site of its investigation, the installation reflects the Kontinuum of the stream and the function of the house in an aesthetic way.
The colored projection collects images of the Chriesbach stream and its inhabitants. Real time images of three cameras are passed through a graphics shader which is based on classical neural network learning rules that “remember” colors in areas of high activity. The resulting video is a collage of combined aspects of the streams’ visual appearance from different times and viewing angles. With their daily changes they serve as a kind of aesthetic weather report.
The black and white projection is a live simulation of a fluid meandering through a rock-strewn valley. Based on an ecosystem of nutrients, primary producers, and grazing microorganisms of the stream, it manifests digitally “how the world would look like, if nature followed these rules”. The formulas that govern the shape of the landscape, the dynamics of flow, and the evolution of life in the simulation are parameterized with values derived from actual real-time measurements of physical water properties. The measurements are performed by a station operated by the research institute just a few meters from where the cameras look onto the stream. Correspondences between measurements and model parameters are chosen so that seasonal changes (in temperature), daily rhythms (in oxygen saturation from photosynthesis) and occasional events (turbidity caused by thunderstorms and construction work) leave their traces in the graphics. Transforming from a valley with a few large boulders to a (virtual) riverbed with many small pebbles, from one emergent biological pattern into another, from a slowly meandering flow into a violent gusher, the simulation reveals itself as a being in permanent flux.
At the right border of each projection, the image logic of the other projection intervenes, so that the color-data of the live stream and the patterns of the black and white simulation intersect: contrasty movements in the colored projection (e.g. reflections of light or swimming leaves) become lines, scratches and holes in the black and white one. They appear as forces, which wipe out organic life and destroy the image. Thus, it becomes clear, that no image and no “reality” stand for themselves, rather they can be questioned, disrupted, or interpreted by manifold approaches.
Video documentation of the installationCameras observing the Chriesbach streamPositioning of the camerasReal time data from Chriesbach for the simulation screenFelix Bonowski installing the cameras
Turnstile, generative Installation von Ursula Damm (Foto Thomas Mayer)
On the front wall of the Schadowstrasse underground station, an LED wall displays a generative video. In front of the wall, a light shaft extends to the surface of the plaza where a video camera is set up. The camera continuously films passing pedestrians on the plaza and streams the feed to a specially developed generative software application (coded by Felix Bonowski) which derives proposed geometries for structures based on the movement patterns of the pedestrians. These interpretations of the real-time video generate new geometries for the location and propose axes and parcels.
Turnstile, Ursula Damm, Schadowstrasse Düsseldorf 2016 (Foto by Thomas Mayer)
Two elevators, to the left and right of the large video image, lead from the plaza to the rail platform.
Pattern drawings on aerial photos of Düsseldorf, Schadowstrasse
Turnstile (Drehkreuz) from resoutionable on Vimeo. On the platform, the geometric structures can be heard as a sound interpretation (by Yunchul Kim). At the centre of the artistic intervention is the video image and its artistic concept. The concept is reflected in the design of the entrance areas. Plates are inserted in the blue glass of the underground station at 21 locations, which display geometries over districts of Düsseldorf.
In the east concourse is the aerial image of the city of Düsseldorf that was analysed according to the geometric concept.
Geometric pattern generation on the spacial structure of Duesseldorf
As excerpts from this aerial picture, 16 locations in Düsseldorf were interpreted at the level of a local aerial image. These urban areas were described with regular polygons as energy centres which fitted themselves together through the development of the city architecture (see the text on the concept of the generated patterns).
Schadowstrasse, pedestrian aerea, location of camera; Photo Achim Kukulies
The fine structure of the patterns juxtaposes both the sensibility of nature and the human, formative gestures against the massive edifice, calling to mind a mode of formation that creates sweeping interconnections through the symbiotic organisation of a multitude of individual elements. In doing so, this formative process completes the social principle through which individuals experience their effect on the whole.
The pattern drawings are generated in slow steps: First a line drawing is created over the image of the city. As this progresses, important motion axes of traffic and pedestrians are emphasised. The areas these axes enclose become polygons. At this point, the angles of the lines and axes are examined in the search for whole-number fractions of regular polygons.
The smallest polygon integrating all of the symmetries at the location (for instance, five-angled and four-angled fragments would be assembled into a 20-sided polygon) is then used to describe an intersection.
A subsequent step is the search for connections (network) between large neighbouring polygons.
Work with the aerial images revealed that the city centre has very small polygons, while outer areas have a significantly more expansive structure. Thus, density is indicated by the presence of small polygons and complex symmetries. Often, the transition from non-rectangles to rectangles can indicate historical breaks in the urban landscape. In this way, the interpretations represent a study of the settlement and planning history of the city.
Pattern drawings over aerial views of Düsseldorf
Drawing Nr. 24, Düsseldorf Flehe
The sound installation
The generative video installation interprets traces of movement created by geometric “agents.” The activity of these agents is translated into sounds which track the visual artefacts. As such, the sounds form the noise that the virtual artefacts generate in their world, and thus represent and extended artistic “level of reality” of the installation.
PROGRAM
Select a location (origin)
Determine the movement axes of people and traffic
Look to see if these axes are at angles to one another, which when mirrored and rotated can form a polygon, the sides of which all extend outward equally
Draw this polygon to approximate the natural geometry of the location
Look to see if, starting from this, the intrinsic geometries of the location can form a surface structure, (tessellation) that periodically repeats the original geometries
Determine whether and how, in the aerial image of the location, the areas fit together in the revealed geometry of the place
Enhance existing structures by developing their geometries
Connect existing structures into the logic of the original geometry
Konzept: Ursula Damm Programmierung: Felix Bonowski Sound: Yunchul Kim
The Urban Development Kit provides tools to ameliorate the atmosphere in contemporary cities. The Urban Development Kit is a collection of tools. Over the time the website aims to become a resource of ideas, concepts and tools for a citizen-driven urban design.
Visualization for the workshop in Osaka
One of our kits supports watchful citizens and plants to compete with pavement, concrete and asphalt. It helps plants to interact with modern cities, to prevail against soil sealing. A website and an interactive map enables the people to collect photos of “asphalt flowers” in Helsinki and other cities and to monitor the progress of the “cultivation”. With respect to urban environmental research, the urban development kit is a statement about the importance to counteract the sealing of the surfaces in the city. Accordingly, in the exhibition can be seen design for urban surfaces which are based on the geometry the plants themselves.
Patterns on polygonium avicularepattern drawing on polygonium aviculare
The work has been developed for the Art&HENVI project, organized by the finish Bioart Society.
counteracting soil sealing
In 2014, a new version of the urban development kit was presented at a creative cloud workshop, organized from ars electronica (see the fotos from the workshop)
Chromatographic Orchestra is an artistic installation which allows a visitor to direct a software framework with an EEG device. In an exhibition environment with semi-transparent video screens a visitor is sitting in an armchair and learns to navigate unconsciously – with his/her brain waves the parameter space of our software – Neurovision.
Neurovision interacts with live video footage of the location of the exhibition and its surroundings. By navigating with his/her own brain waves the visitor can define and navigate the degree of abstraction of a generative (machine learning) algorithm, performed on the footage of different, nearby video cameras.
Vizualisation for ideal set-upideal set-up for the installationOur Operator Lisa sitting in front of the screen with a EEG device
The installation refers back to painting techniques in the late 19th and early 20th century, when painting became more an analysis of the perception of a setting then a mere representation of the latter. Impressionism and Cubism were fragmenting the items of observation while the way of representation was given by the nature of the human sensory system.
The installation “chromatographic orchestra” does not apply arbitrary algorithms to the live footage: we developed a software – the Neurovision framework – which mimics the visual system of the human brain. Thus we question whether our algorithms meet the well being of the spectator by anticipating processing steps of our brain.
Artistic Motivation
How much complexity can our senses endure, or rather how could we make endurable what we see and hear? Many communication tools have been developed, to adjust human capabilities to the requirements of the ever more complex city.
Our installation poses the opposite question: How can information emerging from the city be adjusted to the capabilities of the human brain, so processing them is a pleasure to the eye and the mind?
At the core of our installation is the NeuroVision Sandbox, a custom made framework for generative video processing in the browser based on WebGL shaders.
Inside this Sandbox we developed several sketches, culminating in the “Chromatographic Neural Network”, where both optical flow and color information of the scene are processed, inspired by information processing in the human visual system.
We critically assess the effect of our installation on the human sensory system:
Does it enhance our perception of the city in a meaningful way?
Can it and if so – how will it affect the semantic level of visual experience?
Will it create a symbiotic feedback loop with the visitor’s personal way to interpret a scene?
Will it enable alternate states of consciousness? Could it even allow visitors to experience the site in a sub-conscious state of “computer augmented clairvoyance”
Installation
In a location close to the site a single visitor directs a video-presentation on a large screen with a setup we like to call “the Neural Chromatographic Orchestra” (NCO). Our installation uses an EEG-Device (Emotiv NeuroHeadset) that lets visitors interact with a custom neural network. The setup allows visitors to navigate through various levels of abstraction by altering the parameters of the artificial neural net.
With the NCO device, a visitor can select and explore real-time views provided by three cameras – located in public space – with different perspectives on the passer-byes (birds-eye view and close-ups)
The installation is based on the NeuroVision Sandbox used in the development of “transits”. Other than transits, chromatographic ballads uses multi-channel real-time video-input and enables a visitor to interact with irectly via biofeedback with the neural network.
The Neural Chromatographic Orchestra investigates how human perception reacts to the multifaceted visual impressions of public space via an artistic setting. Using an EEG-Device visitors can interact with a self-organizing neural network and explore real-time views of an adjacent hall from several perspectives and at various levels of abstraction.
Biological Motivation
The Chromatographic Neural Network is a GPU-based video processing tool. It was inspired by parallel information processing in the visual system of the human brain. Visual information processing inside the brain is a complex process involving various processing stages.The visual pathway includes the retina, the Lateral Geniculate Nucleus (LGN) and the visual cortex
Low-level visual processing is already active at the various layers of the retina. The Interconnection of neurons between retina layers, and the ability to retain information using storage or delayed feedback, allows for filtering the visual image in the space and time domain.
Both image filters and motion detection can easily be achieved by accumulating input from neurons in a local neighborhood, in a massively parallel way.
Our Chromatographic Neural Network uses this approach to cluster colors and to compute the visual flow (or retina flow ) from a video source. The resulting attraction-vectors and flow-vectors are used to transform the memory retained in the memory layer.
The visual output of the system directly corresponds to the state of the output layer of the neural network. The neural layers of the Chromatographic Neural Network, are connected to form a feedback loop. This giving rise to a kind of homeostatic-system that is structurally coupled to the visual input but develops its own dynamics over time.
The set-up
A visitor enters the site – a highly frequented passage, a spacious hall or a public place. Two videocameras, mounted on a tripod, can be moved around at will.
Another camera observes the passer-byes – their transits and gatherings – from an elevated location. The video footage from this site is streamed into a neighboring room – the orchestra chamber of the Neural Chromatographic Orchestra.
Here one can see – in front of a a large video wall a monitor displaying the videos from the adjacent room and the “orchestra pit” – an armchair equipped with a touch device and a neuro-headset. The video wall, showing abstract interpretations of the site itsself, should ideally be visible both from the orchestra pit and from the large hall.
The Orchestra Chamber
Inside the chamber the visitor is seated in a comfortable armchair and an assistant helps her put on and adjust the neuro-headset.
The orchestra chamber should be isolated from the public area as much as possible. A sense of deprivation from outside stimuli allows the visitor to gain control over her own perception and achieve a state of mind similar to meditation or clairvoyance.
The Orchestral Performance
Training Cognitive Control
A performance with the Neural Chromatographic Orchestra starts with a training of up to six mental actions, corresponding to the “push/pull”, “left/right“ and “up/down” mental motions provided by the Emotiv Cognitiv suite. The training typically lasts 10 to 30 minutes.
Playing the Sandbox
After successful training the visitor is asked to sit in front of the NeuroVision Sandbox:
The visitor in the orchestra chamber has three modes of conducting the neural network
either the birds-eye view or one of the cameras that take a pedestrian’s perspective
A graphical user interface lets her switch between different neural networks and control their parameters
A menu lets her choose any of the three cameras as a video source:
the NeuroHeadset allows to navigate the parameter space of the selected neural network
Conducting the Orchestra
Once the visitor feels comfortable conducting the NCO on the small screen, she can perform on the large screen, that is also visible from the outside.
On the public screen sliders are not shown, but the conductor may still use a tablet device to access the graphical user interface.
The current position in parameter spaces is represented by a 3d-cursor or wire-frame box, which is very helpful for making the transition from voluntary conduction moves, to a style of conducting that is more directly informed by immersion and interaction with the output of the Chromatographic Neural Network.
The Chromatographic Neural Network
The flow of information is arranged into several processing layers. To realize memory, each processing layer is in turn implemented as stack of one or more memory layers.This allows us to access the state of a neuron at a previous point in time.
Example
The video layer is made up of two layers, so the system can access the state of any input neuron at the current point in time, and its state in the previous cycle.
Processing Layers
The Video layer
The Video layer contains the input neurons. Each neuron corresponds to a pixel of the video source. The Video layer provides the input for the Flow layer.
The Ghost Layer
The Ghost layer represents a haunting image from the past. It implements the long term memory, that interferes and interacts with the current visual input. It does not change over time, and is provided as additional input to the Flow layer
The Flow layer
The Flow layer accumulates the input from the Video layer and the Ghost layer. Each Neuron aggregates input from its neighborhood in the Video Layer at times (t) and (t-1). The computed 2d vector is directly encoded into the the state of the neuron, creating a flow map.
The Blur layers
The Blur layers are used to blur the flow map. While the computation of visual flow is restricted to a very small neighborhood, the blur layer is needed to spread the flow information to a larger region, since flow can only be detected on the edge of motion.
For efficiency reasons the blur function is split into two layers, performing a vertical and a horizontal blur respectively.
Neuron Processing
The state of each neuron corresponds to an RGB color triplet. Every neuron of the Flow layer gets input from corresponding neurons inside a local neighborhood of the input layers. Each of those input samples corresponds to a single synapse. The vector from the center of the neuron towards the input neuron is referred to as the synapse vector.
Color Attraction
To achieve some kind of color dynamics, colors that are close in color space are supposed to attract each other.
The distance between synapse input and the neuron state in RGB color-space, serves as a weight, which is used to scale the synapse vector. The sum of scaled synapse vectors results in a single color attraction vector.
Color Flow
While color attraction is the result of color similarities or differences in space, color flow is the result of a color changes over time. Rather than calculating the distance of the neuron state to a single synapse input, its temporal derivative is calculated by using input from a neuron and its corresponding memory neuron. This time the sum of scaled synapse vectors results in a flow vector.
Encoding
Both color flow and color attraction vectors are added up and their components are encoded in the flow layer.
Parameters
here are various parameters in each layer controlling the amount and direction of color attraction, color flow, the metrics used for calculating color distances, the neuron neighborhood, etc …
Implementation
All neural computation is performed on the GPU using OpenGL and GLSL shaders. This is the mapping from neural metaphors to OpenGL implementation:
In our implementation both color flow and attraction are integrated into a single level flow map. While this generates interesting local interactions, there is little organization on a global level. The work on Multilevel Turing Patterns as popularized by Jonathan McCabe shows that it is possible to obtain complex and visually interesting self organizing patterns without any kind of video input.
Our future research will combine several layers of flow maps, each operating on a different level of detail. Additional directions include alternate color spaces and distance metrics. In the current model input values are mixed and blurred, resulting in a loss of information over time. We have also been experimenting with entropy-conserving models and are planning to further investigate this direction.
enliven public spaces bringing form and structure into the consciousness of the general public. As they are connected with each other, “Fernfuehler” can also play, and can influence the behaviour of these other “Fernfuehler” (or of the people sitting on them). The town-planning interest lies in enlivening urban spaces for passers-by and making these spaces able to be changed. Instead of providing seating in public spaces as permanently fixed architecture, mobile groups of seats are provided which communicate with each other, thereby discovering, through experimentation, the optimal arrangement of elements in the space. Planning from the bottom-up is brought to bear here, instead of planning from on high, so involving the user in the process of shaping public space.
all possible states of fernfuehlerssensory system of fernfuehlers
“Fernfuehler” are seating options that can be moved around at will. The seats are modular. They can be brought together to form ensembles, or they can stand alone. By pulling out their backrests they can be transformed into spatial elements, or, with the backrest pushed in, they can just be seats. “Fernfuehler” detect what other “Fernfuehler” (or the people sitting on them) are doing. And they can react to what the other ones are doing. They are tough and unpretentious. They like people’s company as they always move in their direction. They can hear. When you call them, they come.
Everything that “Fernfuehler” do can be observed in a small computer game. A worm’s eye view displays the area where the “Fernfuehler” are located as a network of nodes.
The birds-eye view of the setting can be made publicly visible for anyone on a hand held computer by passers by who are in the area. The network structure’s nodes, which represent the local arrangement of “Fernfuehler”, can be manipulated by people playing with the “Fernfuehler” on the handheld computers’ displays or on the projected video screen. In this way people can control the paths that the seats follow in the area where they are located.
Is software art a further stage of conceptual art? Works by Dan Graham (“Poem Schema”, 1966 – 1969) or Sol Lewitts Wall drawings, together with his ‚sentences on conceptual Art‘ support this understanding. Tilman Baumgaertel draws a line in his paper ‚EXPERIMENTAL SOFTWARE’ from the instruction of lewitts concepts (which are meant to be machines) to the computers from today. But the software we are writing today is not looking for a crafts man executing our will, but for users who have more degrees of freedom in their behavior. Software art today is more something inbetween the programmer and the user.
The installation „Fernfuehler“ refers with its aesthetics to Sol Lewitts “Serial Project #1″ oder “Serial Project ABCD“. A programmer today still needs formal systems to allow the computer to make comparisons, differentiations, decisions. As the world of the computer is much smaller than our every day life, we have to offer the computer a smaller version of the latter. The pedestrians, visitors and passengers will break up the initially ordered setting of the „Fernfuehler“. The visitors can move the stools around and extend their backrests. The position of the stools react to the neural network and organize themselves in a bottom-up process, according to the presence and usage of the visitors.
A moderate number of “Fernfuehler” occupy the area. “Fernfuehler” are intelligent. They are Items of furniture with rollers and a motor. They can therefore move on their own. As soon as people arrive in the area, they will move towards them, as they have microphones which listen for their voices.
Now people can take their places on the seats, they can form groups or remain alone. Because “Fernfuehler” make first for wherever people are, the arrangement of furniture elements in the area corresponds to the structure of the area, thereby strengthening it. Now you could just find a spot in the area and watch how the seats move around and how other people react to them. Anyone who finds just watching the seats operating automatically too boring, can get out a handheld computer, load the game over a wireless network and use it to activate the “Fernfuehler”. On the screen you see a network structure with dots at each node. Each “Fernfuehler” in the area represents one of the nodes on this network.
The network connects each “Fernfuehler” while at the same time acting as a skin lying over the area. At this point there will be several options for determining the behaviour of the “Fernfuehler” in the area by manipulating the graphical interface. The purpose of the installation is to make public space more attractive, especially to young people. By providing networked seating, they experience the area as a place that changes, one that has moved beyond stable architecture. In addition they can themselves try the role of director, either on the hand held computers or, if they prefer, on the big screen, as they can influence the behaviour of passers-by by re-arranging the positions of the items of furniture. They experience what it is like for computer games to have an effect directly on the surrounding physical space and on the other people there.
move on rollers. When you sit on them, they will be on their frames, which settle down onto the ground on springs. Each seat has two side pieces which can be pulled out and used as backrests or, with both extended, transform the seat into an item that divides physical space.
Each seat is at the same time a node in a virtual network, linking every seat together. The nodes in the network are “neurones”, they learn from the signals which the seats, as it were, receive. The sounds in the public space, and the use made of the seats for sitting on, are the signals feeding the neuronal network. LEDs inside the seat display the seat’s state of activity within the neuronal network, (with a colour or white light).
Each seat has a controller to which a microphone and a pressure sensor are connected. The pressure sensor can detect whether anyone is sitting on the seat, while the microphone picks up surrounding sounds, filtering human voices. If these sensors detect activity, then the seat “learns” that position as “positive”.
Prototype of a stool with LED’s to indicate the inner state of the neuronal network“fernfühler”, an interactive installation for the public space
the game
A game is available over a wireless LAN, representing the spatial arrangement of the seats and making it possible to integrate them. In this way it is possible to use the computer game to instantly intervene not only on the screen, but also into the immediate surroundings and the situation of other players.
Variations of the spacial order:
Autoren:
Ursula Damm, Matthias Weber (dipl. information science)
