Next lines I'm going to present the new project for artistic research in the field of the technological singularity and the relative transhuman reflections. This project born from the idea by Francesco Grillo, visual artist, who proposed to me to create a multimedia sensitive environment developing one of his oil paintings theme: the shell.Thus, after around two months of specific research about neural networks and cross connecting relationships of signification among parts, I decided to develop an interaction system based on two principle:
- sound amplification of the human body
- brain-computer interface for environmental reaction to human emotion
The following lines describes in a summary, and then in details, the planning about the next months research and practice in physical computing and BCI for the creation of Selfscape, the new interactive installation project by Visualpaco and Francesco Grillo.
Selfscape
summary
Selfscape is the “mind landscape” real time generated by audience.
A short dark passage divides the guest from the “mind land”...small particle symbolic lights and their sounds will guide the “crosser” in a fluxus along the corridor, till she faces a translucent
door, osmotic mambrane between this world
and her Self world she’s going to explore...
The presence of the guest in front of the door creates an evanescent “mirror” image of the guest herself, shown from behind...she’s invited to enter.
A chair is enlightened at the center of the dark room. On it two tools, the last technologic
exoskeleton to wear before to allow the experience of the world of ideas, neat thoughts, emotions, Selfscapes.
The guest sits on the chair, the light dims.
The rythm of the blood flowing inside her body begins to fill the room with a progressive composing, real-time generated, sound environment.
Mind reactions to this event, captured by the wearing tools, will produce the generation of moving images of landscapes and skylines and seas in front of the guest. After a while she will choose, with her mind energies, the shape of the nature's element that better represents her Self.
Selfscape is the attempt to investigate the transhumanism and technological singularity themes, among the others, from the profound and sensitive point of view of art and aesthetic sensitive perception. Is the cross-connection between the ancestral and archetypal human media
and the future technologies about neural networks networking.
Fig. F. Grillo, sketch of the space for Selfscape
Selfscape
Concept and poetics
Selfscape is the metaphoric unknown landscape of our Selves. It is the virtual, crossmedia, environment, where our emotional and mental status generates the space surronunding, interacting with it.
It is a room, a real room with a real chair standing enlightened at its center. On the chair lie
two technological instruments, strictly linked to life primary organs, brain and heart. These sophisticated instruments, stolen from the proper fields of medical check up, become the
“signals” to activate mind engines (compare “Consciousness” in “The society of mind”, Marvin Minsky). To wear them will become a sort of ritual as the ones the sorceres of older times used to cast their spells. Wearing these technologies the guest will be able then to transform the environment surrounding: her own “life engines” will start to exploit physical computing
technologies to generate a poetic space as contamination of visual art, oil paintings, video art,
experimental live music. A space where to “enter” own body and mind experiencing it as emotional aesthetical perception. A reflective space where to think about our “shells”, that paradoxical image of a central Self inside the self. Our mental processes often seem to flow as a “stream of consciousness”, a collection of thoughts flowing controlled by the single Self. Selfscapes invites to look at the Self, at the shell, as what avoid us to explore the hidden inside. To think the Self as a central engine for single lives is a confortable way of representing our thinking in practical life: we feel confortable thinking ourselves as personal identities,
individual identities. We feel confortable inside the shell of our Self, the same we are not able to explore because so much of our minds do is hidden from the parts of us that are involved in verbal consciousness. Our remote control. Our trait. Our shape. Or maybe mind is that “cloud”
of networks of relationships presented by some scientist that will bring to posthuman predictions about “singularity”, about the point of transiction for human kind as we know to become pure thought flowing through a neural network after the process of “mind uploading”, as in the predictions about next future by R. Kurzweil.
Thus, the “mainframe” is represented by the shell, the “archetypical” natural element.
A shell is an exoskeleton, a protective outer layer created by a sea creature that usually lacks of backbones, an invertebrate.
Shells have been admired, studied and used by huma
ns for many different purposes throughout
history and pre-history. The most ancient shells belongs to cephalopods, widely regarded as the most intelligent of the invertebrates, the same sea creatures fvourite by studies in neurophysiology. The shells of the most ancient cephalopods are fossils belonging to ancient creatures of species survived till nowadays thanks to a complex nervous system and to their social cohabitation.
Taking inspiration from the “shell theme” (fig. 1) in various paintings by Francesco Grillo (francescogrillo.com) the project Selfscape wants to show the shell as our minds protective outer layer, the layer that had been built by the istitutions of law, religion, philosophy, that adopted specific, authority-schematized, to answers to
Fig. 1 Francesco Grillo, Proiezione 1

“circular questions”. Selfscape wants to open a cleft in emotional human reasoning and thinking about the yet unknown “complex” we call Mind, that sort of “green flash” enarrated by Jules Verne.
Yet, the shell, referred to an operating system in informatics computing, is a program that allows the user to communicate with the system itself. To better understand, shells provide a graphical user interface, the GUI we use each time we sit or stand in front of a computer machine. Even in the world of the machine code the shell represents the last border dividing the “mind” of the machine from its peripherical human-recognizable interfaces.
Selfscape
Space and experience
Selfscape is an installation composed by two spaces: one corridor, one room. It is thought for a traveler at time.
A black veil at the beginning of the corridor provides access to the first empty, dark space. Once inside a series of particles of lights in form of letters and codes, “sparkling” amniotic sounds, begin to appear on our body and then move into space as a fluxus that wants to direct us to the next space. Following the light “arrow” we face a sort of membrane on which will appear the body of ourself seen from behind, in a sort of “back-mirror”. We are invited to enter the new space. It is a dark space as well, but in the middle of the room a perpendicular light is
enlightening a chair. Everything around is dark. On the chair two objects. We already know how to wear them because of their familiar shapes. When we sit on the chair the light dims. Everything is dark now. We begin to hear a sound, and then a rythm. It’s our heart pulsing. The rythm changes in a melodic stream. A far landscape begins to appear in front of us. It’s coming through us and with it a sky. At the same time the sensor we are wearing on the forehead it’s reading our brainwaves and translate them in visual effect and color chainging.
A shell will appear in the middle of the landscape. Our brain waves will decide the shape of it basing on our real time emotional psychology. We are looking at ourSelf shape...
Selfscape
Techniques and technologies (the interactive system)
There are several technologies and techniques involved in designing the interactive system for Selfscape.
The first interaction the audience will face is the particles system of light-letters and codes appearing on the body of the audience and moving into the space to direct the guest towards the mirror-door. Audio effects real time modifyed by the position in the corridor To arrange
that is needed to face with computer vision technology and to build one or more specific softwares using APIs such as MaxMSP Jitter, Processing, Quartz Composer. The source code of the software builded for the interactions in the next room (the source code of the central engine of Selfscape) will be the collection of light-letters seen by the guest in the corridor. This first interactive system exploits: infrared video camera; video projector; computer; sound speakers.
The second interaction is represented by the “mirror-door.” This kind of membrane that divides the guest by the other side is a passage. It will be built using transparent PVC films designed as a traversable veil. Standing in front of this veil (in a range-distance of about 50 cm.) the guest will see appearing her own video image on the door itself. A camera is shooting her from behind and a video projector is beaming from inside the room of Selfscape. Passing the door to enter the room the mirror image will disappear. To arrange such an interaction it’s needed to build a specific software using platfoms for physical computing such as Arduino, and APIs such as MaxMSP Jitter. This interactive system exploits: web cam; infrared distance sensor; computer; video projector; IC for physical computing (Arduino).
The third interaction occurs in the room, the main space of Selfscape. The passage through the mirror-door activates the diffusion of a spot light enlightening the chair in the middle of the room. This is allowed thanks to the infrared sensors installed at the entrance of the room
which data will be transmitted to the computer that will activate the next event, as in a domino inductive effect managed by the audience. The light dims to complete dark when the audience sits on the chair. In this case a small pressure sensor will be hidden in the middle of the sitting table of the wood chair. Physical computing and a small software are needed to control this interaction. Instruments: infrared passive sensor; spot light; computer; Arduino IC.
The mainframe of Selfscape is represented by the interactive tools lying on the chair that the
guest is invited to wear and from their communication with the computer.
The first one has familiar ergonomics, identical to wireless headphones to listen to music. It is a brain-computer interface technology product. Neurosky Mindset (fig. 2), after several researches about EEG, non-invasive, medical instruments, shows the features that are needed to obtain the goals for Selfscape. The exploitation of this technology is grounded in medical research. Neurosciences are developing complex systems to understand and interpret the energy emitted by the brain. Brainwave patterns of varying frequencies combine to form electrical signals detectable on the scalp. The technology exploited by Neurosky ThinkGear senses these analog brainwaves and processes them into digital signals that are translated in specific data by a chip. Raw brain signals are amplified and processed by algorithms—delivering concise input to the device with which the user is interfacing. Algorithms come from both
NeuroSky as well as research institutions and
Fig.2 Neurosky Mindset
universities, and are grounded in decades of clinical research. They currently include the mind status of “attention”, “meditation,” and physical eyeblinks. Many new algorithms are in development by neuroscientists and programmers to improve the emotional EEG (Electroencephalography). Receiving wireless data from brainwaves and interpreting the frequencies on the right range of mind status, will be possible with Application Programming Interfaces such as MaxMSP Jitter, Processing, Quartz Composer, to activate video generative effects and the video graphical reconstruction of the landscapes and of the shells (for more details about brain computer interfaces compare wikipedia).
The second interactive tool is represented by a non-invasive intstrument, handmade with electronics and specific-use builded, that exploits the technology of sphygmomanometers for the measurement and auscultatory of human heart’s palpitations. Several types of mediacal instruments are available in commerce that can be “hacked” and exploited for the goals of Selfscapes. This heart beat amplifyer will represent the generative engine for the sound environment that will be created in real time by the blood pressure itself. It will be exploited as protocol music signal for a digital software synthesizer that will fill the space with the audience’s body rythmics and sounds.
The signals coming from such instruments are processed by the computer. Specific softwares will be built and tested for the specific scopes. It’s needed the exploitation of physical computing for the heart beat data transferring. Either the instruments communicates with the computer via bluetooth, sending radio frequencies to the receiver inside the computer. Here platforms like MaxMSP Jitter, Processing, VVVV, Quatz Composer, will be tested for the best commnunicating protocol with the wireless tools. Ableton Live, appliction interface for live generative music, will be interfaced with the physical computing system for the heart beat.
This interactive system needs the development of research on the following technologies, brain-computer interface, computer graphics, body sound amplifying.
It exploits: video projector, BCI, handmade electronic sphygmomanometer, pressure sensor, sound speakers.
About singularity and direct brain-computer interfaces:
