IN VIVO PHASE I:
May 2019 - Jun 2020

STEAM Imaging II Artist in Residency
Fraunhofer MEVIS, Ars Electronica & NTU ADM

IN VIVO

In Vivo is an interactive art science installation that asks the question, what does it mean to be alive? The project is utilising research collected during a six weeks international “STEAM Imaging”¹ residency co-organised and hosted by Institute for Digital Medicine Fraunhofer MEVIS, the International Fraunhofer Talent School Bremen and Ars Electronica. The project is also done in connection with Nanyang Technological University Singapore.² The residency was awarded in June 2019 though an open art call and took place from October - Novemebr 2019. The installation integrates sensors and microcomputers and uses the possibilities of the virtual and augmented reality to talk about the relationship between man, machine and nature. The project attempts to shed light on the transitional areas between life and death through the artistic means of photography and video installation. More specifically, the installation tracks the rhythms of the action potential in a bonsai plant and directly affect thousands of MR (magnetic resonance) images captured using an MRI machine. Using projection mapping and soundscapes generated and translated from the MR images, the installation “ gives voice” to the bonsai and give the audience an insight into a living being.

The project is a multidisciplinary work that involves, scientists, mathematicians, psychologists, physicists, engineers, programmers and of course artists. The collaboration between the sciences and the arts is a growing field. The project attempts to continue the open dialogues that happen because of these interactions. The installation will be presented next year at the Ars Electronica Festival 2020, one of the world’s most important events for art, technology, and society.

OBJECTIVES

The project’s broad scope is to connect the science and digital medicine technology with the creative arts. Our completed STEAM residency abroad and working with experts³ mathematicians, computer scientists, or physicists in the filed of digital medicine, inspired us to tap into unique capabilities of the art-sci way of knowing.

Working with scientists and getting access to Fraunhofer MEVIS R&D provoked us to develop one of a kind interdisciplinary art, we would have not otherwise considered. The project’s objective is to highligh the unique explorative creativity resulting from combining separate disciplines of science with art. At the same time the installation will be space to contemplate the bigger philosophical question such as what does it mean to be alive?

Using a bonsai as a metaphor for a coma patient, who cannot move and cannot speak, the project makes a comparison between two beings that are alive and if signs of life assume that consciousness exists. Our projects seek to question current ideas about how we can communicate with people who cannot communicate by movement or speech. Can we use existing medical technology to help people who are in comas, to see if there is any mental activity? What are the moral justifications related to coma patients and their welfare if signs of life are found?

The project is supported by medical visualisation technology, in this case an MRI (magnetic resonance imaging) machine and presents it in an interactive and accessible way for people to visualise the process of the image capturing process. By using MRI, can we find out or can we see what is within the living? Working with
scientists, mathematicians and physicists from the Institutute for Digital Medicine, we have collected thousands of images of the bonsai in an MRI machine. The artists’ request presented a novel challenge for the researchers at MEVIS, who usually work on methods to improve MR imaging of human patients. Imaging plants were new to them.

Ars Electronica Festival is one of the largest Media Arts Festival in the world and has a 40 year history that prides itself on the philosophy of ART, TECHNOLOGY and SOCIETY. By presenting at the festival, we hope to connect and contribute to this conversation about new possibilities in art and technology as well as how the project contributes to society and the implications it has in the real world.

The project is a multidisciplinary work that involves, scientists, mathematicians, psychologists, physicists, engineers, programmers and of course artists. The collaboration between the sciences and the arts is a growing field, and the project attempts to continue the open dialogues that happen because of these interactions. The festival draws people from all disciplines and facilitates dialogues between different groups of people throug talks and workshops and being part of the festival would mean forging new connections and potentially future projects.

ART

Projection mapping on Bonsai using its own MRI Slices

3D printed branches and leaves generated from MRI scans attached onto Bonsai

Rapid prototyping process: 3D printing MRI scans at 0.15x scale

Aluminium wires embedded in bonsai during its nurturing phase, a mini surgery was required to remove them

SCIENCE

Understanding and sending Bonsai into MRI Machine with Physcist Matthias Guenther

MRI Scan of Bonsai Tree with a rogue frequency band picked up by Aluminium wires embeded into Bonsai when nurturing its shape

Designing a custom Arduino circuit that is able to track Action Potential of Bonsai

Monitoring Action Potential of Bonsai through live data streamed to IoT Database Adafruit IO

WORKSHOP

Workshop held at Fraunhofer MEVIS in Bremen, Germany

Workshop held at Ars Electronica Center in Linz, Austria

PRESS

1. Announcement of STEAM Imaging II Residency Results – http://www.adm.ntu.edu.sg/NewsnEvents/Pages/News-Detail.aspx?news=cc7336cd-bb48-4e56-97ac-f333450863b6 , https://www.mevis.fraunhofer.de/en/press-and-scicom/institute-news/2019/steam-imaging-ii--artists-in-residence-are-jake-tan-and-ernest-wu.html
2. Residency Kick Off at Fraunhofer MEVIS in Bremen, Germany – https://twitter.com/FraunhoferMEVIS/status/1179351588730736640
3. Press Release – https://www.mevis.fraunhofer.de/en/press-and-scicom/press-release/2019/patient-bonsai.html
4. Ars Electronica Blog Interview – https://ars.electronica.art/aeblog/en/2019/11/04/patient-bonsai/

TEAM

Jake Tan

Artist

Ernest Wu

Artist

Ina Conradi

Project Coordinator Singapore

Bianka Hofmann

Project Development, Mentor & Mediator between Artist and Scientist

Veronika Liebl

Ars Electronica Project Coordinator

Veronika Krenn

Ars Electronica Project Coordinator

Sabrina Haase

STEAM Workshop Leader

Hanne Ballhausen

STEAM Workshop Supporter

Alexander Kohn

Technical Support, MeVisLab Trainer

David Black

Artistic Support (Sound Designer)

Matthias Guenther

MRI Expert, Physcist

¹STEAM stands for the connection of science, technology, and mathematics with the world of art. The “STEAM Imaging” program enables artist residencies focused on connecting science with the approaches of digital art. Integral components
include several workshops that aim at inspiring youth about the unusual alliance between art and science.
²Patient Bonsai Fraunhofer MEVIS 4.11.2019 https://www.mevis.fraunhofer.de/en/press-and-scicom/press-release/2019/patient-bonsai.html
³Physicist Professor Matthias Guenther, Mathematician Sabrina Hasse

About CV

Jake Tan
New Media Artist/
Creative Technologist

IN VIVO PHASE I:
May 2019 - Jun 2020

IN VIVO

OBJECTIVES

ART

SCIENCE

WORKSHOP

PRESS

TEAM

Index Next︎

About CV

Jake Tan New Media Artist/Creative Technologist

IN VIVO PHASE I: May 2019 - Jun 2020

IN VIVO

OBJECTIVES

ART

SCIENCE

WORKSHOP

PRESS

TEAM

Index Next︎

Jake Tan
New Media Artist/
Creative Technologist

IN VIVO PHASE I:
May 2019 - Jun 2020