IN VIVO PHASE I:
STEAM Imaging II Artist in Residency
May 2019 - Jun 2020
Fraunhofer MEVIS, Ars Electronica & NTU ADM
In Vivo is an interactive art science installation that asks the question, what does it mean to be alive? The project is utilising research collected during a six weeks international “STEAM Imaging”1 residency co-organised and hosted by Institute for Digital Medicine Fraunhofer MEVIS, the International Fraunhofer Talent School Bremen and Ars Electronica. The project is also done in connection with Nanyang Technological University Singapore.2 The residency was awarded in June 2019 though an open art call and took place from October - Novemebr 2019. The installation integrates sensors and microcomputers and uses the possibilities of the virtual and augmented reality to talk about the relationship between man, machine and nature. The project attempts to shed light on the transitional areas between life and death through the artistic means of photography and video installation. More specifically, the installation tracks the rhythms of the action potential in a bonsai plant and directly affect thousands of MR (magnetic resonance) images captured using an MRI machine. Using projection mapping and soundscapes generated and translated from the MR images, the installation “ gives voice” to the bonsai and give the audience an insight into a living being.
The project is a multidisciplinary work that involves, scientists, mathematicians, psychologists, physicists, engineers, programmers and of course artists. The collaboration between the sciences and the arts is a growing field. The project attempts to continue the open dialogues that happen because of these interactions. The installation will be presented next year at the Ars Electronica Festival 2020, one of the world’s most important events for art, technology, and society.
OBJECTIVESThe project’s broad scope is to connect the science and digital medicine technology with the creative arts. Our completed STEAM residency abroad and working with experts3 mathematicians, computer scientists, or physicists in the filed of digital medicine, inspired us to tap into unique capabilities of the art-sci way of knowing.
Working with scientists and getting access to Fraunhofer MEVIS R&D provoked us to develop one of a kind interdisciplinary art, we would have not otherwise considered. The project’s objective is to highligh the unique explorative creativity resulting from combining separate disciplines of science with art. At the same time the installation will be space to contemplate the bigger philosophical question such as what does it mean to be alive?
Using a bonsai as a metaphor for a coma patient, who cannot move and cannot speak, the project makes a comparison between two beings that are alive and if signs of life assume that consciousness exists. Our projects seek to question current ideas about how we can communicate with people who cannot communicate by movement or speech. Can we use existing medical technology to help people who are in comas, to see if there is any mental activity? What are the moral justifications related to coma patients and their welfare if signs of life are found?
The project is supported by medical visualisation technology, in this case an MRI (magnetic resonance imaging) machine and presents it in an interactive and accessible way for people to visualise the process of the image capturing process. By using MRI, can we find out or can we see what is within the living? Working with
scientists, mathematicians and physicists from the Institutute for Digital Medicine, we have collected thousands of images of the bonsai in an MRI machine. The artists’ request presented a novel challenge for the researchers at MEVIS, who usually work on methods to improve MR imaging of human patients. Imaging plants were new to them.
Ars Electronica Festival is one of the largest Media Arts Festival in the world and has a 40 year history that prides itself on the philosophy of ART, TECHNOLOGY and SOCIETY. By presenting at the festival, we hope to connect and contribute to this conversation about new possibilities in art and technology as well as how the project contributes to society and the implications it has in the real world.
The project is a multidisciplinary work that involves, scientists, mathematicians, psychologists, physicists, engineers, programmers and of course artists. The collaboration between the sciences and the arts is a growing field, and the project attempts to continue the open dialogues that happen because of these interactions. The festival draws people from all disciplines and facilitates dialogues between different groups of people throug talks and workshops and being part of the festival would mean forging new connections and potentially future projects.
Projection mapping on Bonsai using its own MRI Slices
3D printed branches and leaves generated from MRI scans attached onto Bonsai
Rapid prototyping process: 3D printing MRI scans at 0.15x scale
Aluminium wires embedded in bonsai during its nurturing phase, a mini surgery was required to remove them
SCIENCEUnderstanding and sending Bonsai into MRI Machine with Physcist Matthias Guenther
Monitoring Action Potential of Bonsai through live data streamed to IoT Database Adafruit IO
Workshop held at Ars Electronica Center in Linz, Austria
1. Announcement of STEAM Imaging II Residency Results – http://www.adm.ntu.edu.sg/NewsnEvents/Pages/News-Detail.aspx?news=cc7336cd-bb48-4e56-97ac-f333450863b6 , https://www.mevis.fraunhofer.de/en/press-and-scicom/institute-news/2019/steam-imaging-ii--artists-in-residence-are-jake-tan-and-ernest-wu.html
2. Residency Kick Off at Fraunhofer MEVIS in Bremen, Germany – https://twitter.com/FraunhoferMEVIS/status/1179351588730736640
3. Press Release – https://www.mevis.fraunhofer.de/en/press-and-scicom/press-release/2019/patient-bonsai.html
4. Ars Electronica Blog Interview – https://ars.electronica.art/aeblog/en/2019/11/04/patient-bonsai/
Project Coordinator Singapore
Project Development, Mentor & Mediator between Artist and Scientist
Ars Electronica Project Coordinator
Ars Electronica Project Coordinator
STEAM Workshop Leader
STEAM Workshop Supporter
Technical Support, MeVisLab Trainer
Artistic Support (Sound Designer)
MRI Expert, Physcist
1 STEAM stands for the connection of science, technology, and mathematics with the world of art. The “STEAM Imaging” program enables artist residencies focused on connecting science with the approaches of digital art. Integral components
include several workshops that aim at inspiring youth about the unusual alliance between art and science.
2 Patient Bonsai Fraunhofer MEVIS 4.11.2019 https://www.mevis.fraunhofer.de/en/press-and-scicom/press-release/2019/patient-bonsai.html
3 Physicist Professor Matthias Guenther, Mathematician Sabrina Hasse