Canada Research Chair creates extraordinary art installation in Korea

It’s hard to imagine an aesthetic experience that weaves together art, music, virtual reality, mathematics, philosophy and software engineering to create an out-of-this-world encounter. If you were in Seoul, Korea last fall, you may have been lucky enough to experience this first hand.

Last October, York University Professor Graham Wakefield, Canada Research Chair in Interactive Information Visualization, contributed to an art exhibit at South Korea’s Seoul Museum of Art. “Requiem for Hybrid Life,” curated by Kyoungmi Kim of the New Media Art Research Association, ran from Oct. 17 to 23, 2017, and featured the work of Wakefield and fellow artist-researcher and recent York University Visiting Professor Haru Ji.

What Wakefield and Ji created, after a feverish four-day installation process, was breathtaking.

“The excitement created by new immersive technologies that can generate life-like interactive experiences parallels the enthusiasm brought about by the birth of cinema,” says Wakefield. “Interactive virtual worlds and mixed realities will be increasingly important forms of creative content in the future,” he adds.

Wakefield, who came to York University three years ago, is a core member of the high-profile Vision: Science to Application (VISTA) program and the director of the Alice Lab for Computational Worldmaking in the School of the Arts, Media, Performance & Design (AMPD), which constructs responsive artificial worlds experienced through mixed/hybrid reality technologies, including Virtual and Augmented Reality.

“Conservation of Shadows” integrates with historically charged space, creates something new

The title of Wakefield and Ji’s installation piece in the Seoul show is “Conservation of Shadows” (2017). Part of an ongoing series called “Artificial Nature,” it is composed of 330 kilograms of salt; 12 nD::Node programmable circuit boards used to write and upload computer code developed by fellow AMPD researcher Professor Mark-David Hosale; 72 vibration motors, 132 bells, 150 meters of wire, two Kinect 360s, motion detectors for computers; and one HTC Vive HMD, which is a virtual reality system.

The Seoul exhibition space is quite large and barn-like with old timbers through which one can see the sky during daylight hours. It is rich in history that directly contributed to the installation, as part of the overall experience: “This building, an extension of the Seoul Museum of Art, used to be part of the Korean government’s Centre for Disease Control and Prevention. It was used for the storage of infectious diseases and materials and various other forms of biological matter. So, it has a very charged atmosphere,” Wakefield explains.

What visitors experience is unparalleled

Visitors enter the vast, dimly lit room, which features a series of well-placed bells – 132 miniature bells, in fact – attached to cables hanging from the ceiling like organic tendrils. The bells and their circuits were constructed with the assistance of four students in York’s Digital Media program: Nicholas Abbruzzese, Filiz Eryilmaz, Adiola Palmer and Amir Bahador Rostami.

The resulting sound creates a haunting interactive ambience. “The miniature bells are activated by small vibration motors — the same kind that makes a cell phone vibrate. These bells and motors surround the installation space, hanging down from the rafters at different locations and different heights,” Wakefield explains. “The bells aren’t perfectly manufactured, so each has a slightly different tone, which helps create a richer and more variegated sonic experience,” he adds.

Visitors feel and hear the salt granules crunching underfoot with each step as they progress farther into this engaging environment. Shadowy images, ghostly vortexes that represent other life forces, are projected downward from the ceiling where they mingle with the visitors’ shadows – each such interaction being distinct, unpredictable and impossible to replicate.

Those individuals choosing the virtual reality option can experience another layer, a different reality, where they witness mesmerizing, three-dimensional (3-D) flecks of dancing white formations that move together like a murmuration of birds against a limitless black background. In this alternative reality, fellow visitors to the installation space are captured as mysterious black voids, fully incorporated into the virtual reality setting. In this way, the visitors themselves become shadows that, again, interact with the projected shadows.

The bells are also replicated in the virtual reality space, such that the motors become more active and the bells ring more intensively when triggered. The effect  ̶  organic, technological and metaphysical  ̶  is unforgettable.

Wakefield wanted the images and sensations to swim together to create a compelling imagined world. He describes this process: “We imagined unknown new beings growing fond of the wet texture of old wood [timbers overhead], the fragrance of sunshine smeared between cracks, and the quietness of murmuring and whispering. To let the new beings live, we extended senses to mix realities surrounded by softly ringing bells and the crunch of salt underfoot as their shadows pass by; and an alternate perspective through head-mounted display in which we become the shadows around which new beings play.”

Audience response was overwhelmingly positive. “One of the comments that we received, many times, was how well the installation fit the space, given its unique character and history,” says Wakefield.

Wakefield’s work will shape future of arts and entertainment sectors

Wakefield’s forward-looking work will lead to the development of new artworks and technologies for emerging art forms and creative industries. His research will help meet the demand for more immersive, dynamic and open-ended interactive experiences in the arts and entertainment sectors.

“Entertainment and software industries are already investing heavily in these areas while acknowledging the need for new software and aesthetic practices,” says Wakefield.

To learn more about Wakefield, visit his faculty profile. For more information about “Artificial Nature,” visit the website. To learn more about the Alice Lab for Computational Worldmaking, visit the website.

To learn more about Research & Innovation at York, follow us at @YUResearch, watch the York Research Impact Story and see the snapshot infographic.

By Megan Mueller, manager, research communications, Office of the Vice-President Research & Innovation, York University, muellerm@yorku.ca