- There are three things AR does very well: visualization, annotation and storytelling;
- There are examples in each of these areas that are both timely in the current reality of COVID-19 and which can be built upon once cultural institutions, schools and workplaces reopen their doors;
- From enabling online learning to opening access to cultural events and experiences, applications of AR and VR could help us overcome the isolation of COVID-19 lockdowns.
Augmented Reality (AR) enables digital information to be superimposed and integrated into our physical environment. With many of us now at home during a global pandemic, AR is a tool that can help us transform our immediate surroundings into learning, work and entertainment spaces.
Have you read?
AR can help to bring the outside world in: from a virtual safari with 3D animals in your living room using Google’s AR search on your smartphone to collaborating with avatars of remote colleagues as though you were in the same room by using Spatial.
There are three things AR does very well: visualization, annotation and storytelling. There are examples in each of these areas that are both timely in the current reality of COVID-19 and which can be built upon once cultural institutions, schools and workplaces reopen their doors.
One thing is certain: AR is no longer just about the technology; it’s about defining how we want to live in the real world with this new technology and how we will design experiences that are meaningful and can enrich humanity.
1. Visualization:
AR is a powerful visualization tool. It allows you to bring an object or concept into a reality that is otherwise imagined, inaccessible or difficult to grasp, and can even help to make the invisible visible. Some examples illustrating its potential include:
With classrooms closed around the world, educators are moving to online learning models. All 185 first-year medical students at Case Western Reserve University (CWRU) are using HoloLens and HoloAnatomy, an award-winning AR app by CWRU and Cleveland Clinic, to learn from their own homes. HoloAnatomy helps students learn about the human body in ways not otherwise possible. With access to the minutest details of the human anatomy in 3D, students’ learning is not limited by the availability of cadavers for dissection or 2D medical textbook illustrations.
In December 2019, The New York Times published an AR experience visualizing microscopic pollutant particles that wreak havoc on human health. Using the app, you can see how the world’s most polluted air compares with your local city’s air. This eye-opening experience unfolds in your space depicting moderate to extreme pollution levels from the Bay Area to New Delhi. All data presented in the experience was prior to COVID-19, but AR could be used in a future project to visualize the dramatic impact of nationwide lockdowns on air pollution and the environment.
The Dan Marino Foundation and Magic Leap have created a virtual job interview training system for young adults with Autism Spectrum Disorder to practise in a safe simulated environment that builds competence and reduces anxiety. At the moment, communicating with virtual humans could also be a way to help people feel less isolated in their own homes.
2. Annotation:
Annotation with AR helps guide you through the completion of a task, navigate a new environment or even provide real-time descriptions of what’s happening around you.
Microsoft’s Dynamics 365 Remote Assist on HoloLens and mobile devices enables cross-distance collaboration by sharing a live view with experts for assistance. Experts can directly annotate what you are looking at to guide you through a process. Previously used in the workplace for field service repairs and training, remote assist scenarios can be extended to emergencies in remote locations where an expert, such as a medical professional, may not be readily available.
Elsewhere, London’s National Theatre is using AR to help make its performances more accessible for people who are deaf and hard of hearing. When wearing a pair of smart caption glasses, users see a transcript of the dialogue and descriptions of the sound from a performance displayed on the lenses. Cultural institutions around the world are presently live streaming content amid closures. When organizations reopen their doors and public, in-person experiences return, designing for accessibility will be an area that continues to develop in AR.
Microsoft’s Project Tokyo helps visually impaired people to “see” using AR and AI and the HoloLens. The device can detect the location of people in the user’s environment, and recognize faces, relaying the information to the wearer via audio. Perhaps a future feature could be added to help with physical distancing safety by providing an alert to the wearer when individuals are in proximity of fewer than two metres.
3. Storytelling
AR makes new modes of storytelling and creative expression possible with experiences unfolding in both our homes and public spaces. Introducing new and alternate perspectives, it changes the way we tell, share and even remember stories.
In February 2020, The Los Angeles Times partnered with Yahoo News, media company RYOT and artist Micah 404 to create an AR experience exploring iconic Oscars dresses from the past five decades. With the indefinite postponement of events like the Met Gala 2020 (the annual fundraising gala for the Metropolitan Museum of Art's Costume Institute in New York City) and its corresponding exhibition, AR could provide a way to bring the experience to your home by viewing and even virtually trying-on costumes from the exhibit.
Lessons in Herstory uses AR to help rewrite history books in the classroom and inspire the leaders of tomorrow by featuring stories of powerful women. Use your smartphone to scan over any portrait of a man in the school textbook A History of US, Book 5: Liberty For All? 1820–1860 to unlock a related story about a forgotten woman in history. You can enjoy the AR app from home for free without a textbook by scanning the images here using your smartphone.
The National Gallery of Prague is using haptics (virtual touch feedback) to help people who are blind and visually impaired experience artwork with Touching Masterpieces by Neurodigital. Wearing a pair of haptic gloves, users are able to “see” 3D virtual sculptures like Michelangelo’s David through a series of touch vibrations to the fingertips, palms and hands.
Seeing, hearing, and touching possible realities through the power of AR can stir our willingness to welcome and activate positive change in the world. Let’s make it our collective goal and commitment to design for the best of humanity.
