What is spatial computing, and how will mixed reality blur the lines between the physical and the digital realms?
Spatial computing allows computers to naturally integrate into the physical world. Image: REUTERS/Mike Blake
This article has been updated.
- Spatial computing, which includes technologies like augmented reality (AR), virtual reality (VR) and mixed reality (MR), has been hailed as the next technological evolution. But what is it, and how will it shape the future?
- The World Economic Forum’s Meet The Leader podcast featured futurist and entrepreneur Cathy Hackl of Spatial Dynamics, who discussed how mixed reality will influence our lives and work, particularly through immersive applications and multiplayer environments.
- Hackl says spatial computing will not only lead to new types of data and applications but also raise important issues regarding privacy, talent sourcing and the integration of advanced sensors like LiDAR and depth-sensing cameras.
“Take time to play video games with the kids in your life so you can start to understand what it truly means to be a generation that flows between the physical and the virtual.”
This is the advice Cathy Hackl, futurist and CEO of Spatial Dynamics, gives other business leaders to familiarize themselves with a future where the physical and the virtual worlds will converge. Or, to put it technically, they will need to understand the concept and implications of spatial computing or mixed reality, she says.
Hackl joined the World Economic Forum’s Meet The Leader podcast series to speak about the future of human interaction with technology. Here are some of the key themes that emerged from the conversation.
What is spatial computing?
Spatial computing merges the digital and the physical world, overlaying our environment with virtual and augmented reality.
By using spatial computing, computers can naturally integrate into the physical world. Devices like Microsoft’s HoloLens and Apple’s Vision Pro headsets enable users to see the real world with digital objects superimposed that appear three-dimensional, blending with the real-world environment.
The headsets can turn virtual objects into holograms and make the experience more immersive through spatial sound and an understanding of the user’s surroundings.
Hackl describes spatial computing as the next evolution of computing, after the mainframe, desktop and personal computers and mobile devices.
“Mobile computing ushered in a huge sea change of opportunities and ways of interacting with technology,” she explained. “What you’re starting to see right now is an evolution from mobile computing into spatial computing – computing that extends beyond the screens. It starts to understand the physical world and expands computing further.“
What immersive technologies are involved?
Spatial computing encompasses various immersive technologies, including extended reality (XR), virtual reality (VR), augmented reality (AR) and mixed reality (MR). While these technologies share overlapping features, they differ in their implementation and user experience:
- VR creates fully immersive digital environments, typically using head-mounted displays (HMDs).
- AR overlays digital content onto the real world, often through smartphones or transparent headsets.
- MR blends virtual and real elements, allowing digital objects to interact with physical spaces.
- XR serves as an umbrella term, encompassing AR, VR and MR technologies.
Spatial computing leverages these technologies along with advanced hardware like depth-sensing cameras and inertial measurement units (IMUs) to enable seamless interaction between digital and physical realms. This integration facilitates innovative applications such as immersive virtual field trips and interactive science simulations, revolutionizing fields from education to industrial design
How will spatial computing change the way we work?
Although spatial computing is still a young discipline, there are several examples of how mixed reality could be applied, both by consumers and in the workplace.
Gaming is one obvious application, with Marvel now allowing fans to become part of its superhero universe in an hour-long mixed-reality experience.
But spatial computing isn’t just for would-be superheroes. Nasa used it in construction of the Orion spacecraft, part of its Artemis program to take people back to the moon and, eventually, to Mars. When Lockheed Martin started to assemble the seats in Orion’s crew module, there was no sign of instruction booklets or drawings. Everything the technicians needed to know came from the HoloLens 2 devices they were wearing: voice commands talked them through every step of the assembly, with holographic images of the parts overlaid onto the seats to show where and how they needed to be mounted.
“They found that using spatial computing reduced labour costs and hours by 90%. That is an early example, a very unique case,” Hackl told the World Economic Forum. “It’s starting to prove that spatial computing can give us superhuman powers of sorts, augmenting our workforce.”
Similarly, there are now examples in the medical field, such as a fully functional, immersive model of a human heart. Using Apple’s Vision Pro headset, trainee clinicians can virtually interact with the life-like heart simulation that reacts just like the real thing.
Hackl also pointed to the Forum’s Global Collaboration Village, which encourages problem-solving through partnership, as a stepping stone towards mixed reality.
What will it take for spatial computing to come to fruition?
Four key components make up spatial computing. Hardware is one of them, not just mixed-reality headsets but also new technologies including wearable AI devices such as the AI Pin and Rabbit R1, says Hackl. She foresees an explosion of these over the next few months.
When it comes to the next component, software, she stresses the importance of computer vision to enable computers to understand objects and people around them
“It's really important to understand that everything that's been happening in the AI boom is what's starting to help spatial computing truly happen. Because you need these spatial computers to understand the physical world. They can only do that through computer vision. For that, you need to create and train large models to understand the world.”
Large language and vision models in AI, along with game engines, have been paving the way for this, Hackl adds.
Another key component in spatial computing is data, with new types of data emerging and others being replaced. She points to 2D pixels being replaced by 3D “voxels”.
“For all of this to work and actually happen, you're going to need connectivity at levels we have never seen. 5G is not even going to cut it. You're going to need 6G and everything in between.”
How is the World Economic Forum creating guardrails for Artificial Intelligence?
In what ways does spatial computing pose new challenges?
“Spatial computing isn't one single technology or one single device. Spatial computing is almost like a new technological field,” Hackl points out.
And, as AI has shown over the last few years, new technology paradigms come with new challenges.
“When you expand computing into the physical world, you also have to start thinking about something called virtual air rights.”
“Who owns the air around me? Who has the right to put an ad in front of me? Who has the right to put audio in it? That's where I think virtual air starts to become a bigger conversation that is going to impact absolutely everyone.”
But a big technological shift such as this also has implications for the type of talent companies need to exploit the opportunities opening up. Getting close to a generation that has grown up as digital natives will play a big part in succeeding at this.
“Generation Alpha is a generation I do a lot of work with and study very closely. My three children are Generation Alpha. To them, what happens in the virtual space isn’t less real than what happens in the physical space. They move between both realms.”
“When you look at Gen Alpha, they are more in the gaming space. They're very entrepreneurial. They are ‘world builders’. They have been building worlds in Minecraft and Roblox,” highlights Hackl. “That in itself puts a different spin on what companies need to do to hire professionals, retain them and train them.”
To attract the youngest customer demographic, business leaders will need to immerse themselves in the mixed reality world that comes naturally to Gen Alpha. Hackl recommends the proverbial “getting down with the kids”, joining them in their digital pursuits, to truly get to grips with what makes this generation tick.
The quotes in this article have been lightly edited for clarity.
Don't miss any update on this topic
Create a free account and access your personalized content collection with our latest publications and analyses.
License and Republishing
World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.
The views expressed in this article are those of the author alone and not the World Economic Forum.
The Agenda Weekly
A weekly update of the most important issues driving the global agenda
You can unsubscribe at any time using the link in our emails. For more details, review our privacy policy.