Magic Leap sees a bright future in AR healthcare
The well-funded startup once known for putting an elephant in the palm of your hand is shifting its focus to helping keep people healthy.
Before surgery, surgeons often mentally rehearse what they’re going to do: where they’ll make the initial incision and what comes after that. But what if they could physically simulate the surgery? Augmented reality company Magic Leap has just signed an exclusive deal with Brain Lab, which supplies visualization and training software to surgeons. Through the partnership, Magic Leap will be able to render brain scans into 3D objects, which surgeons can interact with through its signature goggles.
The deal is just the latest health project on Magic Leap’s slate. It also marks a return to a field that CEO Rony Abovitz knows best: surgery. His first company, Mako Surgical, built robots that predetermined bone cut specifications for knee and hip replacement surgeries and assisted in surgery. It was acquired by Stryker Corporation for $1.65 billion in 2013.
Jennifer Esposito [Photo: courtesy of Magic Leap]
Health and wellness is one of five key areas that Magic Leap is targeting for its technology, along with mobility, entertainment, energy and water, and communications. “We’ve been talking about this concept of a digital operating room for a long time,” says Jennifer Esposito, the company’s VP of health and fitness. “Not just Magic Leap, but a lot of people.”
In the last few years, Magic Leap has gotten XRHealth, SyncThink, The Dan Marino Foundation, and Stanford Children’s Hospital to integrate with its spatial computing platform. Its eventual goal is to integrate with a variety of health tech companies, so it can coalesce a spectrum of health data and render a full picture of a patient’s health based on bodily factors as well as social determinants. Using its AR technology, it could provide doctors with a dashboard-like experience that shows a record of a patient’s physical health as well as related environmental factors—all overlaid on the real world.
“If your air quality is bad or the pollen count is high . . . there’s all sorts of things you’ll be able to understand,” says Esposito. She also sees an opportunity for Magic Leap to operate as a collaboration space for doctors and a way for patients to get access to specialists no matter how far away they might be. A lot of what Magic Leap is ultimately able to do will depend on its ability to lock down partners and get developers to create tools and applications on its larger platform known as the Magicverse.
Image: courtesy of Magic Leap
So far, health companies are leveraging Magic Leap as a way of visualizing data, running simulations, and training professionals. SyncThink, for instance, measures brain health using a wearable device that tracks eye movement. Doctors using SyncThink can already review the results remotely, but the company uses Magic Leap’s platform to visualize findings, effectively allowing a faraway patient to appear right in front of a doctor.
Magic Leap is working with the Dan Marino Foundation to train young adults with autism on how to interview for jobs, with a simulation that makes it feel like the person is actually in an interview. At Stanford’s Lucile Packard Children’s Hospital, teachers are already using the Magic Leap platform to simulate medical training. The company is also developing digital humans, avatars that look and behave like us. One day, these characters could be built into the platform to serve as educators or even medical cadavers.
Medical training has long been a key application for Microsoft’s HoloLens AR headset, Magic Leap’s principal rival. When the company first launched HoloLens in 2016, Case Western announced it would use the technology to teach anatomy at its cadaver-free medical school. Now, the HoloLens is used in a variety of health-related training scenarios, including simulated child birth, and has even made its way into hospitals. This year, Microsoft signed a deal with Philips Healthcare to connect Philips’s image-based medical guide to the HoloLens headset for use in precision surgery. Google too has been offering up its Glass for use in medical applications. The company has found a niche making administrative tasks easier.
For Magic Leap, which initially placed a lot of stock in using AR to transform entertainment, the active competition in health-related AR is a good sign. “I think the real point is that this is a really interesting time both for augmented reality and spatial computing, and so it’s great to see other companies thinking just like we are about the many use cases that are out there in the healthcare industry,” says Esposito.