We are not ready for the Apple Vision Pro

What wearing a computer on our face will do to us We are not ready for Apple's Vision Pro headset It’s Adam here. I’ve gone on parental leave until June, but I have one more newsletter for you. Starting next week and for the next several months, Abby Ohlheiser will be handling the newsletter. Now to the big story of the week: Apple’s Vision Pro headset. Whether it succeeds or it flops — as its predecessor Google Glass [ultimately did]( — the [Apple]( Vision Pro, which [goes on sale tomorrow](, will bring lots of fresh attention to the concept of mixed reality. How that attention shapes the development of the technology could dictate how we use computers in the future. Apple reportedly [sold as many as 200,000 units]( in the Vision Pro presale, a sign that at least a few people are interested in spending a lot of money to see what Apple thinks we eventually should do. Will we all walk around town wearing headsets and taking video calls while we watch the sunset? Or will a small group of us end up playing video games in a completely — and [probably lonely]( — virtual world? It’s not clear what Apple thinks you should do with its mixed reality headset right now. The company’s marketing materials seem to suggest you should use it in your living room [like a fancy TV]( that can [also make futuristic video calls](. The device has not one but 12 cameras on the front of it — not to mention a LiDAR sensor, six microphones, and a TrueDepth camera that can scan faces. Like the [Meta]( Quest, the new Apple headset can do fully immersive virtual reality, but it’s also capable of combining virtual elements with images of the real world, kind of like the Google Glass concept did. But whereas Google Glass, which was a pair of smart internet-connected glasses, let viewers look out at the real world with minimal interference, the Apple Vision Pro headset essentially [turns the world into a giant screen](. Inside the headset there are two 4K micro-OLED screens with a staggering 23 million pixels to recreate the real world outside the wearer as a virtual one. All those cameras, sensors, and screens combine to create the headset’s marquee feature, one that truly differentiates it from past smart glasses: [real-time passthrough video](. Passthrough, which also [exists on the Meta Quest 3](, allows the wearer to see the real world while wearing the goggles. Put simply, your eyes seem to pass right through the opaque front of the headset so you can see the room you’re sitting in and even interact with other humans. It’s also a way to get a taste of [that Google Glass fantasy](, where you can walk around with your face computer, looking at a world with a useful digital layer superimposed upon it. The big problem, of course, is that nobody knows how looking at the world through a screen and only a screen could screw up your brain and your relationship with society. Apple's vision and its inevitable distortions A group of researchers at [Stanford’s Virtual Human Interaction Lab]( have been trying to figure out the answer. The researchers tested out the Apple Vision Pro alongside a range of other headsets with passthrough video capabilities, [including the Meta Quest 3](. The researchers explored the psychological impacts of living life in passthrough video by wearing these headsets for hours at a time, even venturing out in public to try things like riding a bike or eating a meal while seeing through a computer. They shared their findings in a new paper [that sounds like a cautionary tale]( for anyone considering wearing the Vision Pro anywhere but the privacy of their own home. One big problem with the passthrough video technology is that cameras — even ones as high-tech as those in the Vision Pro — don’t see the way human eyes see. The cameras [introduce distortion]( and lack the remarkable high resolution in which our brains are capable of seeing the world. What that means is that everything looks mostly real, but not quite. The time spent observing the world through the headset isn’t the most alarming part of the experiment. It’s once they took the headsets off that the researchers’ minds really started playing tricks on them. The human brain [can generally adapt]( to changes in vision and effectively correct for any distortion, and it’s possible that Apple could push software updates that solve the distortion problem. But when the headsets came off, it took time for the researchers’ brains to return to normal, so they’d misjudge distances again. Many also reported symptoms of simulator sickness — nausea, dizziness, headaches — that will sound familiar to anyone who’s spent much time using a VR headset. Then there are the social effects. While wearing Google Glass in public [invited ridicule a decade ago](, headsets like the Meta Quest and, soon enough, the Apple Vision Pro will look more familiar to present-day observers. You don’t see people wearing them in public a lot, but [it’s not out of the question](. The researchers trying to live life with passthrough-capable headsets said they didn’t experience many negative reactions to what they were doing. Instead, they felt incredibly self-conscious to be navigating the world through goggles, and some experienced something called “social absence.” The researchers described this sensation in the paper: “People in the real world simply felt less real. ... Being in public could sometimes feel more like watching TV than interacting face-to-face. It was often embarrassing to interact with strangers while wearing a headset.” Jeremy Bailenson, who led the study and is the founding director of Stanford’s VHIL, has spent the last 25 years studying the psychological effects of virtual and mixed reality. He doesn’t think this technology should be used all day long or even every day, in part because of how socially isolating it can be. His lab has developed a framework it calls DICE. That’s an acronym for “dangerous, impossible, counterproductive, or expensive” — the use cases that make the most sense for employing VR and mixed reality technology. “Training firefighters, rehabilitating stroke victims, learning art history via sculpture museums, time-traveling to understand [climate change]( are all examples that fit squarely in DICE,” Bailenson told Vox. “Checking your email, watching [movies](, and general office work do not. Let’s use the medium when it earns its keep.” The inevitable growing pains of the technological process The Apple Vision Pro may be a marvel of modern industrial design befitting its four-figure price tag, but it still looks like a gaudy set of ski goggles. Apple even tried to make them feel less awkward for those not wearing them with a feature called EyeSight that shows the eyes of the wearer on a display built into the front side of the headset, but [people still think it’s creepy](. Meanwhile, in an effort to make the headset less bulky, Apple gave it a tethered external battery that people have to carry in their pockets, ensuring a bit of a cyborg vibe for the wearer’s outfit. It’s all far from natural. David Lindlbauer, a professor who leads the Augmented Perception Lab at Carnegie Mellon University, doubts that we’ll see people talking to their friends while wearing Vision Pro headsets at coffee shops in the near future. It’s simply strange to talk to someone whose face you can’t fully see. “Socially, we’re not used to it,” Lindlbauer said. “And I don’t think we even know if we ever want to get used to this asymmetry in a communication where I can see that I’m me, aware of the device, can see your face, can see all your mimics, all your gestures, and you only see a fraction of it.” But the situation might be temporary. Let’s not forget that it wasn’t that long ago when wireless earbuds, including Apple AirPods, meant that people were standing on street corners seemingly talking to no one when in fact they were on a phone call. If the hardware [continues to get better and smaller](, we might see people standing on street corners wearing regular-looking eyeglasses, staring off into space, tapping their fingers in the air because their glasses are projecting a digital layer onto the physical world. They could be looking for a friend down the street while simultaneously watching a [TikTok]( video and talking to a voice assistant. That kind of thing will definitely look unusual the first time you see it. This is still the big dream for computing. Google Glass did not end up being the mixed reality device that changed the world, but you have to assume that Apple [is investing billions in mixed reality](. It will likely take years for the Vision Pro and its successors [to take off like the iPhone or iPad](, but one day, Apple wants to see how people use a headset that turns your entire world into a computer. That’s actually what Apple Vision Pro promises to do. “The era of spatial computing has arrived,” said Apple CEO [Tim Cook]( in early January. The Vision Pro, he said, “will redefine how we connect, create, and explore.” But for now, the catch is it will probably just do those things while you’re inside your home. Mixed reality — or spatial computing or whatever you want to call it — is still an experiment. We’re not ready for it on a number of levels. The hardware isn’t there yet or we wouldn’t have to wear heavy ski goggle-shaped headsets and carry battery packs in our pockets. Perhaps if the display technology could replicate the resolution of human eyesight with no distortion or limitations, we would avoid all the negative consequences of passthrough video. The world will also need to ease into the idea of face computers, just like it had to get used to the idea of personal computers, and just like we’re still getting used to the ubiquity of smartphones. And our brains will need to do some adjusting to understand where the virtual world ends and the real one begins. —Adam Clark Estes, senior correspondent [An illustration of Elon Musk attempting to guide a man using a wheelchair into a mysterious, dark tunnel. The man has glowing threads that run from his hand to his head.]( Xinmei Liu for Vox [Elon Musk wants to merge humans with AI. How many brains will be damaged along the way?]( [Neuralink has implanted a chip in its first human brain. But it’s pushing a needlessly risky approach, former employees say.](   [A young boy wearing a red shirt rolls up a white mosquito net covering bedding at a South Sudan refugee camp. Behind him is a makeshift fence of sticks separating the sleeping area from nearby bushes.]( Luke Dray/Getty Images [Why would anyone be against life-saving malaria bednets?]( [The no good, very bad case against malaria bednets, explained.](   [Police officers stand in front of the headquarters of the New York Times on June 28, 2018, in New York City. Pedestrians with umbrellas walk by.]( Drew Angerer/Getty Images [How copyright lawsuits could kill OpenAI]( [The New York Times v. OpenAI, explained.](    [Learn more about RevenueStripe...](   [A photo illustration of a hand reaching out from a cell phone. Behind it are collaged elements including: a roulette wheel, a floral textile and armed men.]( Jared Bartman for Vox [How cyberscams are drawing China into Myanmar’s civil war]( [The surprising connection among cybercrime, human trafficking, and a raging guerrilla war.](   [Three speech bubbles representing the OpenAI GPT chatbot store are floating above a horizon in an etched drawing of a countryside.]( Paige Vickers/Vox; Getty Images [There are too many chatbots]( [Will OpenAI’s new chatbot store finally make AI useful?](   Support our work Vox Technology is free for all, thanks in part to financial support from our readers. Will you join them by making a gift today? [Give](   [Listen To This] [Listen to This]( [Police are solving fewer crimes. Why](? Three years after demands for police reform, police are solving fewer crimes. What happened? [Listen on Apple Podcasts](   [This is cool] [The sun's poles are about to flip. It's awesome — and slightly terrifying.](  [Learn more about RevenueStripe...](   [Facebook]( [Twitter]( [YouTube]( This email was sent to {EMAIL}. Manage your [email preferences]( , or [unsubscribe](param=tech)  to stop receiving emails from Vox Media. View our [Privacy Notice]( and our [Terms of Service](. Vox Media, 1201 Connecticut Ave. NW, Washington, DC 20036. Copyright © 2024. All rights reserved.