Tim Cook unveils Apple Vision Pro mixed reality headset

Apple CEO Tim Cook ushered in a new era for mixed reality with the announcement of Apple Vision Pro, a new augmented reality headset that was the company’s first new product category in many years.

The headset announced at the Apple Worldwide Developer Conference wasn’t a surprise thanks to a number of leaks, but experts had been predicting it would happen since 2015. The device comes early next year at $3,499. That price point is higher than rumored, and the debut is later as well.

The “pro” part of the name suggests this is more for professionals and enterprises than it is for consumers. The idea is that it is a “revolutionary spatial computer” that seamlessly blends digital content with the physical world while allowing users to stay present and connected to others. Cook didn’t mention the word “metaverse,” which has been a favorite of rival Mark Zuckerberg of Meta.

“Just as the Mac introduced us to personal computing, and iPhone introduced us to mobile computing, Apple Vision Pro introduces us to spatial computing,” Cook said. “Built upon decades of Apple innovation, Vision Pro is years ahead and unlike anything created before — with a revolutionary new input system and thousands of groundbreaking innovations. It unlocks incredible experiences for our users and exciting new opportunities for our developers.”

Apple wants to change “change the way users interact with their favorite apps, capture and relive memories, enjoy stunning TV shows and movies, and connect with others in FaceTime.”

You can use the device as a see-through device to view the real world in a new way with animated overlays and infinite screen real estate. Or you can use it as a virtual reality headset to be fully immersed within an environment when you don’t want to be disturbed. Interestingly, people can see your eyes when you want to interact with them, even while you’re wearing the headset.

“This has been a huge day,” Cook said at the closing. “There are certain products that shift the way we look at technology and the role it plays in our lives. Apple Vision Pro introduces an entirely new spatial computing platform. And this is just the beginning.”

The details

Alan Dye, vice president of human interface at Apple, said you control it with your eyes, hands and voice. But in a side comment, Apple said game controllers will work as well.

You can use Siri to quickly open or close apps with your voice. You’re never isolated from the people around you. It displays your eyes, using a tech dubbed EyeSight, to people around you when you are wearing the device if you are near others. This changes when you are fully immersed in an app.

When a person approaches someone wearing Vision Pro, the device feels transparent — letting the user see them while also displaying the user’s eyes. When a user is immersed in an environment or using an app, EyeSight gives visual cues to others about what the user is focused on.

Richard Howarth, vice president of industrial design at Apple, showed off the design of the device, which is full of cameras, sensors and high-resolution displays. It has a computer that quietly draws air through the system. The band that straps around your head is huge, and it looks like no other headset I’ve seen.

Flexible straps ensure audio remains close to the user’s ears, while a head band — available in multiple sizes — is three-dimensionally knitted as a single piece to provide cushioning, breathability and stretch. The band is secured with a simple mechanism, making it easy to change to another size or style of band.

Yet the strap has ribbed cushions for breathability and stretching. You can get custom optical inserts from Zeiss for those who wear glasses. You can connect an external battery to it so you don’t wear that on your head.

The device is expected to have a lot of apps and games available when it ships in the fall. Focus areas include games, wellness, fitness, virtual collaboration, video conferencing, virtual meeting rooms, sports viewing and more.

You can have multiple apps open at once when using it to work. You can turn your head and see Safari or turn your head the other way and see your Mac screen. You can bring your Mac screen into your view at a 4K resolution so you can use Vision Pro side by side with your Mac. Apple said that you’ll be able to sit in a personal movie theater with a screen that feels 100-feet wide, with a spatial audio system.

The Apple Immersive Video will offer 180-degree high-resolution recordings with spatial audio, and users can transport themselves to entirely new places.

It has 23 million pixels, more pixels than a 4K TV for each eye. The device features three levels of “catadioptric” lenses for clarity that “are simply not possible with any other device,” Apple said. It uses micro-OLED screens, with wide color and high dynamic range.

The Apple Vision Pro has an audio vision system for spatial audio and something called audio ray tracing. The device has a high-performance vision tracking system with IR cameras and more. It uses the Apple M2 chip to run it at a comfortable temperature. It has a new chip dubbed R1, to process input from 12 cameras, five sensors and six microphones. The headset streams new images to the displays in 12 milliseconds, or eight times faster than the blink of an eye.

There’s a display on the outside that shows a video of your eyes that others can see. You scan yourself in and it can create a persona, a representation of you that you can use to communicate with others. It has a new operating system dubbed visionOS. It has foveated rendering, a multi-app 3D engine, a real-time subsystem, a core OS, and more. And it supports the low-latency requirements of spatial computing.

By responding dynamically to natural light and casting shadows, it helps the user understand scale and distance. 

You can press a button on the side of the headset and capture a spatial video, seeing it from any angle as you wish. You can also see a panorama photo on a huge virtual screen. You can watch movies on an airplane in an immersive view where you can’t see anyone else. You can watch 3D movies as well, as they will have more depth and motion.

You can see interactive videos with dinosaurs sticking their heads out of the screen. You can play arcade games like NBA 2K23 with a game controller, and there will be more than 100 Apple Arcade games available with immersive audio.

Bob Iger, CEO of Disney, came out on stage to praise Apple Vision Pro. He showed how you can view movies and immersive yourself in screens full of sports information.

He showed you can bring things like the Disneyland parade into your living room. Hundreds of apps and games will be available, and Apple said it has been working with Unity for a long time. (No mention of Epic Games’ Unreal, of course).

Immediate reaction

Apple showed off video conferencing apps that will work with the device and Apple said it had been working with outside developers for a long time.

Meta anticipated Apple will dive into the market. So Meta CEO Mark Zuckerberg went on the offensive, announcing last Thursday that the Meta Quest 3 virtual reality headset will debut in the fall for just $500.

Clearly, Apple has been working on the apps for a long time, and they look great for tasks like collaboration and working with your Mac. I’m looking forward to seeing more gaming apps and VR experiences.

You can view your library of videos and photos as you wish on a huge screen that you see before your eyes.

With more than 20 million Quest and Quest 2 devices sold, Meta has cred in the market. And it is packing a lot of tech into the device, which is 40% thinner than its predecessor and has color passthrough for better mixed reality applications. One developer, who already has the Meta Quest 3 prototype, said that the Meta device could use more memory, but it will be interesting competition for Apple.

But you can see that these devices are very different from each other.

Privacy and security

Apple uses Optic ID to identify you via the uniqueness of your iris. It is encrypted, never leaves your device and is secure on the processor. It works with purchases.

Apple said Optic ID is a new secure authentication system that analyzes a user’s iris under various invisible LED light exposures, and then compares it to the enrolled Optic ID data that is protected by the Secure Enclave to instantly unlock Apple Vision Pro.

Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences. EyeSight also includes a visual indicator that makes it clear to others when a user is capturing a spatial photo or video.