"Entering the Era of Spatial Computing": Apple releases MR headsets that are controlled by hands and eyes, selling for $3,499

Source: The Paper

Reporter Wu Tianyi

"You can interact with digital content in a virtual space just as you would in a physical space. You can control Vision Pro with the most natural and intuitive tools: eyes, hands and voice. Manipulation and viewing are no longer limited by the display screen constraints, the surrounding environment becomes an infinite canvas, and you can use the application anywhere."

"The headset is first to realize the content that Apple's ecology once had in the form of virtual reality."

On June 5, local time, Apple's 2023 Worldwide Developers Conference (WWDC) "Code Out the New Universe" officially kicked off. The long-awaited mixed reality (MR) headset made its debut. This is Apple's first The head-mounted display device is also Apple's first foray into this field. This is the first new category product released by Apple after many years.

It wasn't until the second half of the conference that Apple CEO Tim Cook (Tim Cook) mentioned "One More Thing", which brought exciting content to the industry: the release of the MR headset. Unlike the previously disclosed name "Reality Pro", the headset is named "Apple Vision Pro", uses a dual-chip design, and is equipped with a new VisionOS operating system. Priced at $3499.

The Vision Pro is priced at US$3,499, equivalent to around RMB 25,000.

"You can interact with digital content in a virtual space just as you would in a physical space. You can control Vision Pro with the most natural and intuitive tools, namely eyes, hands and voice. Manipulation and viewing are no longer limited by the display screen. Limits, the surrounding environment becomes an infinite canvas, and you can use the application anywhere." Cook said at the press conference that Vision Pro will open up "a new experience that has never been seen before."

One of the highlights of the head-mounted display is that it is not equipped with any controller. Users can select apps according to their sight, use different gestures to perform operations such as sliding, selecting, and switching, and use voice to control the device through Siri. At the conference, a series of manipulation functions were described as "like mind control".

For related apps, Apple has also made a new design for its own "family bucket" software to adapt to the operation mode of the virtual space. Li Pengpeng, director of XR product technology at Unity China, said to Pengpai Technology (: "This headset first realizes the content that Apple's ecology once had in the form of virtual reality." partner.

Chen Ming, CEO of YVR, previously told The Paper: "If Apple enters the field of MR, it will be an exciting news." Waiting for a "game-changing" player to enter the market and make its application into the public life. Apple has revolutionized all categories of digital products, including personal computers, digital players, mobile phones, tablets, watches and headphones, so it is expected to change the world again.

At least Cook hopes so: "Just as the Mac brought us into personal computing, the iPhone brought us into mobile computing, and the Apple Vision Pro brought us into spatial computing."

Apple released the MR head display device Apple Vision Pro, CEO Tim Cook (Timothy Cook) said it will open a "new experience that has never been seen before." (03:01)

Seamless connection between virtual and reality

This headset focuses on the integration of virtual space and real space, and Apple has carried out a new design on the inside and outside of the headset. Externally, the product is equipped with the "Eyesight" function. An OLED screen is equipped outside the Vision Pro to ensure that after the user wears it, the headset can map the user's eyes in real time to present a "sense of perspective". From the user's perspective, the internal environment of the headset can display the external situation in real time with the help of multiple cameras. From the demonstration point of view, various apps can be suspended in the user's room for manipulation at any time.

All kinds of new functions of the headset seem to serve to blur reality and virtuality. The "Eyesight" function will be turned off when the user is in an immersive experience, and the interior of the virtual space will also shield the external environmental information to a certain extent. Once there are people nearby When approached, the real scene will be displayed immediately in the virtual space, which is convenient for users to interact with people in the real world.

The "Eyesight" feature shows the user's eyes through the headset.

Vision Pro is also designed to "make the App move like a physical world". "The App can freely fill the space around you. It can be displayed in any position and any size in your room." The meeting introduced, "You can put the app anywhere you want. It feels very natural, like Moving real objects around and launching new apps doesn't take up the space of existing apps, just a new space around it. Seeing your favorite apps and experiences right next to you is truly a thing Extraordinary things.” When displaying related functions of private theaters, this experience is quite prominent, and the 100-inch giant screen combined with analog display theaters greatly expands the sense of space.

Users can use gestures to operate the App in the virtual space and place it anywhere in the space.

The fact that users can observe the external environment at any time is also reflected in the use of various Vision Pro functions. In the demonstration of the press conference, users can use gestures to operate the App in the virtual space while operating the Mac computer in the real space. Some people in the industry said that this function may be closer to the concept of augmented reality (AR).

Users can also receive 3D files in the App, and drag the 3D model to a new space with just a single "pull" with their fingers; during the immersive experience, the headset can also reduce the brightness of the real environment at the same time, Increase immersion. In addition, this product is also Apple's first 3D camera, which can not only watch 3D content, but also shoot.

Powerful but worrying about battery life

In order to support the headset, Apple also announced a new operating system "VisionOS", which Apple calls "an operating system designed for spatial computing", which can be synchronized with devices such as IPhone and iPad in real time. Judging from the release presentation, VisionOS still maintains Apple's consistent design language. Apple said that the system is "similar to MaOS and iPadOS, and built on decades of engineering innovation."

The operating system "VisionOS" is similar in style to Apple's previous operating systems.

In terms of hardware, Vision Pro adopts a dual-chip design. Apple has developed a new R1 chip, which is a dedicated image signal processor chip, which is used to process images captured by 12 cameras, 5 sensors, and 6 microphones to restore the surrounding environment. The other is the M2 chip that has been equipped with a variety of Apple products before, and is responsible for computing. With these two chips and an independent operating system, the device can complete calculations independently, does not need to be used with an iPhone, and can directly download the content required by the user, including iCloud data. Li Pengpeng said that from the perspective of the M2 chip, the computing power of this product is quite sufficient.

Vision Pro adopts a dual-chip design, equipped with Apple's self-developed R1 chip and M2 chip.

In appearance, the front of Vision Pro is a whole piece of curved glass, which serves multiple sensors. Apple also designed a solid Apple Watch-like knob for the Vision Pro, with an aluminum frame and a fabric headband. The headband is magnetically designed for easy disassembly at any time, adapting to the head sizes of different users. Myopia is also considered, and myopia lenses can also be mounted magnetically.

Apple claims that the display function of this product is also powerful, equipped with two 23 million-pixel Micro OLED screens, the pixels per eye exceed 4K, and support full 4K rendering. But the price of all kinds of powerful functions may be the issue of battery life. Different from other headsets on the market, Vision Pro uses an external battery to reduce the weight of the head. Although the battery is small and can be easily put in a pocket, the battery life is only two hours.

Vision Pro is equipped with two 23-megapixel Micro OLED screens.

Additionally, the device unlocks using iris scanning, based on a security system called Optic ID that works seamlessly with features like Apple Store purchases and password autofill, where user input is isolated to a single In a separate background process, apps and websites cannot learn about it. Equipped with multiple sensors, Vision Pro can also scan and build virtual avatars for users through neural network technology, and perform facial enhancement and simulate various expressions and actions of users through dynamic matching to realize real-time meeting, chat and other functions.

Content ecology or the future

At the conference, Apple stated that it has established a partnership with Disney on the content of head-mounted display devices. Disney CEO Bob Iger (Bob Iger) also shared his experience of using the device: "What we thought was impossible suddenly became possible." He also said that Disney will create a deep personal experience based on the MR headset , to shorten the distance between fans and characters. The "Disney+" service will also be launched simultaneously.

Disney CEO Bob Iger attends the Vision Pro launch.

In addition to expanding the content ecology, Apple also brings developers related development services for MR devices: "Our development tools are built on the basis of visual expertise. Developers can use familiar tools and frameworks, such as X Code, Swift UI, Reality Kit, which are new application toolkits made for Vision Pro. To simplify the preparation of 3D content for spatial applications, we have also developed Reality Composer Pro, which can easily combine complex scenes with realistic objects combine it all toghther."

Apple said that this means that thousands of iPad and iPhone applications will be released on VisionPro, "Users can use applications such as Adobe on the virtual big screen of MR, and can easily control them with only eyes and hands." Apple also announced a partnership with Unity to bring more apps into its new "MR App Store," and Unity-based games and apps can perfectly adapt to VisionOS features, such as high-resolution rendering and native gestures.

In this regard, Li Pengpeng told The Paper that from the perspective of the conference, whether it is from the design of the hardware itself, computing power, resolution, etc., or the combination with Vision Pro and the work scene as shown in the conference video, Apple's Vision Pro is amazing.