Apple Vision Pro - AR/VR

Apple have just unveiled their new AR (Augmented Reality) Headset. As expected, they have not only created a headset, but they have also completely reimagined the landscape of personal computing. Here’s a look into some of its key features and ideas.

Natural inputs - Eyes, hands and voice

The AR operating system is governed by the user's ocular movements, hand gestures, and verbal commands.

The user can navigate by looking at points of interest, such as the microphone icon and then use a voice command to activate a search, as demonstrated in the second example below.

While eye tracking has been extensively employed in research and practical applications, such as aiding fighter pilots, since the early 2000s, this marks the first instance of its extensive use within the consumer product space.

AR Video Calls

Apple has ventured into the realm of video calls within a spatial environment, enabling users to position individuals within their physical surroundings. This creates a very immersive and intimate experience.

Additionally, it addresses a potential scenario where the headset wearer may be the sole participant without an activated face camera. However, Apple has adeptly tackled this challenge by endowing users with a Persona—a digital representation of themselves—fabricated through the utilization of machine learning techniques.

This solution is remarkably elegant, leveraging the device's inherent capabilities to manifest the persona while harnessing the recently realized potential of AI technology.

Immersive video and spacial audio

Experiencing content through an augmented reality (AR) headset is undeniably captivating, as it engulfs your entire visual field and envelops you with immersive sound. The visualization of this experience in the video below really captures the imagination.

The airplane example below, further exemplifies the desirability of such an immersive experience.

Undoubtedly, the headset offers a compelling option for a fully immersive cinematic journey, and there is certainly a market demand for it. However, there is still an undeniable allure in the simplicity of not having to wear anything and instead, relishing a movie on the couch alongside someone.

EyeSight

Apple has tackled a prevalent issue that plagues current headsets—the loss of face-to-face interaction, particularly the absence of connection and emotional depth derived from gazing into someone's eyes. Exclusive to Vision Pro is a feature dubbed EyeSight, which displays the user's eyes externally when someone approaches.

From a practical perspective, this functionality also alleviates the need to remove the headset, sparing users from disorientation and inconvenience.

While some individuals may find the sight of the user's eyes externally a tad peculiar and unsettling, it is undoubtedly an area that will witness continual refinement and advancement, eventually resembling a more natural pair of glasses.

However witnessing the glow emanating from behind the lens is quite visually striking and will no doubt inspire the design of FUI.

Spacial Operating System

Apple proudly introduces VisionOS as the world's first spatial operating system, which is a significant advancement. They have cleverly devised a framework for creating a working system within an augmented space.

Apple is encouraging developers to delve into this space, which necessitates a meticulously designed system capable of addressing an array of novel challenges. These include seamlessly navigating content within a non-uniform environment, external interferences such as varying light and shadow, and the considerations of scale and distance.

Described as a fully three dimensional user interface ‘3DUI’ on an infinite canvas, this approach may give rise to workspaces that transcend conventional keyboards and monitors, in favour of virtual keyboards and augmented windows. As a result, it has the potential to reshape the entire computing experience.

It's interesting to see how Apple have thought about the integration of the virtual experience with our physical surroundings. This prompts us to examine how we might redefine our living spaces and the concept of home if this were to become the norm in the future. The need to mount a TV above a visually appealing fireplace or have multiple screens becomes obsolete for example. Such possibilities have far-reaching implications for the field of interior design. It's a captivating glimpse into a future where boundaries are redefined, igniting our imagination and sparking intriguing conversations.

3D photos and videos

With Vision Pro, Apple introduces it’s first 3D camera, allow you to capture photos and videos in three dimensions. They highlight the value of this capability for capturing precious moments, such as treasured memories with your children or significant milestones.

This feature not only enhances the level of detail in preserving memories but also offers the unique ability to virtually relive those moments by scaling up your 3D photos or videos to real-life proportions.

This technology presents an unprecedented opportunity to immerse yourself in memories like never before. As such, it raises questions about the potential effects it could have on treating conditions such as dementia and Alzheimer's or any number of other applications.

Privacy

While Apple does address privacy concerns, it remains somewhat disconcerting to introduce a device into your home and daily life that has the capability to capture 3D video, audio, and data. As depicted in the video, it raises concerns about the potential intrusion into personal moments, such as interactions with your children or private moments in your bedroom.

In essence, this represents a significant leap forward in our interaction with computers, heralding an era of intriguing possibilities. It is genuinely captivating to contemplate the potential applications that extend beyond the realm of personal computer experiences, as we eagerly anticipate the continued evolution of this technology.