A3.1 - Interaction Metaphors Definition

Interaction metaphors are the backbone of XR experiences. They are a set of strategies that determine how users will perform activities inside the virtual environment. In most cases, the main activities are exploration of the environment and interaction with virtual objects.

Each of these activities can be decomposed in three fundamental interactions: navigation, selection, and manipulation.

1. Navigation: How will the user move in the XR environment?

   In VR, navigation is based on continuous and non-continuous movement, in continuous movement the user moves from one side to another at a certain speed and without interruption, while in non-continuous, the user is moved from one point to another (commonly called, teleportation). In AR, users typically move physically in their real-world space to reach the virtual content. Alternatively, the virtual content can be designed to move to the user’s location.

2. Selection: How will the user select in the XR environment?

   • Hand and finger interaction: A skeletal model of the user’s hands is created, allowing close interactions with the digital content.
   • Raycast: A “laser” extends from the virtual hand of the userd, with a cursor at the end indicating where it intersects with a target object.
   • Voice commands: Users can give commands without making hand gestures, using concise, simple vocabulary and avoiding similar-sounding commands.
   • Eye and gaze tracking: The direction vector of the user’s head can be determined or eye tracking can be used to interact through the user’s eyesight.
   • Controllers: Refer to devices or mechanisms used to interact through one or more inputs.

3. Manipulation: How to manipulate objects in the XR environment?

   XR manipulation generally takes advantage of raycasting and direct hand interaction. Raycasting allows objects to be reached at greater distances with a higher level of precision and control, and direct hand interaction allows the use of natural gestures and movements, providing a sense of immersion and ease of use.

Input

• User data: movements, gestures, voice, etc.
• Inputs Controllers

Output

XR environment interactions, navigation, selection and manipulation.

Control

• XR environment type (AR or VR)
• Selection and manipulation methods

Resources

• XR platform
• Skeletal model of hands and fingers
• Controllers or other input devices