Realistic Hand Interaction (Beta)

VIVE OpenXR plugin offers the Realistic Hand Interaction feature.
The imported prefabs and samples are located in Packages VIVE OpenXR Plugin > Runtime > Toolkits > RealisticHandInteraction.

• Supported Platforms and Devices
• Specification
• Environment Settings
• Golden Sample
  • Common Use
  • Additional Use
• See Also

Specification

The VIVE OpenXR Unity plugin supports Realistic Hand Interaction with Unity XR Hands. This functionality relies on the Hand Tracking Subsystem, which requires the OpenXR feature group to be enabled.

Environment Settings

The Realistic Hand Interaction toolkit requires Unity 2021.3.31 or newer and Unity XR Hands 1.3.0.

Before using Realistic Hand Interaction, make sure to install Unity XR Hands and checking Hand Tracking Subsystem in OpenXR Settings from Edit > Project Settings > XR Plug-in Management > OpenXR.

Due to some issues with Unity XR Hands on PC, it is not recommended to use Realistic Hand Interaction on PC.

Golden Sample

The Realistic Hand Interaction feature contains two main components: HandGrabInteractor and HandGrabInteractable.

The HandGrabInteractor updates hand tracking data and allows grabbing at gameobjects with the HandGrabInteractable component.

The HandGrabInteractable component allows gameobjects to be grabbable. Currently, only gameobjects with at least one 3D Collider component are supported.

These prefabs include left and right hand grabbers, which you simply need to place into your scene.

Each prefab comes with a hand mesh, hand grabber, hand collider, and custom grab pose.

How to Use Realistic Hand Interaction

Common Use

Only gameobjects with the HandGrabInteractor component are capable of grabbing gameobjects with the HandGrabInteractable component.

1. Add HandGrabInteractor and modify the script options if needed.
   - Handedness : Set the hand for hand tracking.
   - Grab Distance : Set the grabbing distance.
   - Enable Collider : Toggle to enable or disable the hand collider.
   - GrabColliderManager : Set the grab collider manager.

2. Add HandGrabInteractable to gameobject with collider or its children have.

3. If gameobject with collider.
   
   
4. If gameobject without collider but its children have.
   
   
5. Is Grabbable : Allow oneself to be grabbed.
   
6. Force Movable : Allow oneself to be moved. The priority of this option is lower than that of Rigidbody.
   
7. Finger Requirement : Define which fingers are required for grabbing.
   

8. Grab Poses Settings : Define the hand gestures for grabbing.

9. Set the Finger requirement on the HandGrabInteractable of the gameobject, and then build and run to test.
   

10. Ignored: This finger is not needed for grabbing.
11. Optional: This finger can be used for grabbing.
12. Required: This finger is required for grabbing.

Additional Use

Align the hand mesh with the gameobject by using the Grab Pose

When grabbing the gameobject, the hand mesh may overlap with the gameobject. To achieve a more realistic grabbing effect, you can customize the grab pose. This needs to be done in the Unity Editor play mode. Once completed, the grab pose will be saved in the Grab Pose Settings of the gameobject. As a result, the gameobject will be grabbed according to the customized grab pose during grabbing.

Here shows the before-and-after comparison of setting the Grab pose.
Before setting the Grab pose:

After setting the Grab pose:

Customized Grab Pose (Script)

1. Add component CustomGrabPose to the gameobjects with HandMeshManager.
2. Disable Enable Collider of the HandGrabInteractor component.
   
3. In Unity Editor, starting the play mode and set the position of the hand model and the gameobject at the same location.
   
4. Modify the Grab Gesture of the CustomGrabPose and adjust the position of the hand model or the gameobject until it matches your expectation.
   
   
5. Click "Save HandGrab Pose" of the CustomGrabPose before stopping the play mode.
6. Check the Grab Poses settings of the HandGrabInteractable on the gameobject. It should updated.
   

Tip1: Duplicate to other hand
1. You can click the "+" button of Grab Poses settings to create a duplicate.
2. Click "IsLeft" to toggle between left and right hand grabbing.
3. Click the Mirror Pose "Align X axis", "Align Y axis", and "Align Z axis" will project the GrabPose onto the left-right, vertical, and front-back directions.

Tip2: Change Hand Model (HandMeshManager)
1. Make sure your hand model can map to 26 joint nodes.
2. Set the hand model's root transform and mapped joints.
3. If your hand model structure matches the XR Hands Sample, you can simply press "Find Joints". If not, you'll need to set them one by one.

GrabColliderManager

To achieve a more realistic effect, you can add component GrabColliderManager to the gameobject with HandMeshManager.
GrabColliderManager allows the hand model to determine whether it can move a 3D object when touching. Here are a few influencing factors:

1. The Enable Collider property of the HandGrabInteractor needs to be set to true in order to move objects.
2. If a 3D object has a Rigidbody with the Is Kinematic property disabled, then the hand model can move it within the Unity physics engine.
3. If a 3D object has a Rigidbody with the Is Kinematic property enabled, the expected outcome depends on whether the Force Movable property of HandGrabInteractable is enabled:
   • When enabled, the hand model can still move the 3D object.
   • When disabled, the hand model won’t be able to move the 3D object and will remain stationary on it.
   • If a 3D object hasn't HandGrabInteractable, it will behave the same as the disabled scenario described above.
4. If the 3D object hasn't Rigidbody but has HandGrabInteractable, the expected outcome depends on whether the Force Movable property of HandGrabInteractable is enabled:
   • When enabled, the hand model can still move the 3D object.
   • When disabled, the hand model won’t be able to move the 3D object and will remain stationary on it.
     

See Also

Realistic Hand Interaction API