How To

Use the WebVR experience helper


How to use the WebVR experience helper

Introduction

The WebVR Experience Helper provides a quick way to add WebVR support to a Babylon scene.

Features include:

  1. WebVR camera and non-WebVR camera initialization
  2. Enter WebVR button
  3. Teleportation and rotation in the world
  4. Gaze tracking with mesh selection from HMD and controllers

Setup

A VRExperienceHelper can be created directly from the scene.

var scene = new BABYLON.Scene(engine);
var vrHelper = scene.createDefaultVRExperience();

This will initialize a WebVR camera and a non-WebVR camera in the scene. It will also create an enterVR button at the bottom right of the screen which will start rendering to the HMD on click.

Options

  • createDeviceOrientationCamera(default: true): If the non-WebVR camera should be created. To use an existing camera, create it and then initialize the helper with this set to false in the constructor.
  • createFallbackVRDeviceOrientationFreeCamera(default: true): When no HMD is connected, this flag specifies if the VR camera should fallback to a VRDeviceOrientationFreeCamera which will render each eye on the screen. This can be set to false to only enable entering VR if an HMD is connected.

Teleportation and Rotation

To enable teleportation in the scene, create a mesh that the user should be able to teleport to and then enable teleportation with that mesh's name.

var ground = BABYLON.Mesh.CreateGround("ground", 6, 6, 2, scene);
vrHelper.enableTeleportation({floorMeshName: "ground"});

To teleport, hold up on the joystick to display where the user will be teleported to and then release to teleport. To rotate, move the joystick to the left or to the right.

When WebVR controllers are connected, the teleportation will be based on where the controller is pointing.

When WebVR controllers are not connected, the user will teleport to where the user is looking and teleportation can be triggered with an Xbox controller.

Teleportation events

Teleportation has two observables you can subscribe to:

onBeforeCameraTeleport: Observable raised when teleportation is requested, receiving target Vector3 position as parameter:

vrHelper.onBeforeCameraTeleport.add((targetPosition) => {
     //Raised before camera is teleported
});

onAfterCameraTeleport: Observable raised when teleportation animation finishes, receiving target Vector3 position as parameter:

vrHelper.onAfterCameraTeleport.add((targetPosition) => {
     //Raised after teleportation animation finishes
});

To enable teleportation in the scene, create a mesh that the user should be able to teleport to and then enable teleportation with that mesh's name.

var ground = BABYLON.Mesh.CreateGround("ground", 6, 6, 2, scene);
vrHelper.enableTeleportation({floorMeshName: "ground"});

Enabling / disabling teleportation

Teleportation can be enabled or disabled on demand by using the property teleportationEnabled:


// Enable teleportation
vrHelper.teleportationEnabled = true;

//Disable teleportation (teleportation mesh will not be displayed)
vrHelper.teleportationEnabled = false;

To customize the teleportation target mesh the following property can be set to the mesh you'd like to use:

vrHelper.teleportationTarget = BABYLON.Mesh.CreateSphere("sphere1", 4, 0.1, scene);

Accessing cameras

The VR and non-VR camera can be accessed from the helper to handle any application specific logic.

// Initial camera before the user enters VR
vrHelper.deviceOrientationCamera;
// WebVR camera used after the user enters VR
vrHelper.webVRCamera;
// One of the 2 cameras above depending on which one is in use
vrHelper.currentVRCamera;

Accessing controllers

The controllers can be accessed from the helper to handle any application specific logic.

vrHelper.onControllerMeshLoaded.add((webVRController)=>{
    var controllerMesh = webVRController.mesh;
    webVRController.onTriggerStateChangedObservable.add(()=>{
        // Trigger pressed event
    });
});

Please note that the microsoft controllers are using the GLB file format and require the GLTF Loader.

Accessing vr device position and rotation

Position and rotation in Babylon space can be accessed through the webVRCamera's devicePosition and deviceRotationQuaternion

// Left and right hand position/rotation
if(vrHelper.webVRCamera.leftController){
    leftHand.position = vrHelper.webVRCamera.leftController.devicePosition.clone()
    leftHand.rotationQuaternion = vrHelper.webVRCamera.leftController.deviceRotationQuaternion.clone()
}
if(vrHelper.webVRCamera.rightController){
    rightHand.position = vrHelper.webVRCamera.rightController.devicePosition.clone()
    rightHand.rotationQuaternion = vrHelper.webVRCamera.rightController.deviceRotationQuaternion.clone()
}

// Head position/rotation
head.position = vrHelper.webVRCamera.devicePosition.clone()
head.rotationQuaternion = vrHelper.webVRCamera.deviceRotationQuaternion.clone()

See Example -


Gaze and interaction

Gaze and interactions can be enabled through the enableInteractions method. See example -


vrHelper.enableInteractions();

This will start casting a ray from either the user's camera or controllers. Where this ray intersects a mesh in the scene, a small gaze mesh will be placed to indicate to the user what is currently selected.

To filter which meshes the gaze can intersect with, the raySelectionPredicate can be used:

vrHelper.raySelectionPredicate = (mesh) => {
    if (mesh.name.indexOf("Flags") !== -1) {
        return true;
    }
    return false;
};

This will cause the user's gaze to pass through any mesh which results in the raySelectionPredicate returning false.

As the user moves between meshes with their gaze, the onNewMeshSelected event will occur. Note: This only works after interactions have been enabled.

vrHelper.onNewMeshSelected.add((mesh)=>{
    // Mesh has been selected
});

This will return the single closest mesh that was selected.

Prior to onNewMeshSelected an event called onNewMeshPicked is raised when a mesh is selected based on meshSelectionPredicate successful evaluation. This observable notifies a PickingInfo object to subscribers.

vrHelper.onNewMeshPicked.add((pickingInfo) => {
    //Callback receiving ray cast picking info
});

As the user unselects a mesh with their gaze or controller, the onSelectedMeshUnselected event will occur.

vrHelper.onSelectedMeshUnselected.add((mesh) => {
    // Mesh has been unselected
});

You can add your own filtering logic with meshSelectionPredicate. Note: This will be applied after the raySelectionPredicate.

vrHelper.meshSelectionPredicate = (mesh) => {
    if (mesh.name.indexOf("Flags01") !== -1) {
        return true;
    }
    return false;
};

The logic order for raySelectionPredicate, meshSelectionPredicate, onNewMeshPicked, onNewMeshSelected are as followed:

  1. Ray is casted from the controller
  2. When the ray hits an object the raySelectionPredicate will be called and if true the ray will collide there and be stopped otherwise the ray will pass through the object
  3. Teleportation target location is updated to where the ray collided if the collision is also a floor mesh
  4. If the collision object was not collided with on the last frame meshSelectionPredicate is checked, if it returns true the onNewMeshPicked event is fired and then onNewMeshSelected is fired

The gaze tracker can be customized by setting the gazeTrackerMesh. Example -


vrHelper.gazeTrackerMesh = BABYLON.Mesh.CreateSphere("sphere1", 4, 0.1, scene);

Examples

Scenes:

Games: