Reflections and Refractions

About environment mapping

Babylon.js uses environment mapping (wikipedia) to simulate reflection (mirror-like materials) and refraction (glass-like materials).

Environment maps are pictures of the world (as seen from some vantage point) which are transformed and applied to meshes to simulate reflection or refraction. These pictures may be single images or composite panoramas, and may be static snapshots or dynamically updated to track scene changes.

Once created (as described below), environment maps may be used in StandardMaterial's .reflectionTexture or .refractionTexture; environment maps are also used in Physically Based Rendering for sophisticated material surfaces.

Static environment maps (CubeTexture and friends)

CubeTexture instances use six images to make a static wraparound environment map (or "cubemap"). The CubeMap constructor takes a base URL and appends "_px.jpg", "_nx.jpg", "_py.jpg", "_ny.jpg", "_pz.jpg" and "_nz.jpg" to load images for the +x, -x, +y, -y, +z, and -z facing sides of a cube.

(Use the extensions argument to customize these suffixes, e.g. to load .png instead of .jpg images.)

Despite the "Texture" name, CubeTexture can only be used with the .reflectionTexture or .refractionTexture properties of StandardMaterial, not other properties like .diffuseTexture.

Skybox cubemaps

Skybox images may be used directly as environment maps. (Conveniently, skyboxes also typically use CubeTexture.) In this case only the skybox will show up in the reflection/refraction; other scene members will not be reflected, but that may be acceptable for simple scenes or small surfaces.

This classic cloudy skybox helps demonstrate skybox reflection:

+x (right)	-x (left)	-y (up)	-y (down)	+z (back)	-z (front)
+x (right)	-x (left)	-y (up)	-y (down)	+z (back)	-z (front)

Cube Reflecting SkyboxGround Reflecting SkyboxSphere Reflecting Skybox

With two shapes in the scene, it becomes clear that only the skybox is reflected:

Vampire Cube Isn't Reflected

This "vampire effect" can be fixed by carefully adding scene contents to cubemap images, or by using dynamic maps.

Local cubemaps

Environment maps (static or dynamic) are flat images with no depth. By default, they are treated as infinitely far away. This works for distant environments (like skyboxes) or small surfaces, but can cause parallax errors in other cases.

As an alterative, cubemaps can be treated as axis-aligned boxes of specific size and location by setting the cubemap texture's .boundingBoxSize and .boundingBoxPosition to the desired box size and position (as Vector3).

Local cubemap demo

(Local cubemaps must use the default CUBIC_MODE coordinate mode.)

An axis-aligned box is still an approximation of true reflected/refracted scene geometry, but it may be a useful approximation. See "Reflections Based on Local Cubemaps" (from the ARM Developer's Graphics and Gaming Blog) for a good write-up of the concept as used in Unity (Babylon.js's implementation is similar).

EquiRectangularCubeTexture

Equirectangular panoramic images (panotools wiki) squish a spherical panorama into one image (unlike the six sides needed for ordinary CubeTexture) using equirectangular projection (wikipedia).

EquiRectangularCubeTexture instances load an equirectangular panorama image into a CubeTexture-compatible environment map.

HDRCubeTexture

High Dynamic Range (wikipedia) images capture brighter and dimmer colors than a typical monitor can display. This can be useful for environment mapping even if the final output uses standard dynamic range.

HDRCubeTexture instances (Babylon.js v3.2+) loads a Radiance RGBE format (wikipedia) HDR equirectangular panorama image into a CubeTexture-compatible environment map.

Flat textures as environment maps

Wraparound cubemaps work well for environment mapping, but ordinary flat Texture instances can be used for .reflectionTexture and .refractionTexture as well. The texture's .coordinatesMode must be set to define how the flat image should be "wrapped around" as an environment (see "Coordinate Modes").

Mona Lisa in SPAAAACE, reflected using PLANAR_MODE

(Note, cubemaps are normally preferred for environment mapping instead of flat textures.)

Dynamic environment maps (RenderTargetTexture and friends)

RenderTargetTexture instances hold environment maps updated during scene rendering, allowing reflections and refractions to track the scene in real time (unlike the static maps described above).

RenderTargetTexture instances are usually created through classes like ReflectionProbe or MirrorTexture, which manage the dynamic update process.

ReflectionProbe

Each ReflectionProbe instance provides a cubemap (in .cubeTexture) that is dynamically rendered from a specified point in the scene, typically at or near an object that will use the cubemap.

Reflection Probes

You must set each ReflectionProbe's .renderList to an explicit list of meshes to render. Be mindful of efficiency as each probe renders six times (once for each cube face) for every update. See the Reflection Probes main page for more details.

MirrorTexture

MirrorTexture (a RenderTargetTexture subclass) acts as a dynamically rendered environment map for flat mirrors.

Mirrors

As with reflection probes, you must set each MirrorTexture's .renderList to an explicit list of meshes to show in the mirror, keeping efficiency in mind. You must also set each MirrorTexture's .mirrorPlane to the Plane of reflection, with the plane's normal pointing into the mirror (away from the viewer).

You may set Plane coordinates directly, but it can be convenient to build the Plane from mesh geometry:

// Create, position, and rotate a flat mesh surface.
const mesh = BABYLON.MeshBuilder.CreatePlane("mirrorMesh", {width: 5, height: 5}, scene);
mesh.position = new BABYLON.Vector3(0, 0, 4);
mesh.rotation = new BABYLON.Vector3(Math.PI/4, Math.PI/6, Math.PI/8);

// Create the reflective material for the mesh.
mesh.material = new BABYLON.StandardMaterial("mirrorMaterial", scene);
mesh.material.reflectionTexture = new BABYLON.MirrorTexture("mirrorTexture", 512, scene, true);

// Get a normal vector from the mesh and invert it to create the mirror plane.
mesh.material.reflectionTexture.mirrorPlane = BABYLON.Plane.FromPositionAndNormal(
    mesh.position, mesh.getFacetNormal(0).scale(-1));
mesh.material.reflectionTexture.renderList = [... list of meshes ...];

To create a blurred reflection, set MirrorTexture's .blurKernel to the blur kernel size (higher is blurrier) relative to the render texture size (set when MirrorTexture is created). Use .adaptiveBlurKernel instead to scale the blur value by the ratio of the render texture size and viewport size.

Reflection Blur

RefractionTexture

RefractionTexture (also a RenderTargetTexture subclass) is similar to MirrorTexture, but designed for refraction instead of reflection.

RefractionTexture acts as a dynamically rendered single-image environment map for flat refractors (materials like glass or water that bend light). You can use RefractionTexture instances directly in .refractionTexture. (Note, RefractionTexture simulates fixed-thickness plates of refractive material, and does not support lenses or shapes "bent" by immersion.)

Refraction

As with MirrorTexture, you must set each RefractionTexture's .renderList to an explicit list of meshes to show in the refraction, keeping efficiency in mind.

You must also set each RefractionTexture's .refractionPlane to the Plane of refraction, with the plane's normal pointing out of the refractor (toward the viewer). You may set Plane coordinates directly, but it can be convenient to build the Plane from mesh geometry as described above for MirrorTexture (but without the .scale(-1) call).

Finally, you must set each RefractionTexture's .indexOfRefraction and .depth to the index of refraction (wikipedia) and thickness of the refractive plate to simulate.

Changing Index Of RefractionChanging Refraction Depth

Coordinate modes

By default, cubemap textures (static and dynamic) simulate the physical Laws of Reflection (wikipedia), to create realistic shiny surfaces. However, you may set a texture's .coordinatesMode to select different reflection behavior, and flat (non-cubemap) textures always require you to set .coordinatesMode. (Refraction does not use coordinate modes.)

See the source (reflectionFunction.fx, called by default.fragment.fx) for the mathematical definition of each mode.

Examples with a test pattern cubemap

+x (right)	-x (left)	-y (up)	-y (down)	+z (back)	-z (front)
+x (right)	-x (left)	-y (up)	-y (down)	+z (back)	-z (front)

PLANAR, (INV)CUBIC, SKYBOX, and PROJECTION Modes with Test Cubemap

Examples with a test pattern flat texture

PLANAR, EQUIRECTANGULAR, PROJECTION, and SPHERICAL Modes with Flat Test Pattern

The reflection matrix

In addition to selecting the overall coordinate mode, a texture's reflection matrix (.getReflectionTextureMatrix()) may be adjusted to transform the environment map, to match scene changes (e.g. a moving background) or for special effects.

Sphere Reflecting Rotating Skybox

Further reading

Skyboxes

Learn how to add a skybox to your Babylon.js scene.

Introduction to Physically Based Rendering

Start learning about Physically Based Rendering in Babylon.js.

Reflection Probes

Learn how to use reflection probes in your Babylon.js scene.