Reflections and Refractions

About environment mapping

Babylon.js uses environment mapping (wikipedia) to simulate reflection (mirror-like materials) and refraction (glass-like materials).

Environment maps are pictures of the world (as seen from some vantage point) which are transformed and applied to meshes to simulate reflection or refraction. These pictures may be single images or composite panoramas, and may be static snapshots or dynamically updated to track scene changes.

Once created (as described below), environment maps may be used in StandardMaterial's .reflectionTexture or .refractionTexture; environment maps are also used in Physically Based Rendering for sophisticated material surfaces.

Static environment maps (CubeTexture and friends)

CubeTexture instances use six images to make a static wraparound environment map (or "cubemap"). The CubeMap constructor takes a base URL and appends "_px.jpg", "_nx.jpg", "_py.jpg", "_ny.jpg", "_pz.jpg" and "_nz.jpg" to load images for the +x, -x, +y, -y, +z, and -z facing sides of a cube.

Diagram of X/Y/Z axes and CubeTexture sides

(Use the extensions argument to customize these suffixes, e.g. to load .png instead of .jpg images.)

Despite the "Texture" name, CubeTexture can only be used with the .reflectionTexture or .refractionTexture properties of StandardMaterial, not other properties like .diffuseTexture.

Skybox cubemaps

Skybox images may be used directly as environment maps. (Conveniently, skyboxes also typically use CubeTexture.) In this case only the skybox will show up in the reflection/refraction; other scene members will not be reflected, but that may be acceptable for simple scenes or small surfaces.

This classic cloudy skybox helps demonstrate skybox reflection:

+x (right)-x (left)-y (up)-y (down)+z (back)-z (front)
+x (right)
Some clouds
-x (left)
More clouds
-y (up)
The sun overhead
-y (down)
Solid gray
+z (back)
More clouds
-z (front)
More clouds

Cube Reflecting SkyboxGround Reflecting SkyboxSphere Reflecting Skybox

With two shapes in the scene, it becomes clear that only the skybox is reflected:

Vampire Cube Isn't Reflected

This "vampire effect" can be fixed by carefully adding scene contents to cubemap images, or by using dynamic maps.

Local cubemaps

Environment maps (static or dynamic) are flat images with no depth. By default, they are treated as infinitely far away. This works for distant environments (like skyboxes) or small surfaces, but can cause parallax errors in other cases.

As an alterative, cubemaps can be treated as axis-aligned boxes of specific size and location by setting the cubemap texture's .boundingBoxSize and .boundingBoxPosition to the desired box size and position (as Vector3).

Local cubemap demo

(Local cubemaps must use the default CUBIC_MODE coordinate mode.)

An axis-aligned box is still an approximation of true reflected/refracted scene geometry, but it may be a useful approximation. See "Reflections Based on Local Cubemaps" (from the ARM Developer's Graphics and Gaming Blog) for a good write-up of the concept as used in Unity (Babylon.js's implementation is similar).

EquiRectangularCubeTexture

Equirectangular panoramic images (panotools wiki) squish a spherical panorama into one image (unlike the six sides needed for ordinary CubeTexture) using equirectangular projection (wikipedia).

A warped view of a dock with boats
An equirectangular panorama image, flattened out

EquiRectangularCubeTexture instances load an equirectangular panorama image into a CubeTexture-compatible environment map.

Spheres Reflecting Equirectangular Skybox

HDRCubeTexture

High Dynamic Range (wikipedia) images capture brighter and dimmer colors than a typical monitor can display. This can be useful for environment mapping even if the final output uses standard dynamic range.

A warped view of a room with lamps
A panoramic HDR image, flattened and reduced to SDR

HDRCubeTexture instances (Babylon.js v3.2+) loads a Radiance RGBE format (wikipedia) HDR equirectangular panorama image into a CubeTexture-compatible environment map.

HDR Skybox

Flat textures as environment maps

Wraparound cubemaps work well for environment mapping, but ordinary flat Texture instances can be used for .reflectionTexture and .refractionTexture as well. The texture's .coordinatesMode must be set to define how the flat image should be "wrapped around" as an environment (see "Coordinate Modes").

Mona Lisa's head
Example flat texture (Mona Lisa)

Mona Lisa in SPAAAACE, reflected using PLANAR_MODE

(Note, cubemaps are normally preferred for environment mapping instead of flat textures.)

Dynamic environment maps (RenderTargetTexture and friends)

RenderTargetTexture instances hold environment maps updated during scene rendering, allowing reflections and refractions to track the scene in real time (unlike the static maps described above).

RenderTargetTexture instances are usually created through classes like ReflectionProbe or MirrorTexture, which manage the dynamic update process.

ReflectionProbe

Each ReflectionProbe instance provides a cubemap (in .cubeTexture) that is dynamically rendered from a specified point in the scene, typically at or near an object that will use the cubemap.

Reflection Probes

You must set each ReflectionProbe's .renderList to an explicit list of meshes to render. Be mindful of efficiency as each probe renders six times (once for each cube face) for every update. See the Reflection Probes main page for more details.

MirrorTexture

MirrorTexture (a RenderTargetTexture subclass) acts as a dynamically rendered environment map for flat mirrors.

Mirrors

As with reflection probes, you must set each MirrorTexture's .renderList to an explicit list of meshes to show in the mirror, keeping efficiency in mind. You must also set each MirrorTexture's .mirrorPlane to the Plane of reflection, with the plane's normal pointing into the mirror (away from the viewer).

You may set Plane coordinates directly, but it can be convenient to build the Plane from mesh geometry:

// Create, position, and rotate a flat mesh surface.
const mesh = BABYLON.MeshBuilder.CreatePlane("mirrorMesh", {width: 5, height: 5}, scene);
mesh.position = new BABYLON.Vector3(0, 0, 4);
mesh.rotation = new BABYLON.Vector3(Math.PI/4, Math.PI/6, Math.PI/8);
// Create the reflective material for the mesh.
mesh.material = new BABYLON.StandardMaterial("mirrorMaterial", scene);
mesh.material.reflectionTexture = new BABYLON.MirrorTexture("mirrorTexture", 512, scene, true);
// Get a normal vector from the mesh and invert it to create the mirror plane.
mesh.material.reflectionTexture.mirrorPlane = BABYLON.Plane.FromPositionAndNormal(
mesh.position, mesh.getFacetNormal(0).scale(-1));
mesh.material.reflectionTexture.renderList = [... list of meshes ...];

To create a blurred reflection, set MirrorTexture's .blurKernel to the blur kernel size (higher is blurrier) relative to the render texture size (set when MirrorTexture is created). Use .adaptiveBlurKernel instead to scale the blur value by the ratio of the render texture size and viewport size.

Reflection Blur

RefractionTexture

RefractionTexture (also a RenderTargetTexture subclass) is similar to MirrorTexture, but designed for refraction instead of reflection.

RefractionTexture acts as a dynamically rendered single-image environment map for flat refractors (materials like glass or water that bend light). You can use RefractionTexture instances directly in .refractionTexture. (Note, RefractionTexture simulates fixed-thickness plates of refractive material, and does not support lenses or shapes "bent" by immersion.)

Refraction

As with MirrorTexture, you must set each RefractionTexture's .renderList to an explicit list of meshes to show in the refraction, keeping efficiency in mind.

You must also set each RefractionTexture's .refractionPlane to the Plane of refraction, with the plane's normal pointing out of the refractor (toward the viewer). You may set Plane coordinates directly, but it can be convenient to build the Plane from mesh geometry as described above for MirrorTexture (but without the .scale(-1) call).

Finally, you must set each RefractionTexture's .indexOfRefraction and .depth to the index of refraction (wikipedia) and thickness of the refractive plate to simulate.

Changing Index Of RefractionChanging Refraction Depth

Coordinate modes

By default, cubemap textures (static and dynamic) simulate the physical Laws of Reflection (wikipedia), to create realistic shiny surfaces. However, you may set a texture's .coordinatesMode to select different reflection behavior, and flat (non-cubemap) textures always require you to set .coordinatesMode. (Refraction does not use coordinate modes.)

See the source (reflectionFunction.fx, called by default.fragment.fx) for the mathematical definition of each mode.

coordinatesModeDescription
coordinatesMode
CUBIC_MODE
Description
The default mode for cubemaps. Works best with cubemaps, but will project flat textures onto the X/Y axes of an environment sphere as 4 upside-down copies.
coordinatesMode
INVCUBIC_MODE
Description
Like CUBIC_MODE, but inverts texture Y, so flat texture images are right side up.
coordinatesMode
PLANAR_MODE
Description
Like CUBIC_MODE, but handles flat texture scaling and translation slightly better. (Despite the name, PLANAR_MODE is not otherwise plane-oriented.)
coordinatesMode
SKYBOX_MODE
Description
Rather than reflecting, applies texture images directly to the object surface. Mostly used to make skyboxes. Works best for cubemaps, but will project flat textures onto the X/Y axes of the object.
coordinatesMode
EQUI­RECTANGULAR_MODE
Description
Wraps flat textures into an environment map with equirectangular projection (wikipedia). Not recommended for cubemaps.
coordinatesMode
FIXED_EQUIRECTANGULAR_MODE
Description
Like SKYBOX_MODE, but wraps flat textures around the object with equirectangular projection. Does not support reflection matrix transforms. Not recommended for cubemaps.
coordinatesMode
FIXED_EQUIRECTANGULAR_MIRRORED_MODE
Description
Like FIXED_EQUIRECTANGULAR_MODE, but inverts texture X so images look "correct" on the outside of an object.
coordinatesMode
PROJECTION_MODE
Description
Scales flat textures to the screen size and shows them wherever the material is visible, as if projecting from the camera onto the scene. Mostly used for special effects. Not recommended for cubemaps.
coordinatesMode
SPHERICAL_MODE
Description
Performs reflection in screen coordinates rather than world coordinates. Not useful in most cases.
coordinatesMode
EXPLICIT_MODE
Description
Uses only one point of the texture. Not useful in most cases.

Examples with a test pattern cubemap

+x (right)-x (left)-y (up)-y (down)+z (back)-z (front)
+x (right)
RGT test pattern
-x (left)
LFT test pattern
-y (up)
TOP test pattern
-y (down)
BOT test pattern
+z (back)
BCK test pattern
-z (front)
FRT test pattern

PLANAR, (INV)CUBIC, SKYBOX, and PROJECTION Modes with Test Cubemap

Examples with a test pattern flat texture

Letters A-Y in a grid

PLANAR, EQUIRECTANGULAR, PROJECTION, and SPHERICAL Modes with Flat Test Pattern

The reflection matrix

In addition to selecting the overall coordinate mode, a texture's reflection matrix (.getReflectionTextureMatrix()) may be adjusted to transform the environment map, to match scene changes (e.g. a moving background) or for special effects.

Sphere Reflecting Rotating Skybox

Further reading

Skyboxes
Learn how to add a skybox to your Babylon.js scene.
Skyboxes
Introduction to Physically Based Rendering
Start learning about Physically Based Rendering in Babylon.js.
Introduction to Physically Based Rendering
Reflection Probes
Learn how to use reflection probes in your Babylon.js scene.
Reflection Probes