Audio

How To Play Sounds and Music

The Babylon.js sound engine is based on the Web Audio specification. We decided not to offer a fallback to the audio tag or another mechanism. To use our sound engine, you need a Web Audio-compatible browser. Still, if you use it in an incompatible browser, it won’t break the rest of the engine; it will simply play no sound.

The sound engine offers ambient sound, spatialized sound and directional sound. It can be created in code or by loading a .babylon file. It follows the simple & powerful philosophy of the rest of the engine as you’re going to see.

The sound formats supported depend on the browser: .mp3, .ogg, .wav, .m4a, and .mp4. It’s usually at least .mp3 and .wav.

Note: Other formats like .aac can be loaded as well as long as the browser supports them by passing skipCodecCheck to true in the Sound creation options.

Note: all music samples demonstrated in the playground have been composed by David Rousset

Creating an ambient sound or music

Here is the code to create a sound or piece of music playing as ambient audio (not spatialized):

import { Sound } from "@babylonjs/core/Audio/sound";
// Load the sound and play it automatically once ready
const music = new Sound("Music", "music.wav", scene, null, {
    loop: true,
    autoplay: true,
});

1st parameter: the name of your sound.
2nd parameter: URL of the sound to load.
3rd parameter: scene to attach the sound to.
4th parameter: function called back once the sound is ready to be played; we’ll see that later.
5th parameter: a JSON object providing various options we’ll see in detail. But you can already understand the goal of the 2 options provided.

You can test this first sample in our music playground example here:

Simple Music Playback Example

Handling the 'ready to play' callback function

Calling the BABYLON.Sound() constructor with a URL generates 2 phases:

the .wav or .mp3 file is loaded from your webserver using an async XHR
once loaded, the sound is asynchronously decoded by Web Audio. If it succeeds, it raises the callback function you’ve provided.

Here is a sample code:

import { Sound } from "@babylonjs/core/Audio/sound";
const music = new Sound("Music", "music.wav", scene, function () {
    // Sound has been downloaded & decoded
    music.play();
});

This code loads the music.wav file from the web server, decodes it, and plays it in the callback function only once using the play() function. The play function plays the sound immediately if no parameter is passed. You can provide a parameter of type number to play the sound after x seconds.

Test it in this playground: Load and Play Sounds With A Callback

Playing a sound on mouse click or keydown

This sample code plays a gunshot sound if you left-click or press the spacebar:

import { Sound } from "@babylonjs/core/Audio/sound";
const gunshot = new Sound("gunshot", "sounds/gunshot.wav", scene);
window.addEventListener("mousedown", function (evt) {
    // left click to fire
    if (evt.button === 0) {
        gunshot.play();
    }
});
window.addEventListener("keydown", function (evt) {
    // Press space key to fire
    if (evt.keyCode === 32) {
        gunshot.play();
    }
});

Test it on this gunshot sound playground: Playing Sounds With Interaction

Some basic properties

You can set the volume of a sound via the options object or via the setVolume() function. You can set the play rate in the same manner.

You can also be notified when the sound has finished playing by registering a handler on the onended event.

Here is a simple sample code combining all of that:

import { Sound } from "@babylonjs/core/Audio/sound";
const volume = 0.1;
const playbackRate = 0.5;
const gunshot = new Sound("Gunshot", "./gunshot-1.wav", scene, null, {
    playbackRate: playbackRate,
    volume: volume,
});
gunshot.onended = function () {
    if (volume < 1) {
        volume += 0.1;
        gunshot.setVolume(volume);
    }
    playbackRate += 0.1;
    gunshot.setPlaybackRate(playbackRate);
};

The sound is first created with a playbackRate of 0.5 and a volume of 0.1. Every time you play the sound, the onended function will be called at the end and the volume & playbackRate will increase.

Rather than setting the volume on a specific sound, you can also set the global volume of all sounds played by Babylon.js using the setGlobalVolume() function of the audio engine.

import { Engine } from "@babylonjs/core/Engines/engine";
Engine.audioEngine.setGlobalVolume(0.5);

Handling autoplay & unlocking audio on first user interaction

Sometimes you want audio to autoplay on page load, e.g. Ambient background or theme music for a game.

Modern browsers block audio until the user has interacted with the webpage.

Babylon.js supplies a default "unmute" button which unlocks audio once clicked.

We can disable this default unmute button if desired and create our own listener for a user interaction, e.g. any pointer click on the webpage.

import { Engine } from "@babylonjs/core/Engines/engine";
// Disable the default audio unlock button
Engine.audioEngine.useCustomUnlockedButton = true;
// Unlock audio on first user interaction.
window.addEventListener("click", () => {
    if (!Engine.audioEngine.unlocked) {
        Engine.audioEngine.unlock();
    }
}, { once: true });

Test it in our playground here: Custom audio unlock & autoplay

Playing a sound sprite

A sound sprite is a portion of a sound file. You can define a sound sprite when creating a sound by defining an offset and a length (in seconds):

import { Sound } from "@babylonjs/core/Audio/sound";
const soundSprite = new Sound("Violons", "/sounds/6sounds.mp3", scene, null, { loop: true, autoplay: true, length: 9.2, offset: 14.0 });

You can find examples here: Playing Sound Sprites 1

and here: Playing Sound Sprites 2

Playing several sounds simultaneously and synchronized

For that, you need to call the play method on all sounds only once you’re sure they are all ready to be played. You then need to work with the ready to play callback.

import { Sound } from "@babylonjs/core/Audio/sound";
const music1 = new Sound("Violons11", "sounds/violons11.wav", scene, soundReady, { loop: true });
const music2 = new Sound("Violons18", "sounds/violons18.wav", scene, soundReady, { loop: true });
const music3 = new Sound("Cellolong", "sounds/cellolong.wav", scene, soundReady, { loop: true });
const soundsReady = 0;
function soundReady() {
    soundsReady++;
    if (soundsReady === 3) {
        music1.play();
        music2.play();
        music3.play();
    }
}

Test it in our playground here: Playing Sounds Together

Loading a sound from an ArrayBuffer

You can bypass the first phase (the embedded XHR request) if you’re calling the constructor with your own provided ArrayBuffer.

Here is a sample code demonstrating it:

import { Sound } from "@babylonjs/core/Audio/sound";
const gunshotFromAB;
loadArrayBufferFromURL("sounds/gunshot.wav");
function loadArrayBufferFromURL(urlToSound) {
    const request = new XMLHttpRequest();
    request.open("GET", urlToSound, true);
    request.responseType = "arraybuffer";
    request.onreadystatechange = function () {
        if (request.readyState == 4) {
            if (request.status == 200) {
                gunshotFromAB = new Sound("FromArrayBuffer", request.response, scene, soundReadyToBePlayed);
            }
        }
    };
    request.send(null);
}
function soundReadyToBePlayed() {
    gunshotFromAB.play();
}

Test it out in our playground: Loading A Sound From an ArrayBuffer

Here is another example where we load the sound from the audio file: Loading A Sound From the Microphone

Loading a sound using the Assets Manager

The assets manager is pretty useful as it handles for you some great features such as a loading screen.

import { Sound } from "@babylonjs/core/Audio/sound";
import { AssetsManager } from "@babylonjs/core/Misc/assetsManager";
let music1, music2, music3;
// Assets manager
const assetsManager = new AssetsManager(scene);
const binaryTask = assetsManager.addBinaryFileTask("Violons18 task", "sounds/violons18.wav");
binaryTask.onSuccess = function (task) {
    music1 = new Sound("Violons18", task.data, scene, soundReady, {
        loop: true,
    });
};
const binaryTask2 = assetsManager.addBinaryFileTask("Violons11 task", "sounds/violons11.wav");
binaryTask2.onSuccess = function (task) {
    music2 = new Sound("Violons11", task.data, scene, soundReady, {
        loop: true,
    });
};
const binaryTask3 = assetsManager.addBinaryFileTask("Cello task", "sounds/cellolong.wav");
binaryTask3.onSuccess = function (task) {
    music3 = new Sound("Cello", task.data, scene, soundReady, {
        loop: true,
    });
};
const soundsReady = 0;
function soundReady() {
    soundsReady++;
    if (soundsReady === 3) {
        music1.play();
        music2.play();
        music3.play();
    }
}
assetsManager.load();

Test it out in our playground: Loading A Sound with the Asset Manager

Creating a spatial 3D sound

To transform a sound into a spatial sound, you need to specify that via the options:

import { Sound } from "@babylonjs/core/Audio/sound";
const music = new Sound("music", "music.wav", scene, null, {
    loop: true,
    autoplay: true,
    spatialSound: true,
});

Default properties of a spatial sound are:

distanceModel (the attenuation) is using a “linear” equation by default. Other options are “inverse” or “exponential”.
maxDistance is set to 100. This means that once the listener is farther than 100 units from the sound, the volume will be 0. You can’t hear the sound anymore
panningModel is set to “equalpower”; the equal-power panning algorithm is generally regarded as simple and efficient. The other available option is “HRTF”. The specification says it’s: “a higher quality spatialization algorithm using a convolution with measured impulse responses from human subjects. This panning method renders stereo output”. This is the best algorithm when using a headphone.

maxDistance is only used when using the “linear” attenuation. Otherwise, you can tune the attenuation of the other models using the rolloffFactor and refDistance options. Both are set to 1 by default but you can change it of course.

For instance:

import { Sound } from "@babylonjs/core/Audio/sound";
const music = new Sound("music", "music.wav", scene, null, {
    loop: true,
    autoplay: true,
    spatialSound: true,
    distanceModel: "exponential",
    rolloffFactor: 2,
});

Default position of sound in the 3D world is (0,0,0). To change that, use the setPosition() function:

import { Vector3 } from "@babylonjs/core/Maths/math.vector";
music.setPosition(new Vector3(100, 0, 0));

To have a better understanding, please have a look at this sample in our playground: 3D Spatial Sound

Move into the scene using keyboard & mouse. Each sound is represented by a purple sphere. When you enter a sphere, you’ll start hearing one of the music tracks. The sound is louder at the center of the sphere and falls to 0 when you leave the sphere.

Attaching a sound to a mesh

This is probably the simplest way to handle 3D sounds in your scene. Simply create a BABYLON.Sound, attach it to an existing mesh and you’re done! If the mesh is moving, the sound will move with it. You have nothing to do.

Here’s the code to use:

import { Sound } from "@babylonjs/core/Audio/sound";
const music = new Sound("Violons", "sounds/violons11.wav", scene, null, {
    loop: true,
    autoplay: true,
});
// Sound will now follow the box mesh position
music.attachToMesh(box);

Calling the attachToMesh() function on a sound will transform it automatically into a spatial 3D sound. Using the above code, you’ll use the default Babylon.js values: a linear attenuation with a maxDistance of 100 and a panning model of type HRTF.

Put your headphones on and launch this sample in our playground: Sound Attached to a Mesh

Setting a position as the audio listener

By default, the "ears" of your scene - or the listener - is always the currently active camera. Sometimes, for example when making a 3rd person game, you may need to set another mesh as the listener - a character's head for example. This can be achieved by setting the audioListenerPositionProvider property on your scene.

The method you create has to return a valid Vector3 object.

import { Vector3 } from "@babylonjs/core/Maths/math.vector";
// Returns a static position
scene.audioListenerPositionProvider = () => {
    return new Vector3(0, 0, 10);
};
// Returns the current position of a mesh
// !It is recommended to use the 'absolutePosition' property
// to reflect the position of the mesh in the world
scene.audioListenerPositionProvider = () => {
    // Returns a static position
    return myMesh.absolutePosition;
};

To switch back to using your camera as the listener, simply set the property to null.

Creating a spatial directional 3D sound

By default, spatial sounds are omnidirectional. But you can have directional sounds if you’d like to.

Note: directional sounds only work for spatial sounds attached to a mesh.

Here is the code to use:

import { Sound } from "@babylonjs/core/Audio/sound";
import { Vector3 } from "@babylonjs/core/Maths/math.vector";
const music = new Sound("Violons", "violons11.wav", scene, null, {
    loop: true,
    autoplay: true,
});
music.setDirectionalCone(90, 180, 0);
music.setLocalDirectionToMesh(new Vector3(1, 0, 0));
music.attachToMesh(box);

setDirectionalCone takes 3 parameters:

coneInnerAngle: size of the inner cone in degrees
coneOuterAngle: size of the outer cone in degrees
coneOuterGain: volume of the sound when you’re outside the outer cone (between 0.0 and 1.0)

Outer angle of the cone must be superior or equal to the inner angle, otherwise an error will be logged and the directional sound won’t work.

setLocalDirectionToMesh() simply defines the orientation of the cone relative to the mesh the sound is attached to. By default, it’s (1,0,0).

You can play with this sample from our playground to better understand the output: Spatial Directional Sound

Move into the 3D scene. If you’re inside the space defined by the grey cone, you should hear the music, if not you won’t hear it as the coneOuterGain is set to 0.

Creating your own custom attenuation function

If you want to manage the attenuation (or distance model in Web Audio) using a specific algorithm, you can bypass the native Web Audio attenuation using Babylon.js custom attenuation function.

Note: Web Audio is hardware accelerated. It means it’s mainly handled by a dedicated audio chip on your device via native code (the browser). This then almost costs nothing on the performance side for 3D real-time rendering. Switching to custom attenuation will use Babylon.js distance computation based on JavaScript and will be slower.

Moreover, the custom attenuation will only work on spatial sounds and on sounds connected to a Babylon.js mesh. That said, let’s now view the code to do that. First, you must specify it in the options:

import { Sound } from "@babylonjs/core/Audio/sound";
// Create and load the sound async
const music = new Sound("Music", "music.wav", scene, null, {
    loop: true,
    autoplay: true,
    useCustomAttenuation: true,
});

You’ll switch to internal Babylon.js math computations. The default custom attenuation function is a linear one.

To create your own logic, you need such code:

// Creating custom attenuation function. Near the object, volume is almost 0.
// The farest, the louder
music.setAttenuationFunction(function (currentVolume, currentDistance, maxDistance, refDistance, rolloffFactor) {
  return (currentVolume * currentDistance) / maxDistance;
});

You can play with these 5 parameters and do whatever you’d like with it. Simply return a number that will be the volume applied to the sound.

In this example, the logic is a bit unusual because the volume gets louder the farther you are from the mesh. ;-)

Play with it in our Playground: Custom Attenuation Function

Moreover, Firefox currently has a bug in its Web Audio implementation when handling linear attenuation properly. This can be fixed by using the Babylon.js default linear custom attenuation.

Manipulating sound loaded from a .babylon file

Currently only our 3DS Max exporter can export sounds directly to .babylon.

To access a sound loaded by the Babylon.js .babylon file loader, you need to use the getSoundByName() function on the scene object.

Here is a simple sample loading a .babylon scene file embedding some sounds:

import { Engine } from "@babylonjs/core/Engines/engine";
import { LoadSceneAsync } from "@babylonjs/core/Loading/sceneLoader";
const canvas = document.getElementById("renderCanvas");
const engine = new Engine(canvas, true);
LoadSceneAsync("TestScene/testsound.babylon", engine).then(function (newScene) {
    newScene.executeWhenReady(function () {
        newScene.activeCamera.attachControl(canvas);
        const gunshotSound = newScene.getSoundByName("gunshot-1.wav");
        window.addEventListener("keydown", function (evt) {
            if (evt.keyCode === 32 && gunshotSound) {
                gunshotSound.play();
            }
        });
        engine.runRenderLoop(function () {
            newScene.render();
        });
    });
});

Pressing the spacebar will play the gunshot sound.

Using Sound Tracks

It could be useful to isolate your music & sounds on several tracks to better manage volume for a grouped set of sounds. It will also be used in a future release to apply effects on a specific track.

By default, Babylon.js is creating a BABYLON.SoundTrack object to act as its main track. Every time you create a new BABYLON.Sound, it is added to this main track for you.

import { SoundTrack } from "@babylonjs/core/Audio/soundTrack";
const soundTrack = new SoundTrack(scene);
soundTrack.addSound(cellolong);
soundTrack.addSound(violons11);

Using this code, the cellolong and violons11 sounds will be moved from the main Babylon.js track to this specific sound track. This means that you can change the volume of this track, and thus of these 2 sounds, independently from the main track.

The addSound() function will move the sound from its original container (the main track or a specific track) to the new sound track specified. For instance, with this code:

import { SoundTrack } from "@babylonjs/core/Audio/soundTrack";
const soundTrack1 = new SoundTrack(scene);
soundTrack1.addSound(cellolong);
soundTrack1.addSound(violons11);
const soundTrack2 = new SoundTrack(scene);
soundTrack2.addSound(violons11);

The violons11 sound will finally live only in soundTrack2.

Using the Analyser

You can easily analyze in real-time the audio frequencies.

The easiest code to understand how it works is this one:

import { Analyser } from "@babylonjs/core/Audio/analyser";
import { Engine } from "@babylonjs/core/Engines/engine";
const myAnalyser = new Analyser(scene);
Engine.audioEngine.connectToAnalyser(myAnalyser);
myAnalyser.drawDebugCanvas();

This will connect to the global volume of the audio engine and will draw the frequencies of all sounds played together into a 2D canvas display on top of the screen.

You can change the position and size of the debug canvas and use an analyser on a sound track instead of the global audio engine:

import { Analyser } from "@babylonjs/core/Audio/analyser";
const myAnalyser = new Analyser(scene);
soundTrack1.connectToAnalyser(myAnalyser);
myAnalyser.DEBUGCANVASSIZE.width = 160;
myAnalyser.DEBUGCANVASSIZE.height = 100;
myAnalyser.DEBUGCANVASPOS.x = 40;
myAnalyser.DEBUGCANVASPOS.y = 30;
myAnalyser.drawDebugCanvas();

You can also call the analyser functions yourself to create your own usage of it.

Here is a full audio playground example to play with: Full Audio Example