Playing Sounds and Music

The Babylon.js audio engine is based on the Web Audio specification. It features ambient, spatialized and directional sounds, and audio buses for signal routing and mixing.

The audio engine is simple and powerful, and its API is similar to the Babylon.js graphics APIs, which makes it easy to learn if you are already familiar with the Babylon.js framework.

The sound formats supported by the audio engine are dictated by the browser. All browsers support the .mp3 and .wav formats, and most browsers support .ogg, .m4a, and .mp4. Other formats like .aac and .webm are browser-specific. When creating sounds, you can specify an array of sound file URLs to choose from and the first format recognized by the browser will be used. See Using browser-specific audio codecs.

Creating an audio engine

Before any sounds can be played, the audio engine must be created and initialized with the CreateAudioEngineAsync function. The available options for creating an audio engine are listed in the IWebAudioEngineOptions interface documentation.

The CreateAudioEngineAsync function is asynchronous, which means it returns a Promise that will resolve when the audio engine is ready.

You must wait for the Promise to resolve before using the audio engine, which can be done with the await keyword. The following code snippet shows how to do this:

import { CreateAudioEngineAsync } from "@babylonjs/core/AudioV2/webAudio/webAudioEngine";
(async () => {
    const audioEngine = await CreateAudioEngineAsync();
    // Create sounds here, but don't call `play()` on them, yet ...
    // Wait until audio engine is ready to play sounds.
    await audioEngine.unlockAsync();
    // Start sound playback ...
})();

Note that the example code snippet creates an async function and calls it immediately, and that it uses await to wait for the audio engine to be "unlocked" since browsers prevent audio from playing until a user interaction occurs. See Browser autoplay considerations for more information.

iOS and iPadOS ringer switch behavior

By default, the audio engine uses a silent HTML audio element while Babylon.js sounds are active. This workaround allows Web Audio to remain audible on iOS and iPadOS when the device's ringer switch is off. On some affected devices, keeping the iOS media playback session active can reduce rendering performance.

Applications that prioritize rendering performance can disable the workaround by setting disableIOSRingerSwitchWorkaround to true when creating the audio engine:

import { CreateAudioEngineAsync } from "@babylonjs/core/AudioV2/webAudio/webAudioEngine";
const audioEngine = await CreateAudioEngineAsync({
    disableIOSRingerSwitchWorkaround: true,
});

Disabling the workaround prevents the silent HTML audio element from being created, but Web Audio may be muted while the device's ringer switch is off. The option defaults to false, so existing applications continue to use the workaround unless they explicitly disable it.

Playing a sound

The simplest way to play a sound is to create it with the CreateSoundAsync function, and call the sound's play() function after the audio engine is unlocked:

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
import { CreateAudioEngineAsync } from "@babylonjs/core/AudioV2/webAudio/webAudioEngine";
const audioEngine = await CreateAudioEngineAsync();
const gunshot = await CreateSoundAsync("gunshot", "sounds/gunshot.wav");
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
gunshot.play();

Streaming a sound

To stream sounds, use the CreateStreamingSoundAsync function, which plays sounds using the browser's HTMLMediaElement feature.

Streaming sounds keep only a small chunk of the sound file in memory while playing instead of downloading the entire sound into an audio buffer beforehand. As a result, streaming sounds save a significant amount of memory when playing long sound files, and are useful for background music and extended narrations.

The main disadvantage of streaming sounds is that they have fewer playback options than non-streaming static sounds. For example, streaming sounds can not be played for durations shorter than the sound file, they do not have the loopStart and loopEnd options, and initial playback may be delayed while the playback buffer is being filled (note that this can be avoided using the preloadCount option).

Here is an example of playing a streaming sound:

import { CreateStreamingSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const narration = await CreateStreamingSoundAsync("narration", "https://assets.babylonjs.com/sound/testing/60-count.mp3");
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
narration.play();

Sound instances

When a sound's play function is called, it creates a sound "instance" to perform the audio playback using the sound's current settings. Calling play multiple times causes multiple instances to be created, allowing individual sounds to be played simultaneously and overlapped.

You can limit the number of instances that get created using a sound's maxInstances setting. When the maxInstances option or property is exceeded, old instances are stopped and deactivated until the number of active instances is reduced to the given maximum. For example, if maxInstances is set to 2, calling play three times in a row makes the first instance stop automatically so the third instance can play without the maxInstances setting being exceeded.

Some sound functions and settings affect all of the currently playing instances, like the stop function and the volume property. However, the currentTime property only affects the newest playback instance. Other properties only affect new playback instances created by the sound's play function after the properties have been set. For example, changing a sound's loop property from false to true will not make any of the currently playing instances start looping. Only new playback instances will take the updated loop property's value.

Here's an example of playing a sound 3 times with the maxInstances option set to 2. Notice that the first playback instance is stopped when the third instance plays. Also notice that the changes to the playbackRate and pitch properties have no effect on currently playing instances. Only new instances are affected when those properties are changed.

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const sound = await CreateSoundAsync("sound", "https://assets.babylonjs.com/sound/alarm-1.mp3", { maxInstances: 2 });
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
sound.play(); // Instance #1.
setTimeout(() => {
    sound.playbackRate = 2;
    sound.play(); // Instance #2.
    sound.pitch = 100;
    sound.play(); // Instance #3.
}, 4000);

Looping playback

Sounds stop playing automatically when playback reaches the end of the sound file. To make sounds restart from the beginning when they reach the end, set the sound's loop setting using any of the following methods:

1. Set the loop option to true when creating the sound.
2. Set the sound's loop property to true after creating the sound, but before calling the play() function.
3. Call the sound's play function with the loop option set to true.

For example, the following code snippets all loop the sound indefinitely:

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const bounce = await CreateSoundAsync("bounce", "sounds/bounce.wav", { loop: true });
bounce.play();

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const bounce = await CreateSoundAsync("bounce", "sounds/bounce.wav");
bounce.loop = true;
bounce.play();

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const bounce = await CreateSoundAsync("bounce", "sounds/bounce.wav");
bounce.play({ loop: true });

Volume

The sound, bus, and audio engine volume settings adjust sound loudness, with 0 to 1 being the normal range from silent to 100%. Values above 1 boost the sound's signal proportionately.

Volume options and properties are available on sounds and buses, which affect all sounds using the bus, and on the audio engine, which affects all sounds and buses associated with that audio engine.

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
import { CreateAudioEngineAsync } from "@babylonjs/core/AudioV2/webAudio/webAudioEngine";
const audioEngine = await CreateAudioEngineAsync();
audioEngine.volume = 0.5;
const gunshot = await CreateSoundAsync("gunshot", "sounds/gunshot.wav");
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
gunshot.volume = 0.75;
gunshot.play();

The volume can also fade in and out over time using AbstractSound.setVolume, which sets the volume to the given value with optional duration and shape options.

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
import { CreateAudioEngineAsync } from "@babylonjs/core/AudioV2/webAudio/webAudioEngine";
const audioEngine = await CreateAudioEngineAsync();
audioEngine.volume = 0.5;
const tone = await CreateSoundAsync("tone", "https://amf-ms.github.io/AudioAssets/samples/tones/sine-wave-440.wav");
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
tone.volume = 0.2;
tone.play({ loop: true });
// Start a 3 second long fade to zero using a logarithmic shape.
tone.setVolume(0, { duration: 3, shape: AudioParameterRampShape.Logarithmic });

See the AudioParameterRampShape enum for the list of available ramp shapes.

To experiment with fading sound in and out using different ramp shapes, see this example playground:

Stereo pan

The sound and bus stereo.pan setting moves a sound between the left and right speakers. The stereo property is available on sounds and buses, but not main buses. Values can range from -1 to 1. A value of -1 moves sound output to the left speaker, and a value of 1 moves sound output to the right speaker.

To create a sound with the stereo.pan option set, use the stereoPan option.

The following example plays a sound effect in the left speaker using the stereo.pan property:

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
import { CreateAudioEngineAsync } from "@babylonjs/core/AudioV2/webAudio/webAudioEngine";
const audioEngine = await CreateAudioEngineAsync();
audioEngine.volume = 0.5;
const gunshot = await CreateSoundAsync("gunshot", "sounds/gunshot.wav", { stereoEnabled: true });
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
gunshot.stereo.pan = -1;
gunshot.play();

Note that this example creates the sound with the stereoEnabled option set to true. This is done because the underlying stereo property is not enabled by default, so a small delay occurs when enabling it for the first time. Setting the stereoEnabled option to true avoids this delay, as does setting the stereoPan option when the sound is created instead of using the stereo property later. Subsequent changes to the stereo settings are instantaneous after the stereo property has been enabled.

Spatial audio

"Spatial audio" refers to sounds placed in a three-dimensional space. It requires a "listener" to hear the audio, and a "source" to emit the audio. Listeners and sources both have a 3D position and direction, and sources have additional settings for specifying how the sound should propagate in 3D space.

There is one spatial audio listener per audio engine. It can be accessed through the audio engine's listener property.

Sounds and buses expose their spatial settings through their spatial property. See the AbstractSpatialAudio documentation for details on the available spatial audio settings for sound sources.

Attaching meshes

The easiest way to control the position and direction of a spatial sound source or listener is to attach it to a mesh or other graphics object. This can be done with the spatial.attach function, like in the following example:

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const bounce = await CreateSoundAsync("bounce", "sounds/bounce.wav", { spatialEnabled: true });
bounce.spatial.attach(mesh);
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
bounce.play({ loop: true });

Note that this example creates the sound with the spatialEnabled option set to true. This is done because the underlying spatial property is not enabled by default, so a small delay occurs when enabling it for the first time. Setting the spatialEnabled option to true avoids this delay, as does setting any of the spatial audio options when the sound is created.

To experiment with the available spatial sound source settings, a spatial audio visualizer playground is provided:

Audio buses

Audio buses combine the audio output of multiple sound sources into a single audio output. They make it possible to group sound output together, which makes it easier to manage large audio routing graphs and can be used to reduce the amount of CPU used for effects like spatial audio.

Sounds and buses send their audio output to the bus specified by the outBus property, which can be set to a MainAudioBus or a full-featured AudioBus.

Main audio buses

A MainAudioBus is the last bus that audio passes through before reaching the speakers. It always sends its output to the speakers. It cannot send its output to another bus like intermediate audio buses can.

A default MainAudioBus is created automatically by the audio engine, and all sounds and intermediate audio buses have their outBus property set to it by default.

Main buses are simpler than normal buses. They cannot be used as spatial audio sources and they do not support stereo pan. These features are only supported by sounds and intermediate audio buses.

Intermediate audio buses

Sounds send their audio output to a main bus by default, but the outBus can be changed to an intermediate AudioBus instead if needed. This allows multiple sounds to be sent to a single AudioBus that supports stereo pan and can be used as a spatial audio source.

Intermediate AudioBus output can be sent to other intermediate audio buses as long as no cyclic audio routing loops are created. An error will be thrown if a cyclic routing loop is detected when setting the AudioBus.outBus property.

Note that intermediate audio buses cannot be the last bus in the audio routing graph. Only a MainAudioBus can send audio to the speakers.

Here is a simple example of using an intermediate AudioBus:

import { CreateAudioBusAsync, CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const bus = await CreateAudioBusAsync("bus", { spatialEnabled: true });
bus.spatial.attach(mesh);
const bounce = await CreateSoundAsync("bounce", "sounds/bounce.wav");
bounce.outBus = bus;
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
bounce.play({ loop: true });

Analyzer

The sound and bus analyzer property provides real-time analysis of the audio output's frequencies. The analyzer.getFloatFrequencyData() and analyzer.getByteFrequencyData() functions return an array of volumes, in decibels, corresponding to the frequencies analyzed. Calling these functions quickly and repeatedly over time is useful for frequency-based visualizations, like in the following example:

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const bounce = await CreateSoundAsync("bounce", "sounds/bounce.wav", { analyzerEnabled: true });
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
bounce.play({ loop: true });
// Get the audio analyzer frequency data on every frame.
scene.onBeforeRenderObservable.add(() => {
    const frequencies = bounce.analyzer.getByteFrequencyData();
    for (let i = 0; i < 16; i++) {
        columns[i].value = frequencies[i] / 255;
    }
});

Note that this example creates the sound with the analyzerEnabled option set to true. This is done because the underlying analyzer property is not enabled by default, so a small delay occurs when enabling it for the first time. Setting the analyzerEnabled option to true avoids this delay, as does setting any of the analyzer options when the sound is created.

Here is an example playground that uses the analyzer to animate a frequency-based shader.

Sound buffers

When a sound is created with the CreateSoundAsync function using a URL for the source parameter, the sound is downloaded and decoded into an in-memory data structure called a sound buffer which can be reused when creating other sounds referencing the same source URL. This lets you avoid downloading and decoding the same audio data multiple times.

Here is an example of reusing a sound buffer with the StaticSound.buffer property:

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const bounce1 = await CreateSoundAsync("bounce1", "sounds/bounce.wav");
const bounce2 = await CreateSoundAsync("bounce2", bounce1.buffer, { playbackRate: 2 });
// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();
bounce1.play();
bounce2.play();

Note that the first sound uses a URL for its source parameter, and the second sound uses the first sound's buffer for its source parameter.

Sound buffers can also be created using the CreateSoundBufferAsync function, like in this example:

import { CreateSoundAsync, CreateSoundBufferAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const soundBuffer = await CreateSoundBufferAsync(
    "sounds/bounce.wav"
);

const bounce1 = await CreateSoundAsync("bounce1",
    soundBuffer
);

const bounce2 = await CreateSoundAsync("bounce2",
    soundBuffer
    { playbackRate: 2 }
);

// Wait until audio engine is ready to play sounds.
await audioEngine.unlockAsync();

bounce1.play();
bounce2.play();

Using browser-specific audio codecs

Some sound file types like .ac3 and .ogg are not recognized by all browsers. The .ac3 format for example, is recognized by Apple's Safari browser and is the most efficient on Apple platforms, but it is not recognized by other browsers. Conversely, the .ogg format is also efficient but it is not recognized by Safari. Other formats like .mp3 and .wav are lower quality or less efficient but are recognized by all browsers.

To address these differences you can limit the formats used to .mp3 and .wav to maximize compatibility at the expense of quality and efficiency, or you can use an array of sound file URLs when creating sounds and sound buffers to maximize efficiency while retaining quality. The first file in the URL array that is recognized by the browser will be used. This allows you to specify different file formats for different platforms.

For example, if the following array of URLs is used for the CreateSoundAsync source parameter, Safari will see the .ac3 file first and use it, but other browsers will ignore the .ac3 file and skip to the .ogg file, instead. If a browser does not support .ac3 or .ogg, then it will fall through to the .mp3 file that is last in the array.

import { CreateSoundAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
const sound = CreateSoundAsync("sound", [
    "https://assets.babylonjs.com/sound/testing/ac3.ac3",
    "https://assets.babylonjs.com/sound/testing/ogg.ogg",
    "https://assets.babylonjs.com/sound/testing/mp3.mp3",
]);

If needed, this behavior can be disabled by setting the skipCodecCheck option to false when creating a sound or sound buffer.

Microphone input

To capture microphone input, use the CreateMicrophoneSoundSourceAsync function.

const micInput = await CreateMicrophoneSoundSourceAsync("micInput");

Microphone input starts automatically and can be muted by setting its volume to 0, but it will continue streaming until disposed, even when it is muted.

Caution should be used when setting a microphone sound source's outBus property due to the potential for a loud feedback loop from the speakers back into the microphone. To help prevent this, microphone sound sources do not automatically set the outBus property. It can be set after the microphone input is created, but be careful not to route the microphone output to speakers that will be picked up by the microphone and looped back into the speakers.

Note that the browser may show a microphone access request when this function is called, in which case the promise returned by this function will resolve when the user responds to the request. An error is thrown if the request is denied.

WebAudio node sound sources

To create a sound source from a WebAudio node, use the CreateSoundSourceAsync function.

import { CreateSoundSourceAsync } from "@babylonjs/core/AudioV2/abstractAudio/audioEngineV2";
import { CreateAudioEngineAsync } from "@babylonjs/core/AudioV2/webAudio/webAudioEngine";
const audioContext = new AudioContext();
const audioEngine = await CreateAudioEngineAsync({ audioContext });
const audioNode = new OscillatorNode(audioContext, {
    frequency: 440,
    type: "sine",
});
audioNode.start();
await CreateSoundSourceAsync("sine", audioNode, { volume: 0.1 });

A sound source created for a WebAudio node starts automatically and can be muted by setting its volume to 0, but it will continue streaming until disposed, even when it is muted.

Browser autoplay considerations

When working with audio in the browser, autoplay must be considered. Autoplay is generally disabled by default in most browsers and as a result, the Babylon audio engine cannot play sounds until a user interaction has occurred on the page.

By default, the Babylon audio engine shows an unmute button in the top left corner of the render area that when pressed, unlocks the audio engine and lets sounds play. This button can be disabled when creating the audio engine by setting the disableDefaultUI options to false.

The audio engine also adds a click handler to the entire document by default, which unlocks the audio engine on any user interaction, not just the unmute button. This can be disabled by setting the resumeOnInteraction option to false.

In some rare cases, the browser automatically locks the audio engine after it has been unlocked, like when VisionOS enters WebXR immersive mode. By default, the audio engine will attempt to unlock the audio engine automatically when this occurs. This should work in all cases, but it can be disabled by setting the resumeOnPause option to false if needed.

Playing sounds after the audio engine is unlocked

If a sound has its play function called while the audio engine is locked, the sound will not play. If a sound is set to loop and its autoplay setting is turned on or its play function is called while the audio engine is locked, then the sound will start playing automatically when the audio engine is unlocked. If the sound is not set to loop, and its play function is called while the audio engine is locked, then it will not be automatically started when the audio engine is unlocked, even if its autoplay setting is turned on.

Unmute button

The Babylon audio engine's unmute button unlocks the audio engine. It is shown automatically when the audio engine is locked, but it can be disabled when creating the audio engine by setting the disableDefaultUI options to false.

The unmute button's CSS styling can be modified using the #babylonUnmuteButton id, like the following example that moves the button to the bottom left and makes it twice as big:

import { CreateAudioEngineAsync } from "@babylonjs/core/AudioV2/webAudio/webAudioEngine";
const audioEngine = await CreateAudioEngineAsync();
const unmuteButtonStyle = document.createElement("style");
unmuteButtonStyle.appendChild(document.createTextNode(`
        #babylonUnmuteButton {
            top: unset;
            bottom: 35px;
            margin-left: 50px;
            transform: scale(2);
        }
        #babylonUnmuteButton:hover {
            transform: scale(2.5);
        }
    `));
document.head.appendChild(unmuteButtonStyle);

Feature requests and bug fixes

To request new features or bug fixes, create a new topic at https://forum.babylonjs.com and tag @docEdub. Thanks!

Further reading

• Sound
• Streaming Sound
• Audio Buses