Generating geometry textures for VAT

Introduction

VAT (Vertex Animation Texture) or BVA (Baked Vertex Animation) is a way to support a large number of animated meshes by precomputing the animations. For more information, see Baked Texture Animations.

VAT is not supported by the geometry buffer renderer. Therefore, if you use it to generate a normal map (or any other geometry texture) with VAT meshes, it will not work as expected.

For example: VAT and geometry buffer renderer

You can see that the spiders are not animated in the normal texture (which is used as the emissive texture on the plane facing the camera).

Let's use a frame graph to make it work!

Using a frame graph

We will use a frame graph to generate the normal texture, instead of using the geometry buffer renderer as in the previous example. Note that the frame graph will not take control of the entire scene rendering loop: we will not set scene.frameGraph to our frame graph, but rather execute it manually.

The frame graph is actually very simple; we only need a geometry renderer task and to configure it to generate the normal texture.

First, we create the frame graph and the depth texture that will be used by the geometry renderer. We also clear the depth texture:

import { Constants } from "@babylonjs/core/Engines/constants";
import { FrameGraph } from "@babylonjs/core/FrameGraph/frameGraph";
import { FrameGraphClearTextureTask } from "@babylonjs/core/FrameGraph/Tasks/Texture/clearTextureTask";
const frameGraph = new FrameGraph(scene);
const depthTexture = frameGraph.textureManager.createRenderTargetTexture("depth", {
    size: { width: 100, height: 100 },
    sizeIsPercentage: true,
    options: {
        createMipMaps: false,
        types: [Constants.TEXTURETYPE_UNSIGNED_BYTE],
        formats: [Constants.TEXTUREFORMAT_DEPTH32_FLOAT],
        samples: 1,
        useSRGBBuffers: [false],
        labels: ["depth"],
    },
});
const clearTask = new FrameGraphClearTextureTask("clear", frameGraph);
clearTask.clearColor = false;
clearTask.clearDepth = true;
clearTask.depthTexture = depthTexture;
frameGraph.addTask(clearTask);

We can now create the geometry renderer task:

import { Constants } from "@babylonjs/core/Engines/constants";
import { FrameGraphGeometryRendererTask } from "@babylonjs/core/FrameGraph/Tasks/Rendering/geometryRendererTask";
const rlist = {
    meshes: [mesh],
    particleSystems: []
};
const geomRendererTask = new FrameGraphGeometryRendererTask("geomRenderer", frameGraph, scene);
geomRendererTask.depthTexture = clearTask.outputDepthTexture;
geomRendererTask.objectList = rlist;
geomRendererTask.camera = camera;
geomRendererTask.samples = 1;
geomRendererTask.textureDescriptions.push({
    type: Constants.PREPASS_NORMAL_TEXTURE_TYPE,
    textureFormat: Constants.TEXTUREFORMAT_RGBA,
    textureType: Constants.TEXTURETYPE_UNSIGNED_BYTE,
});
frameGraph.addTask(geomRendererTask);

Note how we configure the task to generate the normal texture: this is an entry we add to the geomRendererTask.textureDescriptions array of the task.

That's all it takes to create the graph. Now we need to retrieve the texture and set it as the emissive texture of the plane material:

import { Texture } from "@babylonjs/core/Materials/Textures/texture";
let texture = null;
frameGraph.onBuildObservable.add(() => {
    const geomTexture = frameGraph.textureManager.getTextureFromHandle(geomRendererTask.geometryViewNormalTexture);
    geomTexture.incrementReferences();
    texture?.dispose();
    texture = new Texture("", scene, {
        internalTexture: geomTexture
    });
    texture.name = "GeomRenderer normal";
    mat.emissiveTexture = texture;
});

The texture can only be retrieved after the frame graph has been built, so frameGraph.onBuildObservable is the appropriate place to add our code. The code itself retrieves the actual texture (an InternalTexture instance) from the texture handle. Note that we increment the number of references to the texture because we wrap this internal texture in a Texture instance (an InternalTexture instance cannot be assigned to mat.emissiveTexture), and the Texture.dispose method decrements the number of references.

All that remains is to write the code to build the frame graph and execute it:

await frameGraph.buildAsync();

engine.onResizeObservable.add(async () => {
    await frameGraph.buildAsync();
});

scene.onBeforeRenderObservable.add(() => {
    frameGraph.execute();
});

The full PG: VAT and frame graph

You can see that the spiders are now animated in the normal texture.

Using a node render graph

For completeness, here is the same thing using a node render graph:

Node Render Graph: VAT and frame graph (NRGE)

PG: VAT and frame graph (NRG)

Notes:

• We disable autoFillExternalInputs when loading the node's render graph, because the list of meshes used by the geometry renderer task must not contain the plane on which we display the geometry texture (otherwise, you will get an error such as “GL_INVALID_OPERATION: glDrawElements: feedback loop formed between the frame buffer and the active texture").
• We disable the Output block in the render graph to avoid copying the result of the graph execution to the default frame buffer, which is a waste of GPU resources as it will be overwritten by the actual rendering of the scene.