
## How to use the navigation mesh?

There are many use cases for a navigation mesh: AI and pathfinding, replacing physics for collision detection (allowing the player to go only where possible instead of using collision detection), and many more that Babylon.js users will find.

First, create the navigation plugin.

```javascript
let navigationPlugin = new BABYLON.RecastJSPlugin();
```

Prepare some parameters for the agent constraints (described below).

```javascript
const parameters = {
  cs: 0.2,
  ch: 0.2,
  walkableSlopeAngle: 35,
  walkableHeight: 1,
  walkableClimb: 1,
  walkableRadius: 1,
  maxEdgeLen: 12,
  maxSimplificationError: 1.3,
  minRegionArea: 8,
  mergeRegionArea: 20,
  maxVertsPerPoly: 6,
  detailSampleDist: 6,
  detailSampleMaxError: 1,
};
```

Call navigation mesh generation with the parameters and the list of meshes.

```javascript
navigationPlugin.createNavMesh([groundMesh, wallMesh1, wallMesh2, stair1, stair2], parameters);
```

And that's it! You can now use the navigation mesh with the crowd system or make queries.

Optionally, you can display the navmesh to ensure it corresponds to your space constraints.

```javascript
navmeshdebug = navigationPlugin.createDebugNavMesh(scene);
const matdebug = new BABYLON.StandardMaterial("matdebug", scene);
matdebug.diffuseColor = new BABYLON.Color3(0.1, 0.2, 1);
matdebug.alpha = 0.2;
navmeshdebug.material = matdebug;
```

<Playground id="#KVQP83#0" title="Simple navigation mesh computation" description="Simple navigation mesh computation"/>

## Parameters

| Parameter                | Description                                                                                                                             | Units       |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
| `cs`                     | The meshes are voxelized in order to compute a walkable navmesh. Defines the width and depth of 1 voxel.                                | World units |
| `ch`                     | Same as `cs`, but for the height of the voxel.                                                                                          | World units |
| `walkableSlopeAngle`     | Angle in degrees for the maximum walkable slope.                                                                                        | Degrees     |
| `walkableHeight`         | The height in voxel units that is allowed to walk in.                                                                                   | Voxel units |
| `walkableClimb`          | The delta in voxel units that can be climbed.                                                                                           | Voxel units |
| `walkableRadius`         | The radius in voxel units of the agents.                                                                                                | Voxel units |
| `maxEdgeLen`             | The maximum allowed length for contour edges along the border of the mesh.                                                              | Voxel units |
| `maxSimplificationError` | The maximum distance a simplified contour’s border edges should deviate from the original raw contour.                                  | Voxel units |
| `minRegionArea`          | The minimum number of cells allowed to form isolated island areas.                                                                      | Voxel units |
| `mergeRegionArea`        | Any regions with a span count smaller than this value will, if possible, be merged with larger regions.                                 | Voxel units |
| `maxVertsPerPoly`        | The maximum number of vertices allowed for polygons generated during the contour-to-polygon conversion process. Must be greater than 3. | Count       |
| `detailSampleDist`       | Sets the sampling distance to use when generating the detail mesh.                                                                      | World units |
| `detailSampleMaxError`   | The maximum distance the detail mesh surface should deviate from heightfield data.                                                      | World units |

You can find more information about those parameters on the [Recast documentation](https://recastnav.com/structrcConfig.html).

## Queries

Basically, query functions help get constrained points and vectors from the navigation mesh.

```javascript
getClosestPoint(position: Vector3): Vector3;
getRandomPointAround(position: Vector3, maxRadius: number): Vector3;
moveAlong(position: Vector3, destination: Vector3): Vector3;
```

Respectively:

- get a point on the navmesh close to a given world position.
- get a random world position on the navmesh inside a circle of `maxRadius`.
- constrain a segment by the navmesh and return the ending world position, like walking on the navmesh and stopping at the edge.

When the query can't find a valid solution, the value (0,0,0) is returned.

Those functions use a bounding box for querying the world. The solution returned is within that bound. To properly set the default box extent to get a finer or broader result, call:

```javascript
setDefaultQueryExtent(extent: Vector3): void;
```

If your query returns a point too far from the expected result, use a smaller extent.

It is possible to get a path built for navigation as a point array. It is up to the user to use this array for drawing a predicted path, triggering events, and more.

```javascript
const pathPoints = navigationPlugin.computePath(crowd.getAgentPosition(agent), navigationPlugin.getClosestPoint(destinationPoint));
pathLine = BABYLON.MeshBuilder.CreateDashedLines("ribbon", { points: pathPoints, updatable: true, instance: pathLine }, scene);
```

## Baking result

Building a navigation mesh can take a lot of CPU and network resources. To reduce download size and CPU usage, it is possible to bake the result of the navigation mesh computation to a byte stream. That byte stream can later be restored to get the navigation mesh back.

To retrieve the binary representation of the computed navigation mesh:

```javascript
const binaryData = navigationPlugin.getNavmeshData();
```

`binaryData` is a `Uint8Array` that you can save to a file, for example.
To restore an UInt8Array to a navigation mesh:

```javascript
navigationPlugin.buildFromNavmeshData(uint8array);
```

## Using 3rd party tools

Third-party tools like `navmesh-generator` can help create a navmesh and bake the result to a binary file that is directly usable (the navmesh coordinate system is right-handed): [Link to navmesh-generator](https://navmesh-generator.babylonjs.xyz/)

Navmesh computation runs on dragged-and-dropped glTF files. The export can then be opened as in this Playground:

<img src="/img/pageImages/navmesh-editor.webp" title="Precomputing a navmesh with navmesh-generator"/>

Loading a precomputed navmesh: <Playground id="#KVQP83#92" title="Loading a precomputed navmesh" description="Loading a precomputed navmesh"/>

## Web Worker

Building a navigation mesh can be time- and resource-heavy. For many use cases, it can be necessary to delegate navmesh computation to a web worker. Work will be done in parallel, letting the engine display the content without slowdowns.

To enable a web worker, specify the URL of a web worker `.js` script to use:

```javascript
let navigationPlugin = new BABYLON.RecastJSPlugin();
navigationPlugin.setWorkerURL("workers/navMeshWorker.js");
```

A default web worker is provided [at this URL](https://github.com/BabylonJS/Babylon.js/blob/master/packages/tools/playground/public/workers/navMeshWorker.js)

Then, provide a completion callback to the `createNavMesh` method. This callback will be called when the navigation mesh is computed and ready to be used by the plugin.

```javascript
navigationPlugin.createNavMesh([staticMesh], navmeshParameters,(navmeshData) =>
{
    console.log("got worker data", navmeshData);
    navigationPlugin.buildFromNavmeshData(navmeshData);
    ...
```

`navMeshData` is a binary version of the navmesh, ready to serialize. It can be saved or streamed. The user has to call `buildFromNavmeshData` to deserialize the data. Once the navmesh is fully loaded, it is possible to create a crowd, query the navmesh, and more.

Performance note: The navmesh is constructed from geometry data. If multiple meshes are needed, their geometry will be merged before passing the geometry positions and indices to Recast. This part of the code can be CPU intensive and cannot be done in a worker because of dependencies, copies, memory footprint.

An example of using a web worker: <Playground id="#TN7KNN#2" title="Navigation mesh computation with a web worker" description="Navigation mesh computation with a web worker"/>

## NPM

Loading the Recast-Detour NPM module differs between versions 1.3.0 and 1.4.0+, as the latter version is asynchronous. The user has to use `await` like this:

```javascript
const recast = await Recast();
```



## Crowds and navigation agents

![a crowd ](/img/extensions/navigation/CrowdExample.webp)

Now that we have a navmesh, we can create autonomous agents and make them navigate within its constraints.
The agents will find the best path to that destination while avoiding other crowd agents.
An agent is attached to a Transform. That means you have to attach a mesh to see it, but you can attach pretty much anything.

A demo can be found at: <Playground id="#X5XCVT#240" title="Crowd and Navigation Agents" description="Crowd and navigation agents."/>

Click anywhere on the navmesh to make the agents go to that location.

## How to use it?

The first thing is to create a crowd to which all agents will belong. The parameters are the maximum number of agents in the crowd, the maximum agent radius, and the scene.

```javascript
const crowd = navigationPlugin.createCrowd(10, 0.1, scene);
```

Then, to create an agent and attach it to a transform, call:

```javascript
const agentIndex = crowd.addAgent(position, agentParameters, transform);
```

And that's it! You will get a non-moving agent. We now want to move it.

```javascript
crowd.agentGoto(agentIndex, navigationPlugin.getClosestPoint(endPoint));
```

This code gets the closest position on the navmesh to `endPoint`. Then it asks the agent to go to that position.
Depending on your agent parameters, it will get there faster or slower.

## Agent Parameters

radius - Radius of the agent in world units.

height - Height in world units.

maxAcceleration - Maximum acceleration in world units per second.

maxSpeed - Maximum speed in world units per second.

collisionQueryRange - The agent collision system will take care of other agents within that radius in world units.

pathOptimizationRange - Controls how the path will be optimized and straightened.

separationWeight - Controls how strongly the system will try to separate the agent. A value of 0 means it will not try, and agents might collide.

You can update any of these parameters per agent by calling:

```javascript
// change speed and max speed
crowd.updateAgentParameters(agentIndex, { maxSpeed: 10, maxAcceleration: 200 });
```

## Teleport

You can teleport an agent to any position using this call:

```javascript
crowd.agentTeleport(agentIndex, navigationPlugin.getClosestPoint(destinationPoint));
```

Please note that the navigation state is reset when teleporting. You'll have to call `agentGoto` to choose a new destination.

## Agent orientation and next path target

The Recastjs crowd system does not handle agent orientation. However, velocity is available, and it is possible to orient the geometry toward it.
To do so, you need to use Math.atan2, as in the following example. Please pay attention to the length of the velocity vector. If it is not large enough, you may encounter jittering.

```javascript
let velocity = crowd.getAgentVelocity(agentIndex);
if (velocity.length() > 0.2) {
  const desiredRotation = Math.atan2(velocity.x, velocity.z);
  // interpolate the rotation on Y to get a smoother orientation change
  ag.mesh.rotation.y = ag.mesh.rotation.y + (desiredRotation - ag.mesh.rotation.y) * 0.05;
}
```

In this PG <Playground id="#6AE0RP" title="Agent Orientation and Next Path Targeting" description="Example of agent orientation and next path targeting."/>

The agent's cube is oriented by the velocity and a grey little box is placed at the position of the next path corner.

## Agent reaching target Observer

An observable automatically fires when an agent reaches the destination (i.e., is within the destination radius). By default, the radius is the agent radius, but it can be changed using the `reachRadius` number property in the `IAgentParameters` object.
If there are too many agents in the crowd trying to reach the same destination, a bottleneck can occur and only a few agents will reach the destination. Be sure to set those values properly.
To add an observable, simply add your function:

```javascript
const crowd = navigationPlugin.createCrowd(10, 0.1, scene);
...
crowd.onReachTargetObservable.add((agentInfos) => {
    console.log("agent reach destination: ", agentInfos.agentIndex);
});
```



## Prerequisites

Internally, a navigation mesh with support for obstacles differs from the 'standard' use case of a single static navigation mesh.
Obstacles mark tiles as dirty. Those tiles need to be reprocessed to compute a portion of the navigation mesh.
To do so, Recast introduces the concept of tiles. Each tile is a square portion of the nav mesh. Tiles are processed each frame.
This makes large changes process over a couple of frames.

In order to build a Tiled nav mesh instead of a single mesh, add the following lines to the nav mesh creation parameters:

```javascript
const navmeshParameters = {
  cs: 0.2,
  ch: 0.2,
  walkableSlopeAngle: 0,
  walkableHeight: 0.0,
  walkableClimb: 0,
  walkableRadius: 1,
  maxEdgeLen: 12,
  maxSimplificationError: 1.3,
  minRegionArea: 8,
  mergeRegionArea: 20,
  maxVertsPerPoly: 6,
  detailSampleDist: 6,
  detailSampleMaxError: 15,
  borderSize: 1,
  tileSize: 20,
};
```

Note the addition of `tileSize`. It is the world-unit size of each tile. If it is not present or has a value of zero, it falls back to the standard use case and obstacles will not work.
Also, depending on your use case, this value must be chosen carefully to balance too many tiles against overly CPU-intensive updates.

**Note :**
The `ComputePath` method will handle obstacles only when a crowd has been instantiated. Otherwise, the path will adhere to the navmesh without considering obstacles. If your use case involves computing a path with obstacles, agents are optional, but a crowd is mandatory.

## Obstacles API

Once the navigation mesh is updated to take tiles into account, obstacles are accessible through 3 simple functions:

```javascript
addCylinderObstacle(position: Vector3, radius: number, height: number): IObstacle;
addBoxObstacle(position: Vector3, extent: Vector3, angle: number): IObstacle;
removeObstacle(obstacle: IObstacle): void;
```

Keep a list of added obstacles in order to remove them later. An obstacle that is not colliding with the navigation mesh will have no influence.

<Playground id="#WCSDE1" title="Adding a door to a navigation mesh" description="Example of door simulation using nav mesh obstacles."/>



## Why are there two versions of the Navigation plugins?

V1 uses an old and unmaintained JavaScript package, which often causes issues when used with ES6 and bundlers. V1 is being deprecated.

V2 uses the remarkable [recast-navigation-js](https://github.com/isaac-mason/recast-navigation-js) library by Isaac Mason (thank you Isaac!), which is fully ES6-compliant, up to date, and well maintained.

V2 will eventually replace V1 after it gets thoroughly tested. That is why V2 is in `ADDONS` and not already in the `core` package. Since the two versions should be fully compatible and only the internal implementation details and the underlying Recast/Detour libraries have changed, you can easily switch to V2 without rewriting your code.

There is one change, though. You need to initialize the plugin by calling an `async` factory function instead of using a constructor, as in V1.

## How to use the navigation plugin V2?

```javascript
const navigationPlugin = await ADDONS.CreateNavigationPluginAsync(); // uses WASM under the hood
const navigationPlugin = await ADDONS.CreateNavigationPluginWorkerAsync(); // under construction
```

This call will dynamically load the `recast-navigation-js` library and initialize it.

### The new async createNavMeshAsync function

You can create a navmesh by calling the new async function:
```ts
const result = createNavMeshAsync(meshes: Array<Mesh>, parameters: INavMeshParametersV2): Promise<CreateNavMeshResult>;
```

Important! A worker version is not supported yet, but we are working on it. In a production environment, you would use a pregenerated nav mesh, so this should not be an issue. You can use [NavMesh generator](https://navmesh-generator.babylonjs.xyz/) to generate your nav meshes, or use the `navigationPlugin.getNavmeshData(): Uint8Array` (present in V1 as well) or `navigationPlugin.getTileCacheData(): Uint8Array` (new) functions to get the navmesh binary data. You can import the exported data using:

```ts
navigationPlugin.buildFromNavmeshData(data: Uint8Array): void
navigationPlugin.buildFromTileCacheData(tileCacheData: Uint8Array, tileCacheMeshProcess?: TileCacheMeshProcess): void
```

## ES6 example

```
npm i @babylonjs/addons @recast-navigation/core @recast-navigation/generators
```

```ts
import { CreateNavigationPluginAsync } from "@babylonjs/addons";
import * as RecastCore from "@recast-navigation/core";
import * as RecastGenerators from "@recast-navigation/generators";

await RecastCore.init();
const navigationPlugin = await CreateNavigationPluginAsync({
    instance: {
        ...RecastCore,
        ...RecastGenerators,
    },
});
```

Since you are injecting the `recast-navigation` libraries yourself, you can also use the plugin with the `NullEngine` on a server, because when an instance is injected this way, the libraries will not be loaded using an HTML &lt;script&gt; tag.

## Recommended links not just for first time users

Most of the functions in the navigation plugin interface are self-explanatory, even for first-time users. It is recommended to get familiar with concepts at Recast: [https://recastnav.com/](https://recastnav.com/).

Since the navigation plugin V2 acts as a facade around `recast-navigation-js`, it is recommended to visit the recast-navigation-js [GitHub repo](https://github.com/isaac-mason/recast-navigation-js) as well.

To easily generate your NavMesh and/or TileCache, use the [NavMesh generator](https://navmesh-generator.babylonjs.xyz/)






![Generated Navmesh ](/img/extensions/navigation/NavMeshGeneration.webp)

A Navigation Mesh (or navmesh for short) is a surface topology describing the space where an agent can go based on constraints.
Based on parameters such as agent radius, climbing capability, and height, navmesh generation computes a topology from source meshes (the world geometry).
Then, this topology can be displayed for debugging (the blue mesh in the screenshot above) to validate the parameters.

A demo can be found at: <Playground id="#HFY257#4" title="Crowd Navigation Demo" description="Simple example showcasing the crowd agent and nav mesh systems."/>



# Image Filter Control

The Babylon.js Filter Control is a web control built on top of Babylon.js to apply filters to pictures on web pages at blazing-fast speed.

## Introduction

Filtering images to apply effects can be tedious and slow on the CPU. The best place to do this, for a wide variety of effects, is on the GPU. But setting up an entire WebGL pipeline just to process images can be tricky, especially if you want to benefit from WebGL 2-to-1 fallback and work around well-known platform issues.

To greatly simplify this task we introduced the `ImageFilter` Control.

![ImageFilter](/img/features/controls/imageFilter.webp)

## How to use

### Installation

To get started with the image filter control, you first need to install the controls npm package.

```javascript
npm install @babylonjs/controls
```

To reduce the size of your web page, the controls library is based on the ES6 version of `@babylonjs/core` used as a peer dependency. Therefore, if you are not relying on it so far in your project, you also need to install core:

```javascript
npm install @babylonjs/core
```

### Instantiation

Once done, you can now import the control in your code:

```javascript
import { ImageFilter } from "@babylonjs/controls/imageFilter";
```

And simply instantiate it on your page:

```javascript
const imageFilter = new ImageFilter(filterCanvas);
```

You simply need to provide a canvas on which we can use a WebGL context. You can also provide another Babylon.js control to share the WebGL context.

Also, if you need more specific engine configurations or want to use post processes, you should manually use your own engine:

```javascript
const engine = new Engine(filterCanvas);
const imageFilter = new ImageFilter(engine);
```

By default, the control relies on ThinEngine to optimize your bundle, but it might have limitations you do not want in your experience.

## Using Post Process

In order to apply an existing post process as the image filter, you can use the following code:

```javascript
const blackAndWhitePostProcess = new BlackAndWhitePostProcess("bw", 1, null, undefined, engine);
backAndWhiteFilter.filter(imageToProcess, blackAndWhitePostProcess);
```

Where imageToProcess could be either:

- the url of a picture.
- a video element (the current visible frame of the video will be used)
- another canvas element (the current visible state will be used)

This will apply the post process to the provided input and display it in the canvas.

## Using Custom Shader

In order to apply a custom shader as the image filter, you can use the following code:

```javascript
const customEffectWrapper = new EffectWrapper({
  name: "Custom",
  engine: customFilter.engine,
  fragmentShader: `
        // Samplers
        varying vec2 vUV;
        uniform sampler2D textureSampler;
        
        void main(void) 
        {
            gl_FragColor = texture2D(textureSampler, vUV);

            // Swizzle channels
            float r = gl_FragColor.r;
            gl_FragColor.r = gl_FragColor.b;
            gl_FragColor.b = r;
        }
    `,
  samplerNames: ["textureSampler"],
});
customFilter.filter(imageToProcess, customEffectWrapper);
```

Where imageToProcess could be either:

- the url of a picture.
- a video element (the current visible frame of the video will be used)
- another canvas element (the current visible state will be used)

This will apply the custom shader passed as a parameter to the picture. By default, `vUV` is available as a varying that defines the full output as texture coordinates. `textureSampler` needs to be present and defines the texture corresponding to the input parameter.

Please note that if you need to add custom uniforms or samplers, they should be defined in the effect wrapper lists:

```javascript
const customEffectWrapper = new EffectWrapper({
  name: "Custom",
  engine: customFilter.engine,
  fragmentShader: `
        varying vec2 vUV;
        
        // Default Sampler
        uniform sampler2D textureSampler;

        // Custom uniforms
        uniform sampler2D otherTexture;
        uniform vec3 colorOffset;

        const vec2 scale = vec2(0.25, 1.);

        void main(void) 
        {
            gl_FragColor = texture2D(textureSampler, vUV);

            // Swizzle channels
            float r = gl_FragColor.r;
            gl_FragColor.r = gl_FragColor.b;
            gl_FragColor.b = r;
            gl_FragColor.rgb += clamp(colorOffset, 0., 1.);

            gl_FragColor.rgb *= texture2D(otherTexture, vUV * scale).rgb;
        }
    `,
  // Defines the list of existing samplers (default + customs).
  samplerNames: ["textureSampler", "otherTexture"],
  // Defines the list of existing uniform to be bound.
  uniformNames: ["colorOffset"],
});
```

Then you can simply bind them either in `onApply`:

```javascript
customEffectWrapper.onApplyObservable.add(() => {
  // Sets the custom values.
  customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
  customEffectWrapper.effect.setFloat3("colorOffset", 0.2, 0, 0.2);
});
```

Or during the render loop:

```javascript
// Rely on the underlying engine render loop to update the filter result every frame.
engine.runRenderLoop(() => {
  // Only render if the custom texture is ready (the default one is
  // checked for you by the render function)
  if (!otherTexture.isReady()) {
    return;
  }

  // Sets the custom values.
  time += engine.getDeltaTime() / 1000;
  customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
  customEffectWrapper.effect.setFloat3("colorOffset", Math.cos(time) * 0.5 + 0.5, 0, Math.sin(time) * 0.5 + 0.5);

  // Render. Please note we are using render instead of filter to improve
  // performances of real time filter. filter is creating a promise and will therefore
  // generate some lags and garbage.
  customFilter.render(mainTexture, customEffectWrapper);
});
```

Finally, if you are relying on new textures, you need to wait for them to be ready before rendering.

## Process to the Canvas

This is by far the simplest option if you have a canvas on your page. You simply need to use the following code to fit the provided element to the canvas size:

```javascript
imageFilter.filter(imageToProcess, filter);
```

On the previous line, imageToProcess could be either:

- the url of a picture.
- a video element (the current visible frame of the video will be used)
- another canvas element (the current visible state will be used)

This is the default behavior.

## Process to a Babylon Texture

Instead of filtering directly to a canvas, you may prefer to create only a Babylon.js texture on the GPU. For this, you can use the following function:

```javascript
const texture = imageFilter.getFilteredTexture(imageToResize, { width: 128, height: 100 }, filter);
```

Like before, imageToResize could be either:

- the url of a picture.
- a video element (the current visible frame of the video will be used).
- another canvas element (the current visible state will be used).

You also need to provide the size you want your texture to have on the GPU.

Now you are free to use this texture with any other controls.

## Real-Time Filtering

Instead of filtering only once, you might want to create dynamic real-time effects. For this, you can simply render the effect during the render loop:

```javascript
// Rely on the underlying engine render loop to update the filter result every frame.
engine.runRenderLoop(() => {
  // Only render if the custom texture is ready (the default one is
  // checked for you by the render function)
  if (!otherTexture.isReady()) {
    return;
  }

  // Sets the custom values.
  time += engine.getDeltaTime() / 1000;
  customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
  customEffectWrapper.effect.setFloat3("colorOffset", Math.cos(time) * 0.5 + 0.5, 0, Math.sin(time) * 0.5 + 0.5);

  // Render. Please note we are using render instead of filter to improve
  // performances of real time filter. filter is creating a promise and will therefore
  // generate some lags and garbage.
  customFilter.render(mainTexture, customEffectWrapper);
});
```

This requires using the render function instead of the filter function to improve performance.

## Full Code Sample

You can find the entirety of the code sample above on [Github](https://github.com/BabylonJS/Controls/blob/master/www/imageFilter/index.ts) if you want to see it in action and better understand how some of the features can be used.

## Live Demo

Please have a look at the [Live Image Filter Demo](https://controls.babylonjs.com/imageFilter) to better appreciate how it works.


Crowd Navigation System

Coming next

Introduction to NavigationPlugin V2

Learn about NavigationPlugin V2

Creating A Navigation Mesh

Learn how to create a mesh as a confinement system for crowd agents.

Crowd Agents

Learn how to create crowd agents that follow a path along a navigation mesh.

Adding and removing obstacles

Learn how to add and remove obstacles for crowd agents.