
## Overview

Smart Filters are "full screen" effects built by composing blocks together into a `SmartFilter` object, from which a `SmartFilterRuntime` is created.

You can use the built-in blocks or use blocks you create yourself to build your Smart Filters.

The Smart Filter core comes with an optimizer which can take a `SmartFilter` and return an optimized `SmartFilter` which consolidates the shader blocks into as few as possible. The `RenderTargetGenerator` used when creating a `SmartFilterRuntime` also contains an optimizer which will reuse intermediate textures whenever possible within the Smart Filter.

## How To Explore

### With the Smart Filter Editor

To familiarize yourself with Smart Filters, you can start by trying out the [Smart Filter Editor](https://sfe.babylonjs.com).
![SFE](/img/how_to/smart-filters/sfe-default.webp)

This tool lets you visually construct and run Smart Filters using the provided built-in blocks, as well as custom blocks you create and load into the tool, all before you even clone the repo.

Read more about the Smart Filters Editor [here](../../../../toolsAndResources/sfe/)

### In the Playground

You can also experiment with Smart Filters in the Playground!

<Playground id="#N7VSE6#4" title="Smart Filter Camera" description="Demonstrates how a Smart Filter can be used to apply an effect to the camera in a scene." />
<Playground id="#1VIWKL#5" title="Smart Filter Camera from SFE" description="Demonstrates how a Smart Filter saved from the Smart Filter Editor can be used to apply an effect to the camera in a scene." />

### With Code

Clone [BabylonJS/Babylon.js](https://github.com/BabylonJS/Babylon.js), and you can see how the Smart Filters Editor application uses Smart Filters.

Once you've cloned, run `npm install` then choose one of these approaches to build the Smart Filter Editor:

- Select `Smart Filter Editor development (Edge)` or `Smart Filter Editor development (Chrome)` in Visual Studio Code's `Run and Debug` menu then hit F5
- Run `npm run watch:assets:smart-filters` in a terminal, then `npm run watch:source:smart-filters` in another terminal, and finally `npm run serve -w @tools/smart-filters-editor` in a third terminal

#### Key Files

- [app.ts](https://github.com/BabylonJS/Babylon.js/blob/master/packages/tools/smartFiltersEditor/src/app.ts)
  - The main file for the Smart Filter Editor - it handles loading and saving Smart Filters and launching the editor control
- [smartFilterLoader.ts](https://github.com/BabylonJS/Babylon.js/blob/master/packages/tools/smartFiltersEditor/src/smartFilterLoader.ts)
  - The helper which takes a deserializer and uses it to load Smart Filters
- [smartFilterRenderer.ts](https://github.com/BabylonJS/Babylon.js/blob/master/packages/tools/smartFiltersEditor/src/smartFilterRenderer.ts)
  - The helper that takes a loaded Smart Filter and begins rendering it



## How to Install

Install the Smart Filters core package: `npm install @babylonjs/smart-filters`.

The package has been marked as `sideEffects: false` so you can freely import from the index and know that tree shaking will remove any code you don't end up using.

It requires the following peer dependencies:

- @babylonjs/core

## Important Notes

<Alert severity="info">
Do not rely on the path of files within the `@babylonjs/smart-filters` package.
 - Instead, import from the root index and allow tree-shaking to remove anything you don't use.
 - Otherwise, future updates could move or rename files and require you to update your references.

</Alert>

## Simplest Usage

This sample creates a trivial hardcoded SmartFilter, creates the runtime, and then calls render() to render one frame to the canvas.

<Alert severity="info">
Note: Smart Filters are currently compatible with the Babylon Engine and ThinEngine.
</Alert>

```typescript
// Create a SmartFilter
const smartFilter = new SmartFilter("Simplest");
const titleInput = new InputBlock(smartFilter, "logo", ConnectionPointType.Texture, logoTexture);
titleInput.output.connectTo(smartFilter.output);

// Create a SmartFilterRuntime
const engine = new ThinEngine(canvas, true);
const runtime = await smartFilter.createRuntimeAsync(engine);

// Render one frame
runtime.render();
```

## Using the Optimizer

This sample creates a more interesting SmartFilter, uses the SmartFilterOptimizer to optimize the SmartFilter, and creates a RenderTargetGenerator with optimizations enabled.

```typescript
// Create a SmartFilter
const smartFilter = new SmartFilter("BlackAndWhiteAndPixelate");
const logoInput = new InputBlock(smartFilter, "logo", ConnectionPointType.Texture, createStrongRef(logoTexture));
const blackAndWhite = new BlackAndWhiteBlock(smartFilter, "blackAndWhite");
const pixelate = new PixelateBlock(smartFilter, "pixelate");
const pixelateIntensity = new InputBlock(smartFilter, "intensity", ConnectionPointType.Float, 0.4);

logoInput.output.connectTo(blackAndWhite.input);
blackAndWhite.output.connectTo(pixelate.input);
pixelateIntensity.output.connectTo(pixelate.intensity);
pixelate.output.connectTo(smartFilter.output);

// Create and call the optimizer
const optimizer = new SmartFilterOptimizer(smartFilter);
const optimizedSmartFilter = optimizer.optimize();

// Create a RenderTargetGenerator with optimization enabled
const rtg = new RenderTargetGenerator(true);

// Create the ThinEngine and the SmartFilterRuntime
const engine = new ThinEngine(canvas, true);
const runtime = await optimizedSmartFilter.createRuntimeAsync(engine, rtg);

// Render one frame
runtime.render();
```

## Rendering to Another Render Target

In this example, we build a trivial hardcoded Smart Filter, then have it render to a RenderTargetWrapper that we supply instead of rendering to the canvas:

```typescript
// Create a SmartFilter
const smartFilter = new SmartFilter("Simplest");
const titleInput = new InputBlock(smartFilter, "logo", ConnectionPointType.Texture, logoTexture);
titleInput.output.connectTo(smartFilter.output);

// Tell the SmartFilter to render to RenderTargetWrapper we supply
smartFilter.outputBlock.renderTargetWrapper = myRenderTargetWrapper;

// Create a SmartFilterRuntime
const engine = new ThinEngine(canvas, true);
const runtime = await smartFilter.createRuntimeAsync(engine);

// Render one frame
runtime.render();
```

Notes:

1. The choice between rendering to the canvas and to a renderTargetWrapper is locked in for a runtime when you create the `SmartFilterRuntime`.
   - To change from rendering to the canvas to rendering to a RenderTargetWrapper, or the reverse, create a new runtime.
1. If you started with a RenderTargetWrapper, you can change outputBlock.renderTargetWrapper to a different RenderTargetWrapper at runtime, and the next render() call will target it instead.
   - If the new renderTargetWrapper is a different size from the previous one, call smartFilter.resize() before the next render() call is made.

## Handling Resize

If the canvas changes size (or you are using outputBlock.renderTargetWrapper and change to a new RenderTargetWrapper), be sure to call smartFilter.resize() to ensure all intermediate textures are resized as well.

## Full Example

See [Introduction To Smart Filters](../introductionToSmartFilters) for more information about the demo application included in the Smart Filters repo.



Babylon Smart Filters is a node-based graph system for building GPU-accelerated 2D effects for scenarios like real-time video filters such as background blur. While the Node Material Editor is a graphical tool for creating shaders, Smart Filters is for generating a pipeline of shaders. We call them smart because this feature includes an optimizer that automatically minimizes the number of draw calls and intermediate textures for you.

You can build Smart Filters programmatically:

```javascript
const smartFilter = new SmartFilter("Default");

const pixelateBlock = new PixelateBlock(smartFilter, "Pixelate");

const textureInputBlock = new InputBlock(smartFilter, "Texture", ConnectionPointType.Texture, null);
textureInputBlock.editorData = {
  url: "/assets/logo.png",
  urlTypeHint: "image",
  flipY: true,
  anisotropicFilteringLevel: null,
  forcedExtension: null,
};

const intensityInputBlock = new InputBlock(smartFilter, "Intensity", ConnectionPointType.Float, 0.5);
intensityInputBlock.editorData = {
  animationType: null,
  valueDeltaPerMs: null,
  min: 0.0,
  max: 1.0,
};

textureInputBlock.output.connectTo(pixelateBlock.input);
intensityInputBlock.output.connectTo(pixelateBlock.intensity);
pixelateBlock.output.connectTo(smartFilter.output);
```

Or visually using a new tool, the [Smart Filter Editor](https://sfe.babylonjs.com):
![SFE](/img/how_to/smart-filters/sfe-default.webp)

See the following subtopics to learn how to get started with Smart Filters, how to integrate them into your projects, how to create custom blocks, and more.
