Module: Sprites

Package path: packages/babylon-lite/src/sprite/

This is the standalone, one-shot architecture document for the sprite module. Two sprite families are defined: Sprite2DLayer (the foundation; pixel-coordinate quads, with an opt-in world-anchor adapter for "2.5D" labels) and *BillboardSpriteSystem (world-coordinate, perspective-correct, camera-oriented quads in two current orientation modes).

The engine grows a small registration list. Two kinds of things implement RenderingContext and can be registered with an engine: a SceneContext (via registerScene(scene)) and a SpriteRenderer (via registerSpriteRenderer(sr)). Each is driven once per frame by startEngine(engine) in registration order. Pure-2D experiences (Lottie/Rive-class apps) create one or more SpriteRenderers and register them on the engine — no SceneContext, no addToScene. HUD-on-3D apps register a SceneContext first, then construct a separate SpriteRenderer for the HUD layers and register it after the scene so it draws on top; the HUD's GPU lifetime is tied to the scene via onSceneDispose(scene, () => disposeSpriteRenderer(hud)). Depth-hosted Sprite2DLayers (those with depth: "test" | "test-write") are added through addDepthHostedSpriteLayer(scene, layer). That opt-in sprite module function registers a deferred scene builder; because callers explicitly import this function when they want scene-hosted sprites, its static sprite-renderable import is limited to that opt-in graph. The builder inserts a generic Renderable into the scene, and the frame graph then buckets it alongside meshes by isTransparent / _direct; PBR refraction setup separately uses _transmissive to exclude true refractive surfaces from the opaque-scene RTT.

This document contains the full specification needed to implement the module from scratch — public API, internal architecture, GPU layouts, WGSL composers, picking contributors, lifecycle, handles, parenting, tests, and bundle ceilings. No prior sprite design document is required for context.

Purpose

Lite's design rule is "build things on top of previous things." Sprites are 2D quads. World-anchored ("2.5D") labels are 2D quads whose pixel position is computed each frame from a 3D anchor. Camera-facing world-sized billboards are different geometry (world-unit size, perspective foreshortening, depth participation), and so they remain a separate family.

The module exposes two sprite families:

1. Sprite2DLayer — the foundation. Pixel-coordinate quads, no view matrix, no perspective divide, no required camera. Each layer chooses its depth participation at construction (depth: "none" | "test" | "test-write"), which becomes part of the pipeline cache key. The same layer factory serves three callers:

   • Pure-2D apps (no scene): layers are handed to a SpriteRenderer that the caller constructs and registers on the engine.
   • HUD overlays on a 3D scene: same — a SpriteRenderer constructed by the caller, registered after the scene, with disposal tied to the scene via onSceneDispose. HUD layers (depth: "none") never go through addToScene.
   • Depth-hosted sprites that sort against 3D meshes (depth: "test" or "test-write"): added via addDepthHostedSpriteLayer(scene, layer). That opt-in function queues them into the existing renderable system so they participate in the 3D depth attachment.

   World-anchored sprites are not a separate family. They are Sprite2DLayer sprites with an opt-in AnchorSource adapter that runs on the CPU in a per-frame _beforeRender hook, projects the world anchor through the scene's camera, and writes the resulting layer-space positionPx (and optionally a derived per-instance z) directly into the same per-instance slot a pure-2D sprite uses. The vertex shader, per-instance layout, packed buffer, and pipeline are identical to a pure-2D layer.

2. *BillboardSpriteSystem — two orientation factories (createFacingBillboardSystem and createAxisLockedBillboardSystem), with distinct WGSL basis paths and pipeline keys inside the current compact shared billboard renderable/pipeline module. That module is dynamically imported only by the scene add helpers when a billboard system is actually queued for a scene. World-coordinate quads, world-unit size, perspective foreshortening, full depth participation. Drawn inside the scene's 3D pass; not usable from the pure-2D path (no camera). Yaw-locked billboards are axis-locked with [0, 1, 0] as the lock axis.

Depth-writing sprites can opt into alpha-to-coverage before scene registration:

• setAlphaToCoverage(layer, true) is effective for a scene-hosted Sprite2DLayer with depth: "test-write" on a multisampled target. The layer keeps its selected blend mode; use spriteBlendOpaque for replacement color/depth rendering.
• setAlphaToCoverage(system, true) is effective for a cutout BillboardSpriteSystem on a multisampled target. The A2C variant replaces the binary cutoff with continuous texture-alpha sample coverage.

The cutout billboard variant uses replacement color and per-sample depth writes. Pure-2D/HUD SpriteRenderer layers target a 1x swapchain and retain their normal blend descriptors; A2C is neither useful nor enabled on that path.

SpriteAtlas and SpriteFrame are shared across both families. Clip animation, stable handle objects, and parenting are roadmap modules that must remain additive and separately importable so index-only scenes pay zero bytes for them.

Pillars (front and centre)

• No if on render path. Family selection, anchor mode, and depth mode are all decided at layer/system construction time and baked into the pipeline cache key. The per-frame draw walks fixed arrays, with no per-sprite mode test.
• Pay-for-use. A pure-2D app's static import graph terminates at engine + sprite-atlas + sprite-2d + sprite-renderer. It never imports scene-core, Camera, Mesh, LightBase, depth/MSAA targets, billboard renderables/pipelines, or anchor projection code. Tree-shaking removes them all.
• One engine loop, two registerable kinds. startEngine(engine) walks engine._renderingContexts once per frame. Pure-2D apps register one or more SpriteRenderers; HUD-on-3D apps register a SceneContext followed by a separate SpriteRenderer for the HUD. Depth-hosted Sprite2D layers go through addDepthHostedSpriteLayer and are drawn inside the scene's 3D pass via the existing renderable system. The engine has no notion of "2D vs 3D" — it just iterates registrations; scene contexts execute their frame graph, and sprite renderers open a sprite-only swapchain pass.
• Extensions over hardcoding. Anchoring remains a roadmap tree-shakable add-on that should be imported only when a scene actually uses world anchors. Billboard scene rendering is behind billboard-scene.ts; importing a billboard system factory alone does not import the scene renderable, pipeline, or WGSL body.

Taxonomy — Two Sprite Families

Why anchored is no longer a family

A naive design would split Sprite2DLayer and an AnchoredSpriteLayer into two families because anchored sprites need a viewProjection to project their world anchor, and anchored sprites that should occlude behind 3D geometry need a depth attachment. That separate-family design would have its own WGSL composer, its own 112-byte instance stride (worldPos + offsetPx + depthBias), and its own GPU vertex-stage projection.

That shape is wrong for three concrete reasons:

1. The actual difference is one CPU operation per anchored sprite per frame. Project a world anchor through viewProjection, divide by w, scale to viewport pixels, write the result into the same positionPx[2] slot a pure-2D sprite would use. This is one Mat4 × Vec4 (16 FMAs) plus 2 multiplies and 2 adds. For typical anchored populations (HUD pins, nameplates, map markers — dozens to a few hundred) this is microseconds per frame. Doing it on the CPU keeps the GPU pipeline, the per-instance layout, the packed buffer stride, and the WGSL vertex shader byte- identical to a pure-2D layer.

2. Depth participation is a per-render-pass attachment decision, not a per-family decision. Modelling it as a family leaks a pass-level constraint into the layer type and forces the public API to choose one shape ("anchored") instead of letting any 2D layer opt into depth testing. Modelling it per layer (depth: "none" | "test" | "test-write") is the correct level of granularity. Each value is a pipeline-cache key bit baked once at composition time — never branched at runtime.

3. The 3D scene UBO (viewProjection + camera basis + viewport) was paid for solely to GPU-project anchors. Once we project on the CPU, anchored layers do not need that UBO at all (the camera basis appears only as the viewProjection matrix consumed by the CPU projection helper, and viewportPx already lives in the pure-2D scene UBO). The 3D scene UBO becomes a billboard-only artefact, which it morally always was.

The roadmap "anchor" is a small interface:

export interface AnchorSource {
    /** Project this anchor for the current frame.
     *  Writes into outPx (length 2) and outZ (length 1, view-space depth).
     *  Returns false to hide the sprite this frame (off-screen, behind camera, parent not yet built). */
    readonly project: (outPx: Float32Array, outZ: Float32Array, scene: SceneContext) => boolean;
}

AnchorSource should live in a separate sprite/anchor/sprite-anchor.ts module. A scene that never instantiates an anchor must never import that module or pay bytes for camera-basis projection code.

Why billboards remain a separate family

Billboards are not "Sprite2D + a different anchor source." Their differences are per-vertex, not per-CPU-update:

• World-unit sizing. Billboard quads are extruded in world units along camera basis vectors before projection (cameraRight * sizeWorld.x + cameraUp * sizeWorld.y), which produces correct perspective foreshortening. Anchored sprites are extruded in pixel space after projection. These are opposite contracts (size shrinks with distance vs. size invariant under distance) — the entire reason each variant exists.

• Per-vertex camera basis. Each billboard variant computes (right, up) per vertex from the camera (Facing), or from a lock axis plus camera direction (AxisLocked, which includes yaw-locked as [0, 1, 0]). The pure-2D vertex shader has no camera basis input at all and ships zero camera-basis code.

• Depth-write semantics. Cutout billboards write depth (so they cast/ receive against opaque meshes); anchored sprites never write depth.

Forcing billboards through the Sprite2D pipeline would either require a per-vertex if (isBillboard) { compute world basis } else { compute pixel offset } (violating the no-if-on-render-path rule), or a CPU "project four corners" path (O(N×4) Mat4×Vec4 per frame against tree forests, the exact cost the billboard vertex-shader trick was invented to avoid). Splitting them is correct.

The two orientation factories remain explicit (createFacingBillboardSystem, createAxisLockedBillboardSystem(atlas, axis, opts)) — two shader basis paths, two pipeline keys, no axisLock?: 'none'|'y'|Vec3 flag. Yaw-locked billboards are just createAxisLockedBillboardSystem(atlas, [0, 1, 0], opts).

Modes deliberately not added

• World-aligned non-billboard sprite — use a Mesh with a textured alpha-blended material.
• Tile maps (SpriteMap-like) — separate future module.
• 2D-camera scene with pan/zoom — that is Sprite2DLayer.view (per-layer pan + zoom + rotation), no additional family.

Resolution: One engine loop, two registerable kinds

Decision: the engine has a single registration list. Two kinds of things implement RenderingContext and can be registered with an engine: a SceneContext (via registerScene(scene)) and a SpriteRenderer (via registerSpriteRenderer(sr)). startEngine(engine) walks engine._renderingContexts once per frame. The engine owns the command encoder and swapchain view for the frame; each context runs _update() and _record() against that current frame state. A SceneContext records through its FrameGraph (normally the default swapchain RenderTask), while a SpriteRenderer opens its own sprite pass directly on the swapchain. Pure-2D experiences (Lottie/Rive-class apps) create one or more SpriteRenderers and register them on the engine — they never touch SceneContext. HUD-on-3D apps register a SceneContext first, construct a separate SpriteRenderer for the HUD layers, register it after the scene so it draws on top, and tie its disposal to the scene via onSceneDispose(scene, () => disposeSpriteRenderer(hud)). Depth-hosted Sprite2D layers (depth: "test" | "test-write") are added through addDepthHostedSpriteLayer, which registers a deferred builder through the scene's generic optional-renderable hook. The hook inserts a generic renderable into the scene so it sorts and composites against 3D meshes inside the scene's 3D pass.

The SceneContext shape, the addToScene switch body, and the default frame graph stay sprite-agnostic. The sprite module does not add a second scene type or an implicit HUD layer list to SceneContext; composition remains explicit at the engine-registration level.

Rejected alternatives

| Alternative | Why rejected |\n| -------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Add a Scene2DContext parallel to SceneContext | Two parallel APIs for the same operation. Both kinds of registerable already share RenderingContext — no new umbrella type is required. |\n| Force pure-2D apps through SceneContext | A Lottie/Rive-class app then carries SceneContext's shape and any per-frame branching it implies. Pure-2D experiences should not pay for scene infrastructure. |\n| Pure-2D users own their own requestAnimationFrame loop and call a free renderSprite2DLayers(engine, layers, target) function | Two different loop owners (engine for scene-based, user for pure-2D) means two first-frame-ready handshakes, two fixed-delta contracts, two device-loss recovery paths. Making the engine the sole loop owner via register* is a smaller surface area and frame-graph-shaped. |\n| Auto-route HUD layers through addToScene and have registerScene lazily create + register an internal SpriteRenderer | Adds bytes to scenes that don't have HUDs (everyone pays for the lazy-init code path), couples the sprite module's lifecycle to the scene module, and makes "what is registered on the engine" non-obvious to the caller. Explicit createSpriteRenderer + registerSpriteRenderer + onSceneDispose is three lines of caller code and zero bytes for non-HUD scenes. |\n| Refactor addToScene into method-on-entity routing + per-stage wrappers | Public objects must remain pure state; optional scene-hosted systems use standalone add helpers plus the generic deferred-renderable hook instead of attaching behavior to public data. |\n| Extract every sprite GPU detail into the scene render loop | Couples sprite rendering to scene orchestration. Then the pure-2D path either re-implements the GPU draws (forking) or pulls in scene infrastructure to reach them (wasted bytes). |

The engine's registration list

// src/engine/engine.ts — sprite support adds nothing here;
// this is the existing RenderingContext shape that both SceneContext
// and SpriteRenderer implement.

/** A thing that can be registered with an engine and driven once per frame. */
export interface RenderingContext {
    /** Draw calls produced by pre-pass work during `_update` (shadows + pre-passes). */
    _drawCallsPre: number;
    /** Clear color used when this context is the first active one in a frame. */
    clearColor: GPUColorDict;
    /** Per-frame update: beforeRender hooks, shadow + pre-passes, UBO updates. */
    _update(): void;
    /** Record frame work using the engine's current encoder/swapchain view. Returns draw-call count. */
    _record(): number;
}

/** @internal Inside EngineContext: */
//   _renderingContexts: RenderingContext[];

export function registerRenderingContext(engine: EngineContext, context: RenderingContext): boolean;
export function unregisterRenderingContext(engine: EngineContext, context: RenderingContext): boolean;

/** Drive the engine's render loop. Walks _renderingContexts in order, once per frame. */
export function startEngine(engine: EngineContext): Promise<void>;

startEngine resolves on the first frame after registration completes successfully. Per-frame the engine acquires the swap-chain view + creates one command encoder, then stores them on the internal engine state for the current frame. It calls _update() and _record() on each registered context in order. Scene contexts execute their frame graph tasks; sprite renderers record a sprite-only swapchain pass. There is no per-frame if (is2D) branch; the loop just iterates.

The SpriteRenderer

// src/sprite/sprite-renderer.ts
import type { EngineContext, RenderingContext } from "../engine/engine.js";
import type { Sprite2DLayer } from "./sprite-2d.js";

/** Options accepted by `createSpriteRenderer`. */
export interface SpriteRendererOptions {
    /** Layers to draw, in registration order. The renderer also re-sorts internally
     *  by `layer.order` each frame (TimSort is O(n) on already-sorted input). */
    layers: readonly Sprite2DLayer[];
    /** Default true. Set false for HUD overlays so the sprite pass preserves existing scene color. */
    clear?: boolean;
    /** Default `{ r: 0, g: 0, b: 0, a: 1 }`. Used when `clear` is true. */
    clearValue?: GPUColorDict;
}

/** A `SpriteRenderer` — pure data, plugs into `engine._renderingContexts`. */
export interface SpriteRenderer extends RenderingContext {
    readonly _kind: "sprite-renderer";
    /** Renderer-owned layer membership. Use add/remove helpers to mutate. */
    readonly layers: readonly Sprite2DLayer[];
}

export function createSpriteRenderer(engine: EngineContext, opts: SpriteRendererOptions): SpriteRenderer;
export function addSpriteRendererLayer(sr: SpriteRenderer, layer: Sprite2DLayer): void;
export function removeSpriteRendererLayer(sr: SpriteRenderer, layer: Sprite2DLayer): boolean;
export function registerSpriteRenderer(sr: SpriteRenderer): void;
export function unregisterSpriteRenderer(sr: SpriteRenderer): void;
export function disposeSpriteRenderer(sr: SpriteRenderer): void;
/** Redirect output to an offscreen render texture (render-to-texture), or null = swapchain.
 *  See "Offscreen render target" below. */
export function setSpriteRendererTarget(sr: SpriteRenderer, target: Texture2D | null): void;

A SpriteRenderer does not own persistent color/depth attachments. During _record it opens a sprite-only render pass directly on the per-frame swapchain view using sampleCount = 1, the engine's swapchain format, and no depth attachment. clear: false makes the pass use loadOp: "load", which is the HUD path after a 3D scene has already resolved into the swapchain. Pure-2D layers use a 40-byte instance layout with no Z attribute; depth-hosted layers use the 44-byte layout with per-instance Z.

Internally SpriteRenderer._update:

1. Refreshes cached target dimensions (canvas may have resized).
2. Sorts the renderer-owned layer list in place by layer.order (skipped for the single-layer case).
3. For each visible non-empty layer: ensures GPU resources exist, writes per-instance data and per-layer UBO via device.queue.writeBuffer only when the data has actually changed (compares against a CPU-side shadow of the last bytes written).

Internally SpriteRenderer._record:

1. For each visible non-empty layer: looks up (or builds and caches) a GPURenderBundle that bakes setIndexBuffer + setPipeline + setBindGroup + setVertexBuffer + drawIndexed(6, count). The bundle is rebuilt only when the layer's sprite count changes or its instance buffer is reallocated.
2. Replays all visible bundles with one reused pass.executeBundles(...) array — skips per-call WebGPU validation and IPC, near-zero CPU cost in the steady state.

The SpriteRenderer is the single home for HUD/pure-2D sprite GPU draw logic. Depth-hosted Sprite2D layers (depth: "test" | "test-write") do not go through it — they are drawn by an independent Renderable produced by sprite-renderable.ts (see "Caller 2" below).

Offscreen render target (render-to-texture)

By default a SpriteRenderer records its pass on the per-frame swapchain view (rr._targetView ?? eng._swapchainView). Two small, opt-in, tree-shakable additions let a renderer draw into an offscreen texture instead, which is the building block for full-screen post-processing (the platformer demo's CRT/scanline effect is built entirely on these — no bespoke engine pass):

// src/texture/pixels-texture.ts
export interface RenderTexture2DOptions {
    addressModeU?: GPUAddressMode; // default 'clamp-to-edge'
    addressModeV?: GPUAddressMode; // default 'clamp-to-edge'
    minFilter?: GPUFilterMode; // default 'linear'
    magFilter?: GPUFilterMode; // default 'linear'
    format?: GPUTextureFormat; // default engine.format (REQUIRED for a SpriteRenderer target)
}
/** An empty Texture2D usable as BOTH a render target and a sampled texture
 *  (RENDER_ATTACHMENT | TEXTURE_BINDING | COPY_DST). */
export function createRenderTexture2D(engine: EngineContext, width: number, height: number, options?: RenderTexture2DOptions): Texture2D;

// src/sprite/sprite-renderer.ts
/** Point a renderer at an offscreen render texture (`Texture2D`), or null for the swapchain (default). */
export function setSpriteRendererTarget(sr: SpriteRenderer, target: Texture2D | null): void;

Both default to the swapchain / swapchain format, so every existing scene and demo is byte-for-byte unaffected, and the whole capability tree-shakes away when unused (createRenderTexture2D is a separate import; _targetView defaults to null).

Format constraint: a SpriteRenderer target must use engine.format. Sprite pipelines are created with that format, and WebGPU rejects a render pass whose color attachment format differs from the bound pipeline (validation error at pass begin). So leave RenderTexture2DOptions.format at its default for any texture you pass to setSpriteRendererTarget; a custom format is only for offscreen targets driven by a different pass (e.g. an EffectRenderer).

The render-to-texture pattern is two registered renderers, ordered:

1. Scene pass → setSpriteRendererTarget(scene, rt) so it draws into an offscreen rt = createRenderTexture2D(engine, w, h) (sized to the canvas backing store, swapchain format so it can be sampled and presented).
2. Present pass → a second SpriteRenderer owning one full-screen layer whose atlas IS rt (via createGridSpriteAtlas), drawn with a custom-shader fragment (e.g. CRT curvature + scanlines). It targets the swapchain (target = null) and is registered after the scene pass, so it runs second and samples the finished frame. Toggling the effect off is setSpriteRendererTarget(scene, null) + unregistering the present pass — restoring the direct path for zero overhead.

The offscreen texture and present layer must be rebuilt on canvas resize (the RT is a fixed-size GPU texture). See lab/lite/src/demos/platformer/crt.ts for a complete, toggle-able implementation.

Caller 1: pure-2D — no SceneContext

A Lottie/Rive-class app never creates a scene. It creates a SpriteRenderer, registers it on the engine, and lets startEngine drive the loop:

const engine = await createEngine(canvas);
const atlas = await loadSpriteAtlas(engine, "sprites.png", { gridSize: [32, 32] });
const layer = createSprite2DLayer(atlas);
addSprite2DIndex(layer, { positionPx: [100, 200], sizePx: [64, 64], frame: 0 });

const sr = createSpriteRenderer(engine, {
    layers: [layer],
    clearValue: { r: 0, g: 0, b: 0, a: 1 },
});
registerSpriteRenderer(sr);

await startEngine(engine);

The static import graph is exactly: engine + sprite-atlas + sprite-2d + sprite-renderer. Nothing else. No SceneContext, no addToScene, no registerScene, no Camera, no Mesh, no LightBase, no depth/MSAA target allocator, no PBR, no Standard, no shadow generator, no animation-group walker, no anchor projection, no billboard variants.

Anchored sprites are not supported in this path — there is no camera to project against. They require the scene-based path below.

Caller 2: scene-based — registerScene + addToScene + a separate HUD SpriteRenderer

The new registerScene(scene) is the scene-side analogue of registerSpriteRenderer. It runs the scene's deferred builders and registers the scene as a RenderingContext on the engine. After that, startEngine(engine) drives the scene each frame just like it drives any other registered context.

export function registerScene(scene: SceneContext): Promise<void>;
export function unregisterScene(scene: SceneContext): void;

/** Register a callback to fire when `disposeScene(scene)` is called.
 *  Used to tie user-owned GPU resources (e.g. a HUD `SpriteRenderer`)
 *  to the scene's lifetime — mirrors `onBeforeRender`. */
export function onSceneDispose(scene: SceneContext, cb: () => void): void;

The scene module exposes one generic optional-renderable hook. The sprite module uses it from addDepthHostedSpriteLayer; that opt-in helper statically imports the sprite-owned renderable builder because calling the helper is the explicit feature choice:

import { buildSpriteRenderable } from "./sprite-renderable.js";

export function addDepthHostedSpriteLayer(scene: SceneContext, layer: Sprite2DLayer): void {
    if (layer.depth === "none") {
        throw new Error('Sprite2DLayer with depth: "none" must be rendered via createSpriteRenderer, not addDepthHostedSpriteLayer.');
    }
    addDeferredSceneRenderables(scene, (engine) => {
        const built = buildSpriteRenderable(engine, layer);
        return { renderables: [built.renderable], dispose: built.dispose };
    });
}

buildSpriteRenderable rejects depth: "none", because scene renderables require a depth policy. addDepthHostedSpriteLayer performs the same check before queuing work. HUD layers use SpriteRenderer directly.

The existing "billboard-sprite-system" branch stays as today (with its own _deferredBuild hook); billboards are pushed into the existing scene-level renderable arrays.

registerScene(scene) does exactly two things: runs each queued _deferredBuild (so depth-hosted sprite renderables are wired into the scene's _renderables list), then calls registerRenderingContext(engine, scene). It does not create or register any HUD SpriteRenderer — HUDs are an explicit caller concern (see below).

End-to-end (HUD-on-3D scene, mirrors lab/lite/src/lite/scene52.ts):

const engine = await createEngine(canvas);
const scene = createSceneContext(engine);

addToScene(scene, createDirectionalLight([0, -1, 0]));
addToScene(scene, await loadGltf(engine, "world.glb"));
const trees = createAxisLockedBillboardSystem(treeAtlas, [0, 1, 0]);
addAxisLockedBillboardSystem(scene, trees);

// Depth-hosted anchored labels: same Sprite2DLayer factory, depth:"test"
// routes it through addDepthHostedSpriteLayer → sprite-renderable, drawn
// inside the scene's 3D pass.
const labels = createSprite2DLayer(labelAtlas, { depth: "test" });
addAnchoredSprite2D(labels, {
    anchor: createWorldAnchor([0, 1.8, 0]),
    sizePx: [128, 32],
    frame: "name-bg",
});
addDepthHostedSpriteLayer(scene, labels);

await registerScene(scene);

// HUD overlay: separate Sprite2DLayer with depth:"none", drawn by a
// separate SpriteRenderer registered AFTER the scene so it draws on top.
// `addToScene` is never called for HUD layers — they have no business
// in the 3D pass.
const hud = createSprite2DLayer(hudAtlas, { depth: "none" });
addSprite2DIndex(hud, { positionPx: [16, 16], sizePx: [200, 32], frame: "score" });
const hudRenderer = createSpriteRenderer(engine, { layers: [hud], clear: false });
registerSpriteRenderer(hudRenderer);
// Tie HUD lifetime to the scene — disposeScene fires this and frees the GPU buffers.
onSceneDispose(scene, () => disposeSpriteRenderer(hudRenderer));

await startEngine(engine);

The punch line: one createSprite2DLayer factory and one public Index API, with the per-layer instance layout fixed at creation. The depth option chooses which path the layer ends up in:

• depth: "none" → caller hands the layer to a SpriteRenderer (pure-2D or HUD); never goes through addToScene. The layer uses a 13-float / 52-byte instance layout and the pure shader keeps clip-space Z constant.
• depth: "test" | "test-write" → caller passes the layer to addDepthHostedSpriteLayer, which statically imports the depth-hosted renderable builder inside the opt-in scene integration module; layer is drawn inside the scene's 3D pass and sorts against meshes by per-instance Z. The layer uses the 14-float / 56-byte layout with slot [13] exposed to the shader as @location(6) iZ.

The world-anchor adapter described below is roadmap. The current depth-hosted slice accepts explicit per-sprite z values and does not expose anchor helpers.

Public API Surface

Shared — Atlas and Frames

// src/sprite/shared/sprite-atlas.ts
import type { EngineContext } from "../../engine/engine.js";
import type { Texture2D, Texture2DOptions } from "../../texture/texture-2d.js";

export type SpriteSampling = "linear" | "nearest";

/** A single frame in an atlas. UVs in [0,1]; pivot in [0,1] of the frame. */
export interface SpriteFrame {
    readonly name?: string;
    readonly uvMin: readonly [number, number];
    readonly uvMax: readonly [number, number];
    readonly sourceSizePx: readonly [number, number];
    readonly pivot: readonly [number, number];
}

export interface SpriteAtlas {
    readonly texture: Texture2D;
    readonly textureSizePx: readonly [number, number];
    readonly frames: readonly SpriteFrame[];
    readonly premultipliedAlpha: boolean;
}

export interface GridAtlasOptions {
    cellWidthPx: number;
    cellHeightPx: number;
    columns?: number;
    rows?: number;
    marginPx?: number;
    spacingPx?: number;
    pivot?: readonly [number, number];
    premultipliedAlpha?: boolean;
}

export interface LoadAtlasOptions {
    gridSize?: readonly [number, number];
    /** Reserved for a future TexturePacker-style JSON loader. Throws today. */
    metadataUrl?: string;
    sampling?: SpriteSampling;
    premultipliedAlpha?: boolean;
    premultiplyOnLoad?: boolean;
    textureOptions?: Texture2DOptions;
}

export function loadSpriteAtlas(engine: EngineContext, textureUrl: string, options?: LoadAtlasOptions): Promise<SpriteAtlas>;
export function createGridSpriteAtlas(texture: Texture2D, options: GridAtlasOptions): SpriteAtlas;
/** @internal */
export function resolveSpriteFrame(atlas: SpriteAtlas, frame: number): number;

A SpriteAtlas is a shared resource: the same atlas may back multiple layers/systems across one or many scenes. Its Texture2D is uploaded once at loadSpriteAtlas. Layers hold a reference; the atlas is released only when no layer holds it (regular Texture2D lifetime).

SpriteFrame.pivot is in normalised [0, 1] of the frame — (0.5, 0.5) centres the quad on the sprite's anchor. Frames are addressed by numeric index today. Named frames, clip animation, and TexturePacker metadata are roadmap additions and must land without adding bytes to numeric-index callers.

Family 1 — Sprite2DLayer (foundation)

// src/sprite/sprite-2d.ts
import type { SpriteAtlas } from "./shared/sprite-atlas.js";
import type { SpriteBlendDescriptor } from "./sprite-blend.js";
import type { Sprite2DCustomShader } from "./sprite-custom-shader.js";

// Blend modes are importable, pure-data descriptor values (see `sprite-blend.ts`), not a
// string union. `SpriteBlendMode` is a type alias of `SpriteBlendDescriptor`; pass one of the
// exported `spriteBlend*` values as `Sprite2DLayerOptions.blendMode`.
export type SpriteBlendMode = SpriteBlendDescriptor;
export type Sprite2DDepthMode = "none" | "test" | "test-write";

export interface Sprite2DView {
    positionPx: [number, number];
    zoom: number;
    rotation: number;
}

export interface Sprite2DLayerOptions {
    capacity?: number;
    blendMode?: SpriteBlendMode;
    opacity?: number;
    visible?: boolean;
    order?: number;
    view?: Partial<Sprite2DView>;
    /**
     * Layer-wide rotation / scaling pivot in normalised sprite-local space
     * (`[0,0]` = top-left, `[0.5, 0.5]` = center, `[1,1]` = bottom-right).
     * Defaults to `[0.5, 0.5]`. Per-sprite pivot is a future PR — most 2D
     * HUD layers want one uniform pivot anyway.
     */
    pivot?: [number, number];
    /**
     * Depth participation:
     *  - "none"        (default) → drawn by a `SpriteRenderer` registered on the engine. Pipeline has no depth attachment; the per-instance layout has no Z slot. HUD overlays must use this mode and live on a `SpriteRenderer` (not on `addDepthHostedSpriteLayer`).
     *  - "test"                  → drawn inside the scene's 3D pass via `addDepthHostedSpriteLayer` with `depthCompare: "greater-equal"`, `depthWrite: false`. Sprites occlude behind 3D geometry but do not write depth.
     *  - "test-write"            → drawn inside the scene's 3D pass via `addDepthHostedSpriteLayer` with `depthCompare: "greater-equal"`, `depthWrite: true`. Sprites direct-draw after cached opaque meshes and before transparent renderables.
     *  Each value is a pipeline-cache key bit, baked at composition time. No runtime branch.
     *  Pure-2D engines (no scene) can only use `"none"` — they have no depth attachment.
     */
    depth?: Sprite2DDepthMode;
    /**
     * Opt-in per-layer custom fragment shader (see `createSprite2DCustomShader`). Works on both
     * pure-2D (`depth: "none"`) and depth-hosted (`depth: "test" | "test-write"`) layers. Drives
     * procedural effects (animated sky, clouds, water/heat shimmer, twinkle, vignette) from a
     * built-in `fx.time` clock plus an optional `fx.params` vec4 set via `setSprite2DShaderParams`.
     * Absent on plain layers, so the always-loaded path carries zero custom-shader bytes (see
     * `sprite-fx-hook.ts`).
     */
    customShader?: Sprite2DCustomShader;
    /**
     * Default per-instance NDC depth (`0` = near, `1` = far) for sprites added to this
     * layer when their `Sprite2DProps.z` is omitted. Only stored and consumed by `depth: "test" |
     * "test-write"` layers; HUD/pure-2D layers use a 13-float layout and allocate no
     * per-instance Z attribute. Defaults to `0.5`. Mutating `layer.layerZ` after
     * sprites have been added does **not** retroactively change them — call
     * `updateSprite2DIndex(layer, idx, { z: … })` to move an existing sprite.
     */
    layerZ?: number;
    /**
     * Opt-in per-sprite UV scroll. When `true`, every sprite gains two extra instance floats
     * (`uvOffset.xy`) added to its sampled UV in the vertex stage — enabling parallax /
     * infinite-scroll backgrounds. Set the offset per sprite via `Sprite2DProps.uvOffset` (on add)
     * or `setSprite2DUvOffset` (live). Layers created without `uvScroll` keep the narrow 13/14-float
     * layout, the base vertex attributes, and the base WGSL — they ship none of the widening; the
     * wider stride, the extra `@location(7)` attribute, and the `+ iUvOffset` WGSL are gated on this
     * flag. Pairs naturally with a tileable atlas sampled in `repeat` wrap mode. Defaults to `false`.
     */
    uvScroll?: boolean;
}

export interface Sprite2DLayer {
    readonly _entityType: "sprite-2d-layer";
    readonly atlas: SpriteAtlas;
    readonly depth: Sprite2DDepthMode;
    readonly blendMode: SpriteBlendMode;
    opacity: number;
    visible: boolean;
    order: number;
    view: Sprite2DView;
    pivot: [number, number];
    /**
     * Opt-in custom fragment shader; see `Sprite2DLayerOptions.customShader`. **Absent** (not
     * `null`) on plain layers — never default-initialized, so the always-loaded path carries zero
     * custom-shader bytes (see `sprite-fx-hook.ts`).
     */
    readonly customShader?: Sprite2DCustomShader;
    /**
     * User `fx.params` vec4 fed to a custom shader each frame; mutate via `setSprite2DShaderParams`.
     * **Absent** on plain layers (allocated only for custom-shader layers, or lazily by the setter).
     */
    shaderParams?: [number, number, number, number];
    /**
     * Opt-in per-sprite UV-scroll flag; see `Sprite2DLayerOptions.uvScroll`. **Absent** (not
     * `false`) on plain layers, so non-scroll scenes keep the narrow layout and base shader. Present
     * (`true`) only when the layer was created with `uvScroll: true`, which widens the instance
     * stride by two floats (`uvOffset.xy`).
     */
    readonly _uvScroll?: boolean;
    /** Default per-instance Z applied to newly added sprites whose `Sprite2DProps.z` is omitted. */
    layerZ: number;
    readonly count: number;
}

export interface Sprite2DProps {
    positionPx: [number, number];
    sizePx?: [number, number];
    /** Atlas-frame index. Pre-resolved at the call site (e.g. via `getSpriteAtlasFrameIndex`). */
    frame?: number;
    rotation?: number;
    color?: [number, number, number, number];
    flipX?: boolean;
    flipY?: boolean;
    visible?: boolean;
    /** Reserved for clip animation (later PR). Accepted but unused today. */
    clip?: unknown;
    /**
     * Per-sprite NDC depth (`0` = near, `1` = far). Only consumed by depth-hosted layers
     * (`depth: "test" | "test-write"`); pure-2D / HUD layers ignore it. When omitted on
     * add, defaults to the owning layer's `layerZ` at the moment of insertion. When omitted
     * on update, the sprite's existing Z is preserved.
     */
    z?: number;
    /**
     * Per-sprite UV-scroll offset, added to the sampled UV in the vertex stage. Only consumed by
     * `uvScroll` layers (created with `Sprite2DLayerOptions.uvScroll: true`); non-scroll layers use
     * the narrow 13/14-float layout and allocate no uvOffset slot. Defaults to `[0, 0]` on add;
     * preserved when omitted on update. Live updates: `setSprite2DUvOffset`.
     */
    uvOffset?: [number, number];
}

export function createSprite2DLayer(atlas: SpriteAtlas, opts?: Sprite2DLayerOptions): Sprite2DLayer;

// Index API — low-level, parallels ThinInstance.
export function addSprite2DIndex(layer: Sprite2DLayer, props: Sprite2DProps): number;
export function updateSprite2DIndex(layer: Sprite2DLayer, index: number, patch: Partial<Sprite2DProps>): void;
export function removeSprite2DIndex(layer: Sprite2DLayer, index: number): void;
export function clearSprite2DLayer(layer: Sprite2DLayer): void;
export function setSprite2DFrameIndex(layer: Sprite2DLayer, index: number, frame: number): void;
// Custom-shader + UV-scroll setters (no-ops / cheap on layers that did not opt in).
export function setSprite2DShaderParams(layer: Sprite2DLayer, params: readonly [number, number, number, number]): void;
export function setSprite2DUvOffset(layer: Sprite2DLayer, index: number, uvOffset: readonly [number, number]): void;

Sprite2D view utilities

sprite-2d-view.ts owns the CPU form of the same transform used by the Sprite2D vertex shader. A view maps its positionPx to logical screen (0, 0), then applies rotation and zoom. Screen dimensions and coordinates are backing-store pixels with a top-left origin. Bounds2D is a reusable math type; the visible-bounds helper writes world-space values into it.

export interface Bounds2D {
    minX: number;
    minY: number;
    maxX: number;
    maxY: number;
}

export function sprite2DWorldToScreenToRef<T extends Vec2>(view: Sprite2DView, worldX: number, worldY: number, result: T): T;
export function sprite2DScreenToWorldToRef<T extends Vec2>(view: Sprite2DView, screenX: number, screenY: number, result: T): T;
export function getSprite2DVisibleBoundsToRef<T extends Bounds2D>(view: Sprite2DView, screenWidthPx: number, screenHeightPx: number, result: T): T;
export function centerSprite2DView<T extends Sprite2DView>(view: T, worldX: number, worldY: number, screenWidthPx: number, screenHeightPx: number): T;

For a SpriteRenderer, and for a depth-hosted layer rendered through a full camera viewport, logical screen coordinates are absolute framebuffer coordinates. Pointer callers first convert CSS pixels by the canvas backing- store scale. A depth-hosted layer under a non-full camera viewport keeps the full render target as its logical screen size because the render pass applies the camera viewport after the Sprite2D vertex transform. Use resolveCameraViewport, then remap an absolute framebuffer point before unprojecting it:

const screenX = ((framebufferX - viewport.x) * targetWidth) / viewport.width;
const screenY = ((framebufferY - viewport.y) * targetHeight) / viewport.height;
sprite2DScreenToWorldToRef(layer.view, screenX, screenY, worldPosition);

The inverse mapping places a projected Sprite2D point in the absolute framebuffer: framebufferX = viewport.x + screenX * viewport.width / targetWidth, with the equivalent expression for Y. Pass the full render-target width and height to getSprite2DVisibleBoundsToRef and centerSprite2DView, not the camera viewport's reduced pixel size.

The ToRef functions write into caller-owned objects, and centering mutates the supplied view, so steady-state camera and culling updates allocate nothing. Negative zoom remains valid and mirrors the renderer. Inverse projection, visible bounds, and centering throw for zoom === 0, where no inverse exists; world-to-screen projection still matches the renderer and collapses to (0, 0).

The module has only type imports and pure functions. Nothing in sprite-2d.ts, sprite-pipeline.ts, or the render loop imports it, so applications that do not reference these helpers retain no runtime bytes or branches from them.

The Handle API (addSprite2D / removeSprite2D, returning a Sprite2DHandle with a stable id) lives in the separately importable sprite-2d-handle.ts module so Index-only scenes do not pull handle code, maps, or typed arrays (see Handles). Observable fields and parenting remain additive follow-up modules.

// src/sprite/sprite-2d-handle.ts — optional, tree-shakable Handle API.
export interface Sprite2DHandle {
    readonly _entityType: "sprite-2d-handle";
    readonly layer: Sprite2DLayer;
    readonly id: number;
}

export function addSprite2D(layer: Sprite2DLayer, props: Sprite2DProps): Sprite2DHandle;
export function updateSprite2D(handle: Sprite2DHandle, patch: Partial<Sprite2DProps>): void;
export function removeSprite2D(handle: Sprite2DHandle): void;
export function setSprite2DFrame(handle: Sprite2DHandle, frame: number): void;
export function getSprite2DHandleIndex(handle: Sprite2DHandle): number;
export function isSprite2DHandleAlive(handle: Sprite2DHandle): boolean;

Optional renderer-native Sprite2D Y-sort

sprite-2d-y-sort.ts is an explicitly imported extension for top-down, isometric, and 2.5D overlap inside one pure-2D layer. It never reorders Sprite2DLayer._instanceData: numeric Index API slots, swap-remove semantics, stable handle ids, and handle-to-index mappings remain canonical. Instead, it owns a reusable GPU-order permutation and packed upload buffer. The renderer uploads the packed representation, while public reads and mutations continue to address logical slots.

The initial support boundary is deliberately depth: "none". Depth-hosted layers can be visually resolved by per-sprite Z, depth writes, and intervening scene geometry, so a CPU Y-order alone cannot define their visually topmost sprite or make pickSprite2D agree with the depth attachment. Enabling Y-sort on depth: "test" or "test-write" therefore throws before allocating state.

// src/sprite/sprite-2d-y-sort.ts — optional Index API extension.
import type { Sprite2DLayer } from "./sprite-2d.js";

/** Options validated on every enable call and applied when creating Y-sort state. */
export interface Sprite2DYSortOptions {
    /** Bias assigned to existing and newly inserted sprites until individually changed. Default 0. */
    defaultBias?: number;
}

/** Layer-owned, pure state returned by `enableSprite2DYSort`. */
export interface Sprite2DYSortState {
    /** Layer whose GPU representation is Y-sorted. */
    readonly layer: Sprite2DLayer;
    /** Bias assigned to existing and future sprites by default. */
    readonly defaultBias: number;
    /** Whether this state remains installed on its layer. */
    readonly enabled: boolean;

    /** @internal Draw slot -> logical slot. */
    _permutation: Uint32Array;
    /** @internal Logical slot -> draw slot. */
    _inversePermutation: Uint32Array;
    /** @internal Stable merge-sort scratch. */
    _mergeScratch: Uint32Array;
    /** @internal Persistent insertion serial per logical slot. */
    _serials: Float64Array;
    /** @internal Per-logical-slot bias. */
    _biases: Float64Array;
    /** @internal Cached `positionPx.y + bias` per logical slot. */
    _keys: Float64Array;
    /** @internal Packed GPU-order instance records. */
    _packedInstances: Float32Array;
    /** @internal Capacity represented by the metadata arrays. */
    _capacity: number;
    /** @internal Instance-record float width represented by `_packedInstances`. */
    _packedStride: number;
    /** @internal Number of logical slots currently represented by metadata. */
    _activeCount: number;
    /** @internal Next persistent insertion serial. */
    _nextSerial: number;
    /** @internal Whether `_permutation` must be recomputed. */
    _sortDirty: boolean;
    /** @internal Whether the next upload must repack every active record. */
    _fullUpload: boolean;
    /** @internal Minimum dirty packed draw slot, inclusive. */
    _dirtyMin: number;
    /** @internal Maximum dirty packed draw slot, exclusive. */
    _dirtyMax: number;
}

/** Enable stable ascending Y-sort. Valid repeated calls return the installed state unchanged. */
export function enableSprite2DYSort(layer: Sprite2DLayer, options?: Sprite2DYSortOptions): Sprite2DYSortState;

/** Disable Y-sort, release its state, and mark canonical logical order for upload. */
export function disableSprite2DYSort(layer: Sprite2DLayer): boolean;

/** Set the finite ordering bias for one current logical Index API slot. */
export function setSprite2DYSortBias(layer: Sprite2DLayer, index: number, bias: number): void;

// src/sprite/sprite-2d-handle-y-sort.ts — optional Handle API companion.
import type { Sprite2DHandle } from "./sprite-2d-handle.js";

/** Resolve the handle's current logical slot, then set its finite Y-sort bias. */
export function setSprite2DYSortHandleBias(handle: Sprite2DHandle, bias: number): void;

defaultBias is validated on every enable call, including calls made while state is already installed. Every setter bias must also be finite; NaN, positive/negative infinity, and out-of-range indices throw without changing state. A valid repeated enable is idempotent and returns the installed state; its original defaultBias remains first-options-wins. The handle setter uses getSprite2DHandleIndex, so a removed handle throws with the same stable-handle error as other handle mutations. Keeping the handle companion in its own module means Index-only Y-sort users do not import handle maps.

The draw key is ascending (positionPx.y + bias, insertionSerial) under the module's +Y-down layer-space convention: smaller Y is uploaded first, larger Y is uploaded later and composites on top. insertionSerial is monotonically assigned when a sprite enters the enabled layer. It is metadata for the live sprite, not its current logical slot; swap-remove moves the serial and bias with the last sprite into the vacated slot. Equal-key ties therefore retain their original insertion order even after unrelated removals. Enabling an already populated layer assigns serials in its current logical order. clearSprite2DLayer clears active metadata but does not rewind the serial counter.

Hidden sprites retain their key, bias, serial, and position in the permutation. Their packed size remains zero, so they rasterize no fragments; restoring visibility cannot perturb equal-key neighbors. Layer ordering is unchanged: SpriteRenderer still sorts whole layers by layer.order, and Y-sort applies only inside each enabled layer.

The implementation module registers one lazy null hook when the first layer is enabled. The neutral hook slot and opaque contract are co-located in sprite-2d.ts, which is already required by every layer consumer; this avoids retaining a separate hook module in Rolldown builds that do not enable Y-sort. The always-loaded mutation, upload, and picker modules know only that opaque contract; all state field names, key semantics, sorting code, and typed-array allocations remain in sprite-2d-y-sort.ts. Importing or root-exporting the enabler without using it creates no state and performs no module-level allocation.

Mutation behavior is exact:

1. Enabling allocates permutation, inverse, merge scratch, serial, bias, key, and packed staging arrays at the layer's current capacity, initializes all live metadata, and marks the whole layer dirty.
2. Add grows all arrays only when layer capacity grows, assigns the next serial and default bias, then invalidates order.
3. Any dirty logical range is inspected after the canonical write. A changed Y key invalidates order. Color, UV, frame, size, X-only position, rotation, visibility, and Z-independent changes leave order intact and map the logical dirty slots through _inversePermutation into a packed draw-slot dirty range.
4. Bias changes recompute one key and invalidate order only when that key changes.
5. Swap-remove moves bias, serial, and cached key from last to index, clears the retired tail metadata, and invalidates order. Canonical instance and handle storage keep their existing swap-remove behavior.
6. Clear zeroes the former active metadata and count-dependent permutation state, preserving allocated capacity and the next insertion serial.

Disabling marks the returned state enabled: false, removes the layer's state attachment so its arrays can be released, and dirties the full live range. While the layer remains disabled, its next upload therefore restores canonical logical order. Re-enabling does not resume the old state: it creates fresh arrays, assigns every active sprite the new defaultBias, assigns insertion serials 0..count - 1 in current logical order, and sets the next serial to count. Thus disable/re-enable deliberately resets per-sprite bias overrides and serial history, while valid repeated enables preserve both unchanged.

When order is dirty, a bottom-up stable merge sort fills _permutation by key then serial, rebuilds _inversePermutation, packs every active record into _packedInstances, and performs one full queue.writeBuffer. When order is unchanged, only the packed draw-slot interval reached through the inverse map is refreshed and uploaded. Sorting, packing, and partial updates allocate nothing in steady state: all loops copy scalar lanes into reused typed arrays and never create per-sprite subarray views. A structural instance-stride opt-in such as UV scroll may replace the packed buffer once to match the new record width; ordinary operation grows storage only with layer capacity.

The sorted staging path is inside shared uploadSpriteInstances, so both renderer resource growth and device recovery remain automatic. A newly created GPU instance buffer passes uploadedVersion === -1, which always causes a full repack and sorted upload from CPU state. The disabled path's full canonical dirty range lets the same helper restore ordinary logical-order upload without a special upload API.

pickSprite2D asks the optional hook for current draw order before traversing a layer. The hook sorts the CPU permutation if a same-frame Y/bias mutation made it stale, without packing or touching the GPU. Picking then walks draw slots in reverse and returns the mapped logical slot, so the result agrees with rendered topmost order, including persistent-serial ties. Layers themselves are still tested in reverse array draw order.

No explicit GPU disposal exists because the extension owns no GPU handles. The state is attached to, and has the same GC lifetime as, its caller-owned layer. disableSprite2DYSort is the explicit early-release/behavior-off switch; callers that drop the layer can simply drop the returned state as well.

Optional NPE → Sprite2D bridge integration

particle/particle-sprite-2d.ts is an opt-in consumer of the Sprite2D foundation. It creates one centered depth: "none" layer per NPE particle system and renders those layers through a caller-owned SpriteRenderer. The particle module owns coordinate, sprite-sheet, blend, synchronization, registration, and disposal policy; the sprite module continues to own layer storage and rendering. The complete bridge contract lives in 42-node-particle.md.

particle/particle-sprite-2d-blend-modes.ts is a separate particle-owned opt-in for exact NPE blend modes. It reuses ordinary Sprite2DLayers and the existing custom-fragment API; mode 4 registers two equal-order layers consecutively so the renderer's stable sort preserves Multiply then Add. Generic Sprite2D does not interpret particle blend numbers, animate particle systems, or own multipass policy, and the baseline bridge does not import this advanced module. The exact API, shader formula, lifecycle, and tests live in 42-node-particle.md.

A bridge exclusively owns its layer's complete live range. Each synchronization writes NPE columns directly into the existing 13-float pure-2D instance layout and saved-size buffer, then performs one count update and at most one dirty-range update. A bridge-owned layer cannot mix independent Index API sprites and cannot install the optional Handle API; both would violate full-range ownership. Render it through SpriteRenderer, not the depth-hosted scene path.

The dependency remains one-way: both particle bridge modules import Sprite2D modules, and no sprite module imports either bridge. Although the package root exports both APIs, side-effect-free tree-shaking means static Sprite2D users that do not import them pay zero bridge bytes, while baseline bridge users pay no exact descriptor, custom-shader, or second-pass bytes.

Blend modes — tree-shakable descriptors

Blend mode is not a string union backed by a static lookup table. Each mode is an importable, pure-data descriptor value, so a scene ships only the descriptor(s) it imports — a default (alpha) scene references spriteBlendAlpha and nothing else; importing spriteBlendAdditive does not drag in spriteBlendPremultiplied. Adding a future blend mode costs zero bytes to scenes that don't use it. The pipeline reads _descriptor / _key / _premultipliedOpacity directly off the value, so there is no runtime string switch for the bundler to retain. The descriptors are byte-identical GPUBlendState to the old table, so visual parity is unchanged.

// src/sprite/sprite-blend.ts
export interface SpriteBlendDescriptor {
    /** @internal Pipeline-cache discriminator (distinguishes blend variants of one pipeline). */
    readonly _key: string;
    /** @internal Color-target blend state; `undefined` means no color blend (opaque). */
    readonly _descriptor?: GPUBlendState;
    /** @internal When true, per-layer opacity scales RGB *and* A (premultiplied fade). */
    readonly _premultipliedOpacity?: boolean;
}

export const spriteBlendOpaque: SpriteBlendDescriptor; // replacement color; no blend
export const spriteBlendAlpha: SpriteBlendDescriptor; // straight-alpha "over" (default)
export const spriteBlendPremultiplied: SpriteBlendDescriptor; // premultiplied "over"
export const spriteBlendAdditive: SpriteBlendDescriptor; // glows/sparks: src*alpha + dst
export const spriteBlendOneOne: SpriteBlendDescriptor; // pure additive: src + dst
export const spriteBlendMultiply: SpriteBlendDescriptor; // shadow/tint: result = src * dst

// src/sprite/billboard-blend.ts — mirrors the above for world-space billboards.
export interface BillboardBlendDescriptor extends SpriteBlendDescriptor {
    /** @internal Depth/blend pipeline path this mode selects ("transparent" | "cutout"). */
    readonly _depthMode: BillboardDepthMode;
}

export const billboardBlendAlpha: BillboardBlendDescriptor;
export const billboardBlendPremultiplied: BillboardBlendDescriptor;
export const billboardBlendCutout: BillboardBlendDescriptor; // alpha-test, depth-writing
export const billboardBlendAdditive: BillboardBlendDescriptor; // transparent, no depth write

Sprite2D supports opaque, alpha, premultiplied, additive, one-one, and multiply descriptors. spriteBlendAdditive uses color factors src-alpha / one, so source RGB is weighted by source alpha before it is added to the destination. spriteBlendOneOne uses one / one for both color and alpha, so source RGB is added without alpha weighting. Its color factors match Babylon.js BLENDMODE_ONEONE / ALPHA_ONEONE, but its generic Sprite2D alpha factors are one / one; exact particle OneOne instead uses zero / one. Every operation in both additive descriptors is add.

Billboards support alpha, premultiplied, cutout, and additive publicly — multiply is intentionally not offered for billboards. The shared blend-descriptors.ts holds the two common states (_ALPHA_BLEND_STATE, _PREMULTIPLIED_BLEND_STATE) so alpha/premultiplied don't duplicate bytes.

Per-layer custom fragment shaders

A layer/system may supply its own WGSL fragment body while the engine keeps ownership of the transform, instancing, sorting, and depth. The factory returns an opaque, compiled-on-demand descriptor passed as customShader:

// src/sprite/sprite-custom-shader.ts
export type Sprite2DCustomTexture = CustomShaderTexture; // becomes `<name>Tex` + `<name>Samp` in WGSL
export interface Sprite2DCustomShaderOptions {
    readonly fragment: string; // WGSL fragment body
    readonly extraTextures?: readonly Sprite2DCustomTexture[];
}
export function createSprite2DCustomShader(options: Sprite2DCustomShaderOptions): Sprite2DCustomShader;

// src/sprite/billboard-custom-shader.ts — parallel API for world-space billboards.
export function createBillboardCustomShader(options: BillboardCustomShaderOptions): BillboardCustomShader;

The supplied fragment body has in scope: in: VOut (a 2D layer exposes uv: vec2<f32> and tint: vec4<f32>; a billboard additionally exposes viewDist / worldPos); the layer atlas as atlasTex / atlasSamp at bindings 1/2; each extra texture as <name>Tex / <name>Samp; the fx UBO (fx.time, fx.params); and the L layer UBO (e.g. L.opacityMul). It must return vec4<f32>(...) and may discard; the body owns all alpha handling (no per-layer opacity is applied automatically). The constant fx.params vec4 is fed each frame via setSprite2DShaderParams(layer, params) / setBillboardShaderParams(system, params). Runtime WGSL identifier re-mangling keeps user fragment code working in minified bundles.

Tree-shaking via a lazy null registry (sprite/sprite-fx-hook.ts). This is the architectural crux, and it mirrors the PBR extension registry (pbr-flags.ts). The module declares two hook interfaces (SpriteFxHook, BillboardFxHook) whose methods (initLayer, pipelineKeyPart, shaderModule, layoutEntries, createLayerFx, updateFx, bindEntries, disposeFx) each take the layer/system opaquely, so every layer.customShader / layer.shaderParams property read happens inside the tree-shaken impl. It holds two module-level slots (let _spriteFxHook: SpriteFxHook | null = null, _billboardFxHook) and the _registerSpriteFxHook / _getSpriteFxHook / _registerBillboardFxHook / _getBillboardFxHook functions — with no module-level side effects. The always-loaded sprite / billboard / pipeline / renderer modules only ever call _getSpriteFxHook()?.method(...); the slot stays null until createSprite2DCustomShader / createBillboardCustomShader registers the impl. So a plain sprite scene ships zero custom-shader tokens — not even the public customShader / shaderParams field-name strings. sprite-pipeline.ts's spritePipelineKey reaches the feature only through _getSpriteFxHook()?.pipelineKeyPart(layer) (the cs${customKey} segment). The 2D and billboard composers share their mechanics via custom-shader-core.ts (extra-texture bindings, name validation, the SpriteFx UBO, key allocation) but keep their own vertex stage and varying contract.

Opt-in per-sprite UV scroll (parallax)

uvScroll is a structural opt-in, not a per-sprite value: enabling it changes the GPU data layout and the compiled shader, so it follows the same pay-for-use gate as depth. When a layer is created with uvScroll: true, every sprite gains two extra instance floats (uvOffset.xy) that are added to the sampled UV in the vertex stage — enabling parallax / infinite-scroll backgrounds without re-uploading texture coordinates. The offset is set per sprite via Sprite2DProps.uvOffset (on add) or setSprite2DUvOffset(layer, index, uvOffset) (live).

Enabling the flag (1) widens the instance stride by two floats, (2) adds a @location(7) iUvOffset: vec2<f32> vertex attribute, (3) selects a distinct WGSL variant (let uv = mix(...) + in.iUvOffset), and (4) adds a :uv${uvKey} segment to the pipeline key so variants don't collide. The widening is orthogonal to depth and lands after the base layout:

pure-2D + uvScroll  (15 floats = 60 bytes):  [13..14] uvOffset.xy (float32x2 @ byte offset 52)
depth   + uvScroll  (16 floats = 64 bytes):  [14..15] uvOffset.xy (float32x2 @ byte offset 56)

(Source constants: PURE_2D_UVSCROLL_STRIDE_BYTES, DEPTH_UVSCROLL_STRIDE_BYTES, SPRITE_UVOFFSET_OFFSET_PURE_2D_BYTES = 52, SPRITE_UVOFFSET_OFFSET_DEPTH_BYTES = 56.) Layers created without uvScroll keep the narrow 13/14-float layout, the base attributes, and the base WGSL — they ship none of the widening. The feature pairs naturally with a tileable atlas sampled in repeat wrap mode (loadSpriteAtlas(..., { textureOptions: { addressModeU: "repeat", addressModeV: "repeat" } })).

Roadmap — AnchorSource opt-in 3D bridge for Sprite2DLayer

The APIs in this section are not part of the current root exports.

// src/sprite/anchor/sprite-anchor.ts — separate module, dynamic-imported on first use.
import type { Sprite2DLayer, Sprite2DProps } from "../sprite-2d.js";
import type { SceneContext } from "../../scene/scene-core.js";
import type { IWorldMatrixProvider } from "../../scene/parenting.js";

export interface AnchorSource {
    readonly project: (outPx: Float32Array, outZ: Float32Array, scene: SceneContext) => boolean;
}

/** Static world-space anchor. */
export function createWorldAnchor(worldPos: [number, number, number]): AnchorSource;

/** World anchor that follows a moving entity (mesh, transform node, sprite handle). */
export function createParentAnchor(parent: IWorldMatrixProvider, localOffset?: [number, number, number]): AnchorSource;

/** Attach an AnchorSource to a sprite. The sprite's positionPx is overwritten each frame
 *  by the projection result. Layer must have depth !== "none" for occlusion against 3D geometry. */
export interface AnchoredSprite2DInit extends Sprite2DProps {
    anchor: AnchorSource;
    offsetPx?: [number, number];
    /** NDC-z bias added after projection (positive = pushed toward camera). Default 0. */
    depthBias?: number;
}

export function addAnchoredSprite2D(layer: Sprite2DLayer, init: AnchoredSprite2DInit): number;
export function setSprite2DAnchor(layer: Sprite2DLayer, index: number, anchor: AnchorSource | null): void;

The first call to addAnchoredSprite2D (or setSprite2DAnchor with a non-null anchor) on a given layer:

1. Lazy-allocates a sparse Map<number, AnchoredEntry> on the layer (sprites without an anchor have no entry).
2. Installs a per-frame hook into scene._beforeRender (via unshift, so it runs before user onBeforeRender callbacks) that walks the layer's anchored map, calls each anchor.project(), and writes the resulting positionPx, optional layerZ (mapped from view-Z), and depthBias-adjusted ordering into the layer's flat storage via the same code path updateSprite2DIndex uses. Sprites whose project returns false get sizePx = [0, 0] written into their slot (degenerate quad — same trick as visible: false).
3. Registers a single disposable that removes the hook when the layer is disposed or its anchored map becomes empty.

// In sprite-anchor.ts internal:
interface AnchoredEntry {
    anchor: AnchorSource;
    offsetPx: [number, number];
    depthBias: number;
}

A scene that has zero anchored sprites never imports sprite-anchor.ts, never allocates the sparse map, never installs the projection hook, and never pays for viewProjection on the CPU.

Family 2 — *BillboardSpriteSystem

The current implementation slice is deliberately narrower than the full billboard roadmap: it ships Facing and Axis-Locked billboards (with yaw-locked as a special case of axis-locked using [0, 1, 0]), with the low-level index API, explicit scene opt-in helpers, and CPU-side transparent sorting for the current compact vertex-buffer upload. It also ships the production cutout path: alpha-tested, depth-writing billboard systems selected via billboardBlendCutout, plus additive blending via billboardBlendAdditive and opt-in per-system custom fragment shaders via createBillboardCustomShader (see Per-layer custom fragment shaders). Clip playback, observable handle fields, parenting, picking, storage-buffer sort indirection, and a billboard multiply blend mode are additive follow-up modules. Stable handle identity itself lives in billboard-sprite-handle.ts. That split keeps the first billboard path small and keeps pure-2D sprite bundles from importing scene rendering code.

// src/sprite/billboard-sprite.ts
import type { SpriteAtlas } from "./shared/sprite-atlas.js";
import type { BillboardBlendDescriptor } from "./billboard-blend.js";
import type { BillboardCustomShader } from "./billboard-custom-shader.js";

export interface BillboardSpriteSystemOptions {
    capacity?: number;
    blendMode?: BillboardBlendMode;
    alphaCutoff?: number;
    opacity?: number;
    visible?: boolean;
    order?: number;
    /** Opt-in per-system custom fragment shader; see `createBillboardCustomShader`. */
    customShader?: BillboardCustomShader;
}

// Blend mode is a descriptor value (see `billboard-blend.ts`), not a string. Billboards accept
// alpha / premultiplied / cutout / additive; `multiply` is intentionally not offered.
export type BillboardBlendMode = BillboardBlendDescriptor;
export type BillboardOrientation = "facing" | "axis-locked";
export type BillboardDepthMode = "transparent" | "cutout";

export interface BillboardSpriteSystem<TOrientation extends BillboardOrientation = BillboardOrientation> {
    readonly _entityType: "billboard-sprite-system";
    readonly atlas: SpriteAtlas;
    readonly blendMode: BillboardBlendMode;
    alphaCutoff: number;
    opacity: number;
    visible: boolean;
    readonly order: number;
    readonly count: number;

    readonly _orientation: TOrientation;
    readonly _depthMode: BillboardDepthMode;
    readonly _axis: readonly [number, number, number];
    _capacity: number;
    readonly _instanceFloatsPerSprite: number;
    readonly _instanceStrideBytes: number;
    _instanceData: Float32Array;
    _savedSize: Float32Array;
    _version: number;
    _dirtyMin: number;
    _dirtyMax: number;
}

export type FacingBillboardSpriteSystem = BillboardSpriteSystem<"facing">;
export type AxisLockedBillboardSpriteSystem = BillboardSpriteSystem<"axis-locked">;

export interface BillboardSpriteInit {
    position: [number, number, number];
    sizeWorld: [number, number];
    frame?: number;
    rotation?: number;
    pivot?: [number, number];
    color?: [number, number, number, number];
    flipX?: boolean;
    flipY?: boolean;
    visible?: boolean;
}

export function createFacingBillboardSystem(atlas: SpriteAtlas, opts?: BillboardSpriteSystemOptions): FacingBillboardSpriteSystem;
export function createAxisLockedBillboardSystem(atlas: SpriteAtlas, axis: readonly [number, number, number], opts?: BillboardSpriteSystemOptions): AxisLockedBillboardSpriteSystem;

// Index API — low-level, parallels ThinInstance and existing Sprite2DLayer index calls.
export function addBillboardSpriteIndex(system: BillboardSpriteSystem, init: BillboardSpriteInit): number;
export function updateBillboardSpriteIndex(system: BillboardSpriteSystem, index: number, patch: Partial<BillboardSpriteInit>): void;
export function removeBillboardSpriteIndex(system: BillboardSpriteSystem, index: number): void;
export function clearBillboardSprites(system: BillboardSpriteSystem): void;
export function setBillboardSpriteFrameIndex(system: BillboardSpriteSystem, index: number, frame: number): void;

// src/sprite/billboard-sprite-handle.ts — optional, tree-shakable Handle API.
export interface BillboardSpriteHandle {
    readonly _entityType: "billboard-sprite-handle";
    readonly system: BillboardSpriteSystem;
    readonly id: number;
}

export function addBillboardSprite(system: BillboardSpriteSystem, init: BillboardSpriteInit): BillboardSpriteHandle;
export function updateBillboardSprite(handle: BillboardSpriteHandle, patch: Partial<BillboardSpriteInit>): void;
export function removeBillboardSprite(handle: BillboardSpriteHandle): void;
export function setBillboardSpriteFrame(handle: BillboardSpriteHandle, frame: number): void;
export function getBillboardSpriteHandleIndex(handle: BillboardSpriteHandle): number;
export function isBillboardSpriteHandleAlive(handle: BillboardSpriteHandle): boolean;

// src/sprite/billboard-scene.ts
export function addFacingBillboardSystem(scene: SceneContext, system: FacingBillboardSpriteSystem): void;
export function addAxisLockedBillboardSystem(scene: SceneContext, system: AxisLockedBillboardSpriteSystem): void;

BillboardSpriteSystem.order is construction-time state: the renderable copies it when the scene helper builds GPU resources during registerScene. Create a new system when the render order must change after registration.

setSprite2DFrameIndex and setBillboardSpriteFrameIndex are UV-only helpers. They preserve the sprite's explicit sizePx/sizeWorld even when the target atlas frame has a different sourceSizePx. Callers that want atlas-driven size changes should use updateSprite2DIndex(layer, index, { frame, sizePx }) or updateBillboardSpriteIndex(system, index, { frame, sizeWorld }) with an explicit size policy. For billboards, pixel frame size never implies a world-space size. setBillboardSpriteFrameIndex also preserves the sprite's current pivot; callers that want atlas-driven pivot changes during animation should use updateBillboardSpriteIndex(system, index, { frame, pivot }).

Flip state is encoded by swapping UV min/max endpoints, so frame setters preserve flip state for non-degenerate atlas frames (uvMin.x !== uvMax.x and uvMin.y !== uvMax.y). Zero-area atlas frames cannot encode flipped and unflipped states distinctly; callers should avoid authoring collapsed UV ranges or pass explicit flipX/flipY through the full update helpers when recovering from such data.

addFacingBillboardSystem(scene, system) and addAxisLockedBillboardSystem(scene, system) are the scene integration points. They queue a deferred renderable builder through addDeferredSceneRenderables so scene-core and addToScene stay sprite-agnostic. The builder dynamically imports billboard-renderable.ts, which imports billboard-pipeline.ts, getSceneBindGroupLayout, and the WGSL composer. A scene that never queues a billboard system never runtime-fetches that chunk. The helper signatures use orientation-specific system types so normal TypeScript callers cannot pass an axis-locked system to the facing helper or vice versa.

Internally, the shared renderable routes through system._orientation and system._depthMode; the current public factories set "facing" or "axis-locked", and each blend descriptor carries its own _depthMode — "transparent" for alpha / premultiplied / additive, and "cutout" for billboardBlendCutout. Transparent systems are alpha-blended, depth-tested, depth-write disabled, maintain a _worldCenter for the scene's transparent bucket sort, and sort individual billboard instances far-to-near before upload. _worldCenter is the center of the active anchor bounds, not the arithmetic mean, because it is a proxy for the system footprint when the render task sorts this one renderable against other transparent renderables. The render pass runs binding updates before transparent-bucket sorting so _worldCenter reflects any same-frame billboard mutations before draw order is chosen. Cutout systems have no blend state, discard fragments whose sampled texture alpha is below alphaCutoff, write depth, and route through the non-transparent direct-draw bucket with _direct rather than _transmissive.

Yaw-locked billboards (world-Y axis constraint) are created via createAxisLockedBillboardSystem(atlas, [0, 1, 0], opts).

Picking

The picking APIs below are exported from the package root (pickSprite2D, pickBillboardSprite, and the SpritePickInfo / BillboardPickInfo result types).

// src/sprite/picking/pick-sprite-2d.ts — handles every Sprite2DLayer
// reachable from the scene (depth-hosted layers added via addToScene)
// AND the layers of any SpriteRenderer the caller hands in.
export function pickSprite2D(layers: ReadonlyArray<Sprite2DLayer>, xPx: number, yPx: number): SpritePickInfo | null;

// src/sprite/picking/pick-billboard.ts — GPU picker into the shared mesh pick pass.
// An optional caller-owned `picker` (from createGpuPicker) is reused across calls for
// high-frequency picking; when omitted, a transient picker is created and disposed per call.
export function pickBillboardSprite(scene: SceneContext, xPx: number, yPx: number, picker?: GpuPicker): Promise<BillboardPickInfo | null>;

pickSprite2D is the pure-2D picker: the caller passes whichever Sprite2DLayer array it owns — spriteRenderer.layers for HUD overlays, or scene._renderables (filtered to the Sprite2D Renderables that addToScene pushed there). The picker iterates in reverse insertion order and, for each candidate sprite, rotates the screen point into its pivot-aware local rectangle. For anchored layers positionPx has already been projected CPU-side this frame, so the picker hits the same screen rectangle the GPU draws. No GPU pick pass for Sprite2D.

pickBillboardSprite draws into the shared mesh pick pass on the GPU; the full design is specified under Picking below.

Pure-2D usage — no scene

A pure-2D app skips the scene entirely. It creates a SpriteRenderer, registers it on the engine, and lets startEngine drive the loop:

import { createEngine, loadSpriteAtlas, createSprite2DLayer, addSprite2DIndex, createSpriteRenderer, registerSpriteRenderer, startEngine } from "@babylonjs/lite";

const engine = await createEngine(canvas);
const atlas = await loadSpriteAtlas(engine, "sprites.png", { gridSize: [32, 32] });
const layer = createSprite2DLayer(atlas);
addSprite2DIndex(layer, { positionPx: [100, 200], sizePx: [64, 64], frame: 0 });

const sr = createSpriteRenderer(engine, {
    layers: [layer],
    clearValue: { r: 0, g: 0, b: 0, a: 1 },
});
registerSpriteRenderer(sr);

await startEngine(engine);

Nothing in src/scene/* is reachable from this code path — not even scene-core.ts. Anchored sprites are unsupported here (they require a camera).

Internal Architecture

Core Rule: No if Across Modes (still)

There is still no shared createSprite(), no SpriteMode enum, no per- frame if (sprite.kind === ...). The two families have separate composers, separate renderables, separate WGSL. The unification happens at the scene layer, not at the sprite-shader layer. The AnchorSource projection is a CPU step on a sparse per-layer map; it writes into the owning layer's already-fixed instance layout (52 B for pure-2D, 56 B for depth-hosted).

Per-Instance GPU Layout

Sprite2DLayer fixes its instance stride at creation from depth. depth: "none" uses 52 bytes / 13 floats and has no Z lane. depth: "test" | "test-write" uses 56 bytes / 14 floats and exposes slot [13] as the per-instance depth attribute. Anchor data lives off-instance in a sparse JS map. Per-layer constants (view, screen size, pivot, opacity) live in a separate 48-byte UBO bound at @group(0) @binding(0) for the pure renderer and @group(1) @binding(0) for the depth-hosted scene renderable.

Sprite2DLayer pure per-instance vertex buffer (52 B = 13 floats)

Sprite2DLayer depth-hosted extension (56 B = 14 floats)

Depth-hosted layers use the same first 52 bytes, plus:

Sprite2DLayer uvScroll extension (opt-in; +8 B = +2 floats)

Layers created with uvScroll: true append two more floats (uvOffset.xy) after the base layout, orthogonally to depth. The wider stride, the extra @location(7) attribute, and the + iUvOffset WGSL are gated on the per-layer flag, so non-scroll scenes ship none of it.

The vertex stage adds in.iUvOffset to the sampled UV (let uv = mix(uvMin, uvMax, corner) + in.iUvOffset), and the pipeline key gains a :uv${uvKey} segment so scroll/non-scroll variants never collide.

Visibility (visible: false) is implemented by zeroing slots [2..3]; the sprite's true size lives in layer._savedSize so a later visible: true (without re-supplying sizePx) can restore it.

Per-instance Z, not per-layer, for depth-hosted layers. A single depth-hosted layer can mix sprites at different depths — useful where one sprite goes in front of a 3D object and another behind it. Pure-2D layers do not allocate, upload, declare, or fetch a Z slot; z remains accepted by the public props shape but is ignored before storage.

Sprite2DLayer Layer UBO (48 B = 12 floats; std140-aligned)

Bound at @group(0) @binding(0). Updated each frame from current view + target dimensions; written via device.queue.writeBuffer only when the 12-float scratch buffer differs from the previously-written copy.

Total 48 bytes — vec4<f32> forces 16-byte struct alignment and 48 = 3 × 16 is naturally aligned; no padding required.

Current BillboardSpriteSystem instance layout (64 B = 16 floats)

The first facing-billboard slice uses a compact per-instance vertex buffer, not a storage buffer or sort indirection. The buffer is bound as the sole vertex buffer with stepMode: "instance"; the shader uses @builtin(vertex_index) for the four quad corners and drawIndexed(6, system.count, 0, 0, 0) for all active sprites.

position is the only field with a CPU-side shadow beyond _savedSize. system._anchor holds the same three components in F64, 24 B per sprite, maintained in lockstep with _instanceData by writeInstance, growCapacity, removeBillboardSpriteIndex and clearBillboardSprites.

It exists because floating origin cannot rescue a value that was already quantized. The eye-relative subtraction in billboard-pipeline.ts must read a source that still has the low bits, so it reads _anchor; subtracting from the F32 _instanceData afterwards would recover nothing, because the detail was lost one step earlier at the F32 store. The comment on that subtraction notes anchors "live at world scale (~5e6)", where the F32 ULP is 0.5 m — comparable to a whole particle sprite. This mirrors the rule packMat4IntoF32WithOffset follows for meshes (36-high-precision-matrix.md): subtract in F64, store once in F32.

The shadow is allocated unconditionally, not gated on useFloatingOrigin. A sprite system is created standalone and registered into a scene later, so createBillboardSystem has no engine to ask, and the anchor must already be F64-accurate by the time the first sprite is written — too early to gate on anything. Non-FO scenes therefore pay 24 B per sprite of CPU memory (against the existing 64 B instance record) for a value they never read. They pay no per-frame cost: the FO upload path already copied every instance into scratch each frame, because every anchor depends on the live camera offset, and the non-FO path keeps its partial dirty-range uploads straight from _instanceData.

_instanceData still carries the F32 position, and picking (billboard-pick-pipeline.ts) and the bounding centre (refreshBillboardWorldCenter) both continue to read it. Both work at world scale, where F32 is proportionate.

The current system UBO is 32 bytes:

• opacityMul at byte offset 0: vec4<f32>. Straight-alpha and cutout write (1, 1, 1, opacity); premultiplied writes (opacity, opacity, opacity, opacity) so the shader is branch-free.
• axisAndCutoff at byte offset 16: vec4<f32>. xyz is the normalized axis for axis-locked systems and zero for facing systems. w is the system alphaCutoff used only by cutout shaders.

The render pipeline uses the scene UBO at group 0 and the billboard UBO/atlas bind group at group 1, with pipeline keys including render target format, sample count, _orientation, blend mode, _depthMode, and depth format. The shader module cache also keys by _depthMode: transparent shaders have no discard path, while cutout shaders sample texture alpha, discard below billboards.axisAndCutoff.w, and return the sampled color multiplied by tint and opacityMul. The "transparent" depth mode uses depth compare greater-equal, depth write off, no culling, and alpha, premultiplied, or additive blending. The "cutout" depth mode uses depth compare greater-equal, depth write on, no blend state, and no culling. The unsupported billboard multiply blend mode throws during system creation.

Transparent billboard systems sort per billboard before upload, not by mutating system._instanceData. When DrawUpdateContext supplies the active pass camera, the billboard renderable gets the cached camera view matrix and fills renderable-owned scratch arrays: view-space depths, stable index order, and a sorted 64-byte-instance staging buffer. Depth is the view-space z of each anchor (view[2] * x + view[6] * y + view[10] * z + view[14]), sorted descending so farther billboards upload first. The same compact GPU vertex buffer is written from the sorted staging buffer with one full queue.writeBuffer when either the system version, camera identity, or camera.worldMatrixVersion changes. Repeated frames with unchanged system data and unchanged camera view skip the instance upload. Cutout billboard systems skip this transparent sort path entirely and upload dirty ranges directly from system._instanceData in logical insertion order, matching the direct depth-write bucket semantics used by depth-hosted sprite layers.

This camera-dependent transparent sort is intentionally conservative: when the camera moves, relative billboard order may change, so the renderable recomputes depths and uploads the sorted compact buffer. Large transparent systems should be split by region or depth band if exact interleaving with other transparent renderables matters; one billboard system remains one transparent renderable in the global scene sort.

Future target BillboardSpriteSystem (96 B = 24 floats)

The later handle/picking/storage-indirection roadmap moves billboard data to a storage buffer at @group(1) @binding(3) (not a vertex buffer — 3D sprite families read sprite data through a storage buffer indexed by a sort-indirection vertex attribute, see below). The 24-float layout:

The lock axis (axis-locked variant only) lives in the system UBO, not per-sprite. The reserved slots at floats 3..5 stay in the layout because the same packed-buffer layout and pack helper signature is shared with the depth-hosted Sprite2DLayer's anchored-write path (which uses those slots for depthBias and offsetPx); for billboard-only systems the CPU pack helper writes 0.0.

Sort Indirection + Storage Buffer

The future storage-buffer path keeps the same public invariant as the current CPU-staging path: billboard systems never reorder the packed sprite buffer. Sorting is expressed entirely through a separate Uint32Array indirection buffer of sprite indices, uploaded once per frame as a per-instance vertex attribute at @location(0). The shader reads sortIndex and indexes into the packed sprite storage buffer to fetch the actual record. This keeps sort upload cost O(N), not O(N × stride).

Packed sprite buffer. Allocated by sprite-gpu.ts with usage: GPUBufferUsage.VERTEX | GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST.

Sort indirection vertex buffer. Per-instance Uint32 buffer at @location(0) with stepMode: "instance", arrayStride: 4, attribute format uint32. One u32 per active sprite. Recreated when storage capacity grows.

Storage buffer binding. Bound at @group(1) @binding(3) as var<storage, read> sprites: array<SpriteData>. Bind-group layout entry uses buffer: { type: "read-only-storage" } with GPUShaderStage.VERTEX visibility. The renderable rebuilds the layer bind group lazily — only when system._storage.gpuBuffer (the JS pointer) changes between frames (capacity grew, or first sync).

Shared WGSL. sprite/shared/sprite-3d-instance-wgsl.ts exports two TS string consts that every billboard variant shader includes:

// SPRITE_3D_DATA_WGSL — 96 B / 24-float storage record.
struct SpriteData {
    worldPos: vec3<f32>,
    depthBias_or_reserved: f32,        // anchored: depthBias; billboard: 0
    offsetPx_or_reserved: vec2<f32>,   // anchored: offsetPx; billboard: (0,0)
    sizePxOrWorld: vec2<f32>,          // anchored: sizePx;   billboard: sizeWorld
    pivot: vec2<f32>,
    sinCos: vec2<f32>,
    uvRect: vec4<f32>,
    color: vec4<f32>,
    flagsAndPad: vec4<f32>,            // .x flipX, .y flipY, .z reserved, .w reserved
};
@group(1) @binding(3) var<storage, read> sprites: array<SpriteData>;

// SPRITE_3D_VS_IN_WGSL — input/output structs + helpers.
struct VSIn {
    @builtin(vertex_index) vid: u32,
    @location(0) sortIndex: u32,
};
struct VSOut {
    @builtin(position) pos: vec4<f32>,
    @location(0) uv: vec2<f32>,
    @location(1) color: vec4<f32>,
};
fn rotate2(p: vec2<f32>, sinCos: vec2<f32>) -> vec2<f32> { /* ... */ }
fn cornerOf(vid: u32) -> vec2<f32> { /* 6-corner triangle list */ }
fn cornerUV(corner: vec2<f32>, rect: vec4<f32>, flipX: f32, flipY: f32) -> vec2<f32> { /* ... */ }

SpriteData field names are deliberately unified (depthBias_or_reserved, offsetPx_or_reserved, sizePxOrWorld) so a single struct definition serves both billboard variants and the anchored-write path inside Sprite2DLayer. Billboard shaders ignore depthBias_or_reserved and read sizePxOrWorld as world size.

Re-sort triggers. A re-sort runs only when at least one of the following changed since the last sync:

• _sortVersion (bumped by add / remove / position update).
• Camera world-position (only matters for blended systems — cutout systems do not back-to-front sort).
• Sprite count (forces re-upload after grow).

Cutout vs. blended. Cutout systems always emit a sequential 0..N-1 indirection (no per-frame back-to-front cost) so the shader path stays uniform. Blended systems use insertion sort over squared camera distance — fast for small N and near-sorted lists, which is the typical case as the camera moves smoothly.

SpriteSortState (lives in sprite/shared/sprite-sort.ts):

export interface SpriteSortState {
    indexBuffer: GPUBuffer | null;
    indices: Uint32Array;
    distances: Float32Array;
    lastSortVersion: number;
    lastCamX: number;
    lastCamY: number;
    lastCamZ: number;
    lastUploadedCount: number;
    blended: boolean;
    boundsCenter: [number, number, number];
}

Bounds center for engine-wide transparent sort. computeSpriteBoundsCenter(state, storage) walks the first three floats of every active slot, computes the center of the active anchor bounds, writes it into state.boundsCenter, and returns it. The renderable copies this into Renderable._worldCenter every frame so the engine-wide transparent sort orders billboard systems correctly against transparent meshes.

Helpers exported by sprite-sort.ts:

• createSpriteSortState(blended) — allocate state. GPU buffer is created lazily on first sync.
• syncSpriteSortIndices(engine, state, storage, sortVersion, camX, camY, camZ, label) — ensures capacity, runs sort if any trigger fired, uploads via a single writeBuffer.
• computeSpriteBoundsCenter(state, storage) — center of active anchor bounds.
• disposeSpriteSortState(state) — release the GPU index buffer.

Vertexless Quad

No vertex buffer for positions. Six invocations per instance from @builtin(vertex_index) (triangle list):

const QUAD_CORNERS: array<vec2<f32>, 6> = array<vec2<f32>, 6>(
    vec2<f32>(0, 0), vec2<f32>(1, 0), vec2<f32>(1, 1),
    vec2<f32>(0, 0), vec2<f32>(1, 1), vec2<f32>(0, 1),
);

Draw call: pass.draw(6, batch.count) with topology: 'triangle-list'. Triangle-list (not triangle-strip) eliminates a class of corner-case driver differences across WebGPU implementations.

CPU → GPU Sync (sprite-gpu.ts)

Each layer/system owns a single Float32Array packed buffer sized at capacity × stride. On per-frame sync:

1. If _version === _gpuVersion, skip.
2. Otherwise, walk [dirtyMin, dirtyMax] and for each dirty slot pack the 20- or 24-float record. Resolve frame to UV rect via atlas.frames[frameIndex].
3. Single device.queue.writeBuffer(_gpuBuffer, dirtyMin*stride, _data.buffer, dirtyMin*stride, (dirtyMax - dirtyMin + 1) * stride).
4. _gpuVersion = _version.

Capacity grows 2× on overflow (fresh allocation + copy). The renderable's GPU buffer reference is rebuilt internally on grow and the new buffer is rebound at the next frame's draw() — callers hold no GPU buffer handles, so no caller action is required. Sprite indices remain stable across grows. Removal is swap-remove (last slot moves into the gap; that slot's _dirty is bumped). Same pattern as mesh/thin-instance.ts.

This module is dynamically imported by every family renderable, so a 2D-only scene does not bundle billboard or anchored code.

Anchor projection feeds the dirty-range mechanism via the same updateSprite2DIndex write path used by every other update. Anchor sprites whose projected position changes every frame (the common case) are effectively a full re-upload of the anchored sprites' contiguous slot range each frame — same cost profile as a per-frame-moving particle layer. Static anchors (parent never moves, camera never moves) skip upload via the _version === _gpuVersion short-circuit.

Dirty / Version Tracking

Visibility (visible: false)

Toggling visible: false on a sprite does not compact the array or shift indices. The pack step writes sizePx = [0, 0] (or sizeWorld = [0, 0]) into the slot; the vertex shader collapses all six vertices to a single point and the rasterizer emits zero fragments. Indices stay stable, sort order is unaffected, and toggling visibility is just a regular update*({ visible }) call that bumps _version. Trade-off: invisible sprites still cost their stride bytes in the per-frame upload range. For layers with dense visibility churn (rare in practice), split into two layers instead.

Hook Registration Order

Per-layer animation/clip ticks AND the per-layer anchor-projection hook both register into scene._beforeRender via unshift, so they run before any user onBeforeRender callback. This is required by the freeze-flag contract: applications that drive deterministic capture (e.g. seekTime reference scenes) advance N frames and then set a freeze flag in their own onBeforeRender; that callback must observe the fully-advanced clip state on the freeze frame, otherwise the layer loses one tick of animation in the captured image. All sprite families (Sprite2D, anchored Sprite2D, Billboard) share this convention.

Pipeline Configuration

Shared Across All Layers

Sprite2DLayer per-depth Pipeline State

The sprite pipeline cache key includes (format, sampleCount, blendMode, hasDepth, depthWrite, depthStencilFormat), plus a cs${customKey} segment for custom-shader layers (contributed opaquely via _getSpriteFxHook()?.pipelineKeyPart(layer)) and a :uv${uvKey} segment for uvScroll layers. SpriteRenderer layers always request hasDepth = false and sampleCount = 1, so their pipelines are built without a depth-stencil descriptor. Depth-hosted layers request hasDepth = true, use the target depth-stencil format provided by the frame graph, and set depthWrite from the layer's depth mode.

Bind Group Layouts

Sprite2DLayer uses the Layer UBO described in the per-instance section above: 48 bytes at the sprite bind group's binding 0, plus atlas texture and sampler at bindings 1 and 2. Pure SpriteRenderer layers bind that group at group 0. Depth-hosted Sprite2D renderables bind the scene's existing group 0 and the sprite layer group at group 1.

Billboards do not allocate a separate sprite-only scene UBO in the current implementation. The render pass task already owns the canonical scene UBO and binds it at group 0; billboard WGSL reads scene.viewProjection and scene.view from that existing binding. Mesh-only scenes still pay zero billboard bytes because the billboard renderable/pipeline module is behind the dynamic import in billboard-scene.ts.

The billboard bind group is group 1:

BillboardSystem.axisAndCutoff.xyz is the normalized lock axis for axis-locked systems and zero for facing systems. axisAndCutoff.w is the runtime alphaCutoff consumed only by cutout shaders. opacityMul is CPU-shaped per blend mode: straight-alpha and cutout write (1, 1, 1, opacity), while premultiplied writes (opacity, opacity, opacity, opacity) so the fragment shader has no blend-mode branch.

Pipeline Cache

Per-device, lazy. Current keys:

• Sprite2DLayer: (format, sampleCount, blendMode, hasDepth, depthWrite, depthStencilFormat).
• BillboardSpriteSystem: (format, sampleCount, orientation, blendMode, depthMode, depthStencilFormat).

alphaCutoff and opacity are not in the billboard pipeline key. Both live in the 32-byte system UBO, so animating opacity or tuning cutoff is a UBO write, not a pipeline recompile. flipX / flipY are stored by swapping UV min/max in the per-instance data, not by changing pipelines.

Shader Logic

Composers (one per family / billboard variant) emit complete WGSL strings. Three composers total: composeSprite2D (covers both pure-2D and anchored layers — the WGSL is identical), composeFacingBillboard, and composeAxisLockedBillboard.

Sprite2DLayer Vertex Shader (covers pure 2D AND anchored)

@group(0) @binding(0) var<uniform> scene: Sprite2DSceneUBO;
@group(1) @binding(2) var<uniform> layer: SpriteLayerUBO;

@vertex fn vs(in: VSIn) -> VSOut {
    let corner = cornerOf(in.vid);
    let localPx = (corner - in.pivot) * in.sizePx;
    let rotated = rotate2(localPx, in.sinCos);
    let layerPx = in.positionPx + rotated;
    let sc = vec2<f32>(sin(scene.viewRotation), cos(scene.viewRotation));
    let viewed = rotate2(layerPx - scene.viewPositionPx, sc) * scene.zoom;
    // PIXEL_SNAP: composer emits floor(viewed + 0.5) when pixelSnap is true.
    let snapped = viewed;
    let ndc = vec2<f32>(
         snapped.x * scene.invViewportPx.x * 2.0 - 1.0,
        1.0 - snapped.y * scene.invViewportPx.y * 2.0,
    );
    // For depth: "none" layers, z is ignored. For depth: "test" / "test-write",
    // layerZ ∈ [0,1] is mapped to NDC depth ∈ [1,0]. The CPU anchor sync writes
    // the projected NDC-z (with depthBias applied) into in.layerZ for anchored sprites.
    let z = 1.0 - clamp(in.layerZ, 0.0, 1.0);
    var out: VSOut;
    out.pos = vec4<f32>(ndc, z, 1.0);
    out.uv = cornerUV(corner, in.uvRect, in.flipX > 0.5, in.flipY > 0.5);
    out.color = in.color;
    return out;
}

Crucially: in.positionPx already carries the projected layer-space pixel for anchored sprites, written by the CPU sync hook before this frame's GPU upload. The shader has no idea whether the sprite is anchored. There is no if (anchored), no per-instance world-position field, and no wasted bytes for non-anchored sprites.

Billboard Vertex Shaders

The current billboard path uses a 64-byte per-instance vertex buffer and @builtin(vertex_index) for the four quad corners. The shared composer emits one of two basis functions from system._orientation; the rest of the vertex shader is identical.

Facing (spherical) basis

fn getBillboardBasis(_anchor: vec3<f32>) -> BillboardBasis {
  let cameraRight = normalize(vec3<f32>(scene.view[0][0], scene.view[1][0], scene.view[2][0]));
  let cameraUp = normalize(vec3<f32>(scene.view[0][1], scene.view[1][1], scene.view[2][1]));
  return BillboardBasis(cameraRight, -cameraUp);
}

Axis-Locked basis

fn getBillboardBasis(_anchor: vec3<f32>) -> BillboardBasis {
  let lockAxis = normalize(billboards.axisAndCutoff.xyz);
  let cameraRight = normalize(vec3<f32>(scene.view[0][0], scene.view[1][0], scene.view[2][0]));
  let projectedRight = cameraRight - lockAxis * dot(cameraRight, lockAxis);
  let projectedRightLen = length(projectedRight);
  let safeProjectedRightLen = max(projectedRightLen, 1e-4);
  let fallbackSeed = select(vec3<f32>(0.0, 0.0, 1.0), vec3<f32>(1.0, 0.0, 0.0), abs(lockAxis.z) > 0.999);
  let fallbackRightRaw = cross(lockAxis, fallbackSeed);
  let fallbackRight = fallbackRightRaw / max(length(fallbackRightRaw), 1e-4);
  let right = select(fallbackRight, projectedRight / safeProjectedRightLen, projectedRightLen > 1e-4);
  return BillboardBasis(right, -lockAxis);
}

Shared vertex body

@vertex
fn vs(in: VIn) -> VOut {
  let corner = vec2<f32>(select(0.0, 1.0, in.vid == 1u || in.vid == 2u), select(0.0, 1.0, in.vid >= 2u));
  let local = (corner - in.iPivot) * in.iSize;
  let cosRot = cos(in.iRot);
  let sinRot = sin(in.iRot);
  let rotated = vec2<f32>(local.x * cosRot - local.y * sinRot, local.x * sinRot + local.y * cosRot);
  let basis = getBillboardBasis(in.iPos);
  let worldPos = in.iPos + basis.right * rotated.x + basis.up * rotated.y;
  var out: VOut;
  out.pos = scene.viewProjection * vec4<f32>(worldPos, 1.0);
  out.uv = mix(in.iUvMin, in.iUvMax, corner);
  out.tint = in.iColor;
  return out;
}

Billboard Fragment Shader

Transparent billboard shaders sample the atlas and return sampleColor * tint * billboards.opacityMul. Cutout billboard shaders use the same multiply but first discard fragments whose sampled alpha is below billboards.axisAndCutoff.w.

@fragment
fn fs(in: VOut) -> @location(0) vec4<f32> {
  let sampleColor = textureSample(atlasTex, atlasSamp, in.uv);
  if (sampleColor.a < billboards.axisAndCutoff.w) {
    discard;
  }
  return sampleColor * in.tint * billboards.opacityMul;
}

alphaCutoff and opacity both live in the billboard system UBO. Neither value is baked into WGSL or entered into the pipeline cache key.

Sorting and Transparency

Depth-hosted Sprite2D layers do not Y-sort sprites individually — each layer becomes one Renderable and the GPU's depth test resolves overlap between sprites in the same layer (cutout) or between layers that share the depth buffer. Within a depth-hosted layer, sprites draw in insertion order, and the per-instance Z (slot [13]) is used as the depth-test value. Pure-2D layers have no slot [13]; they retain logical order unless the optional Y-sort extension supplies a packed GPU-order representation. Current transparent billboard systems sort into renderable-owned scratch buffers and upload the sorted 64-byte instances without mutating system._instanceData. Cutout billboard systems skip the sort and upload dirty ranges in logical insertion order.

Picking

Two pickers cover the two families, each matched to where its family lives:

• pickSprite2D is pure CPU. Sprite2DLayer sprites are screen-space rectangles, and pure-2D apps have no scene, camera, or depth target for a GPU pass to hook into.
• pickBillboardSprite is GPU. Billboards live in the scene's 3D depth buffer, so picking one must respect occlusion by meshes and nearer billboards; it draws into the same gpu-picker.ts 1×1 pick pass the mesh picker uses.

pickSprite2D — CPU rectangle test

pickSprite2D(layers, xPx, yPx) takes the caller's Sprite2DLayer array (spriteRenderer.layers for HUD overlays, or the Sprite2D Renderables a scene holds) and walks it in reverse draw order: last layer first, and within a layer the last GPU draw slot first, so the topmost-drawn sprite wins. An ordinary layer maps draw slot directly to logical slot; a Y-sorted layer maps through its current stable permutation and still returns the canonical logical slot. Hidden sprites (visible: false, stored as a zero-size quad) are skipped. For each candidate it rotates the query point into the sprite's pivot-aware local rectangle — inverting the vertex shader's pivot offset and rotation — and a point that lands in [0, 1]² is a hit. It returns the first hit as SpritePickInfo { layer, spriteIndex, u, v }, where (u, v) is the within-quad fraction at the hit point, or null. (xPx, yPx) are in the layers' shared local space (the same space as each sprite's positionPx); for a panned/zoomed layer the caller maps the pointer into that space first. There is no global registry — the caller chooses what to pick by passing the right layer array.

pickBillboardSprite — GPU pick into the shared mesh pass

pickBillboardSprite(scene, xPx, yPx, picker?) runs one pickAsync and returns the billboard payload that pass attaches on a hit, or null. By default it creates a transient GpuPicker and disposes it per call, so a one-off pick (a click) retains no GPU state; for high-frequency picking (hover, drag) the caller passes a reusable GpuPicker from createGpuPicker, so the 1×1 targets, scene UBO, and per-source contributors (which compile the pick pipelines) are reused across calls and the caller disposes it once. Billboards are drawn into the same 1×1 pick render pass as meshes, sharing the pick-ID colour target, the r32float depth target, and the greater reverse-Z depth test — so a billboard occluded by a mesh or a nearer billboard loses the pick.

Billboards plug into the picker through the generic PickContributor seam (registerPickSource / scene._pickSources) — the seam itself is documented in 18-picking.md. addFacingBillboardSystem / addAxisLockedBillboardSystem register one pick source per billboard system in billboard-scene.ts. A source is pure data + a dynamic-import thunk, so billboard rendering pulls no pick code; only on the first pick does the picker import billboard-pick-pipeline.ts, build the contributor via createPickContributor, and cache its per-picker GPU resources in the contributor closure.

Draw + pick-ID ranges. On draw(pickCtx, baseId) the billboard contributor consumes system.count pick IDs — even for a hidden or empty system, so the id-to-sprite mapping stays positional — and, when there is something to draw, calls drawBillboardSystemForPicking, which rebinds @group(0) to the shared pick scene UBO (an earlier contributor may have rebound group 0) and issues the draw.

Per-system 48-byte pick UBO (BILLBOARD_PICK_UBO_BYTES = 48), packed by packBillboardPickUbo; the float layout matches the WGSL BB struct (vec3 + scalar interleave):

camRight / camUp are the camera basis rows, reproducing the render shader's normalize(view rowN) so the picked pixel matches the drawn quad. No separate billboard scene UBO is bound in the pick pass.

Bind groups. @group(0) = the shared pick scene UBO (the same one mesh picking uses). @group(1) @binding(0) = the per-system pick UBO; cutout systems additionally bind the atlas texture@1 + sampler@2 for the alpha-test discard. Per-instance data is uploaded to a vertex buffer in logical (un-sorted) order, so pickId − baseId is the sprite index directly; a shared two-triangle index buffer ([0, 1, 2, 0, 2, 3]) draws each quad. A system's instance buffer grows if the system outgrows it between picks.

Pipeline cache. Keyed "${orientation}|${isCutout ? 1 : 0}" — at most four entries (facing / axis-locked × cutout flag). Each pipeline bakes the variant's basis math in WGSL: facing uses normalize(camRight) / normalize(camUp); axis-locked projects camRight off the normalized lock axis. The fragment writes the pick ID as RGB at @location(0) and NDC depth at @location(1), matching the mesh picker's two-attachment contract; cutout pipelines discard below cutoff. Depth state is depth24plus, compare greater, write enabled.

Resolution. When the picker reads back a pick ID that the billboard contributor owns, it calls the contributor's resolve(info, spriteIndex), which sets info._spritePick = { system, spriteIndex, pickedPoint, distance } (the local id is the sprite index directly). The picked point and distance come from the shared pick-depth readback (the same world-point reconstruction the mesh picker performs, done once before resolve); there is no per-billboard CPU ray test and no per-hit UV reconstruction.

State Machine / Lifecycle

Atlas + Layer Creation

loadSpriteAtlas(engine, url, opts) → SpriteAtlas

createSprite2DLayer(atlas, { depth })
  └─> { atlas, depth, capacity, _data (Float32Array), _animations,
        _anchored: null,                                      // sparse map; null until first anchor
        _deferredBuild,
        _version, _gpuVersion, _entityType: "sprite-2d-layer" }

createAxisLockedBillboardSystem(atlas, axis, opts)
  └─> { ..., _entityType: "billboard-sprite-system", _orientation: "axis-locked", _axis: normalized(axis), ... }

Depth-Hosted Scene Admission

Depth-hosted sprite admission lives in the sprite module as an opt-in addDepthHostedSpriteLayer helper. Scene core exposes only a generic addDeferredSceneRenderables hook; addToScene remains sprite-agnostic. Renderable construction and depth-mode validation stay in sprite-renderable.ts:

addDepthHostedSpriteLayer(scene, layer)
  └─> rejects depth === "none"
  └─> addDeferredSceneRenderables(scene, builder)
  └─> call statically imported buildSpriteRenderable
        └─> buildSpriteRenderable(engine, layer)
        └─> push renderable/disposable into the scene

Pure-2D-only apps never call addToScene at all — they create a SpriteRenderer directly, register it on the engine, and own their layers without any scene. HUD-on-3D apps hand their HUD layers to a separate SpriteRenderer and register it on the engine after registerScene; depth-hosted layers go through addDepthHostedSpriteLayer.

Build (at registerScene)

registerScene(scene) runs each _deferredBuild. The sprite deferred builder calls the statically imported buildSpriteRenderable, builds the pipeline (cache-keyed), allocates the per-layer GPU instance buffer + UBO, and creates bind groups. The depth-hosted Renderable (one per Sprite2D layer added through addDepthHostedSpriteLayer) is pushed into scene._renderables with order = 100 and _direct = true for depth: "test-write", or order = 200 for blended / depth: "test", so the scene's existing renderable loop picks it up alongside opaque and transparent meshes.

registerScene does not create or own any SpriteRenderer. HUD overlays are entirely caller-managed.

Per-Frame Render

startEngine(engine) per-frame:
  1. Acquire swap-chain view + create command encoder.
  2. For each context c in engine._renderingContexts (in registration order):
       a. Pre-pass updates: c._update()
            Scene context:
              - Run scene._beforeRender hooks: clip ticks; anchor projection writes positionPx.
              - Run pre-pass renderables and scene uniform updaters.
            SpriteRenderer:
              - For each dirty layer: upload the canonical dirty range, or let the
                optional Y-sort hook pack/upload its GPU-order range; update layer UBO.
       b. Record draws: c._record()
            Scene context:
              - Execute its frame graph. The default swapchain RenderTask
                buckets and draws renderables sorted by `order`:
                  * meshes opaque (order 0)
                  * Sprite2D depth-hosted cutout / `"test-write"` (order 100)
                  * meshes transparent
                  * Sprite2D depth-hosted blended / `"test"` (order 200)
                  * billboards (order 200)
            SpriteRenderer:
              - Open a sprite-only sampleCount=1 pass directly on the
                per-frame swapchain view with no depth attachment.
              - For each layer in this.layers (sorted by layer.order):
                  bind pipeline + groups; drawIndexed(6, count).
  3. Submit command buffer.

No if (is2D) anywhere. The engine just walks its _renderingContexts list and asks each one to update and record. A SpriteRenderer with zero non-empty layers skips its pass entirely. A scene with no Sprite2D renderables draws only meshes and billboards.

Disposal

disposeScene(scene) invokes every callback in scene._disposables, including the per-renderable GPU buffer / bind group / pipeline cleanups, the per-layer anchor hook removal, and the per-target depth/MSAA attachment releases (existing code; sprites add no new attachments).

Any HUD SpriteRenderer the caller registered on the engine is separately the caller's responsibility to dispose. The recommended pattern is to wire HUD disposal to the scene lifetime via onSceneDispose(scene, () => disposeSpriteRenderer(hud)) after registerSpriteRenderer, so a single disposeScene(scene) call also takes the HUD down.

disposeSpriteRenderer(sr) releases the renderer's pipeline cache (per- device, max four entries), any per-renderer UBO, and is safe to call without having called unregisterSpriteRenderer first — it does both.

Y-sort state is CPU-only and layer-owned, not renderer-owned. Renderer disposal therefore does not disable or destroy it: the same layer can be attached to a new renderer and its next uploadedVersion === -1 sync rebuilds the sorted GPU representation. disableSprite2DYSort(layer) drops the attachment and forces a canonical-order upload while disabled when callers want to release the arrays before dropping the layer. A later enable creates new state with new-default biases and insertion serials restarted from zero; it never revives released state.

Handles, Identity, and Parenting

Stable handle identity is implemented as an optional layer over the Index API. Parenting and observable field objects remain roadmap extensions.

Sprites in Babylon Lite use a two-tier API that mirrors the Index/Handle split common in data-oriented engines (and parallels Lite's ThinInstance vs. Mesh split for 3D geometry).

Two-tier API design