2 yıl önce · 0bcc426d7d
--- a/README.md
+++ b/README.md
  - [@threepipe/plugins-extra-importers](#threepipeplugins-extra-importers) - Plugin for loading more file types supported by loaders in three.js
  - [@threepipe/plugin-blend-importer](#threepipeplugin-blend-importer) - Blender to add support for loading .blend file
  - [@threepipe/plugin-geometry-generator](#threepipeplugin-geometry-generator) - Generate parametric geometry types that can be re-generated from UI/API.
  - [@threepipe/plugin-gaussian-splatting](#threepipeplugin-gaussian-splatting) - Gaussian Splatting plugin for loading and rendering splat files
 ## Getting Started
 generator.uiConfig.uiRefresh?.()
 ```
 ## @threepipe/plugin-gaussian-splatting
 Exports [GaussianSplattingPlugin](https://threepipe.org/plugins/gaussian-splatting/docs/classes/GaussianSplattingPlugin.html) which adds support for loading .blend files.
 It uses [`three-gaussian-splat`](./plugins/gaussian-splatting/src/three-gaussian-splat), a rewrite of [@zappar/three-guassian-splat](https://github.com/zappar-xr/three-gaussian-splat) (and [gsplat.js](https://github.com/huggingface/gsplat.js) and [antimatter15/splat](https://github.com/antimatter15/splat)) for loading splat files and rendering gaussian splats.
 [Example](https://threepipe.org/examples/#splat-load/) &mdash;
 [Source Code](./plugins/gaussian-splatting/src/index.ts) &mdash;
 [API Reference](https://threepipe.org/plugins/gaussian-splatting/docs)
 NPM: `npm install @threepipe/plugin-gaussian-splatting`
 Note: This is still a WIP.
 Currently working:
 * Importing .splat files (just array buffer of gaussian splat attributes)
 * ThreeGaussianSplatPlugin (Same as GaussianSplattingPlugin), add importer and update events to the viewer
 * GaussianSplatMaterialExtension for adding gaussian splat functionality to any material like Unlit, Physical
 * GaussianSplatMesh a subclass of Mesh2 for holding the gaussian splat geometry and a material with gaussian splat extension. also handles basic raycast in the splat geometry. (assuming simple points)
 * GaussianSplatGeometry holds the geometry data and and the sort worker. Computes correct bounding box and sphere.
 * SplatLoader for loading splat files and creating the geometry and material.
 * GaussianSplatMaterialUnlit, GaussianSplatMaterialRaw
 * GaussianSplatMaterialPhysical, working but normals are hardcoded to 0,1,0
 TBD: 
 * Exporting/embedding splat files into glb
 * Rendering to depth/gbuffer
 * Estimate normals/read from file
 * Lighting in GaussianSplatMaterialPhysical
 ```typescript
 import {ThreeViewer} from 'threepipe'
 import {GaussianSplattingPlugin} from '@threepipe/plugin-gaussian-splatting'
 const viewer = new ThreeViewer({...})
 viewer.addPluginSync(GaussianSplattingPlugin)
 // Now load any .splat file.
 const model = await viewer.load<GaussianSplatMesh>('path/to/file.splat')
 ```
--- a/examples/index.html
+++ b/examples/index.html
            <li><a href="./ktx2-load/">KTX2 Load </a></li>
            <li><a href="./ktx-load/">KTX Load </a></li>
            <li><a href="./blend-load/">BLEND Load </a></li>
            <li><a href="./splat-load/">SPLAT Load<br/>(Gaussian Splatting) </a></li>
            <li><a href="./extra-importer-plugins/">Extra(3ds, 3mf, collada, amf, bvh, vox, gcode, mdd, pcd, tilt, wrl, ldraw, vtk, xyz) Load </a></li>
        </ul>
        <h2 class="category">Export</h2>
--- a/examples/splat-load/index.html
+++ b/examples/splat-load/index.html
 <!DOCTYPE html>
 <html lang="en">
 <head>
    <meta charset="UTF-8">
    <title>Blend Load</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <!-- Import maps polyfill -->
    <!-- Remove this when import maps will be widely supported -->
    <script async src="https://unpkg.com/es-module-shims@1.6.3/dist/es-module-shims.js"></script>
    <script type="importmap">
    {
        "imports": {
          "threepipe": "./../../dist/index.mjs",
          "@threepipe/plugin-gaussian-splatting": "./../../plugins/gaussian-splatting/dist/index.mjs"
        }
    }
    </script>
    <style id="example-style">
        html, body, #canvas-container, #mcanvas {
            width: 100%;
            height: 100%;
            margin: 0;
            overflow: hidden;
        }
    </style>
    <script type="module" src="../examples-utils/simple-code-preview.mjs"></script>
    <script id="example-script" type="module">
        import {_testFinish, ThreeViewer} from 'threepipe'
        import {GaussianSplattingPlugin} from '@threepipe/plugin-gaussian-splatting'
        const viewer = new ThreeViewer({canvas: document.getElementById('mcanvas')})
        viewer.addPluginsSync([GaussianSplattingPlugin])
        async function init() {
            await viewer.setEnvironmentMap('https://threejs.org/examples/textures/equirectangular/venice_sunset_1k.hdr')
            const result = await viewer.load('https://asset-samples.threepipe.org/splat/bonsai.splat', {
                autoCenter: true,
                autoScale: true,
            })
            console.log(result)
        }
        init().then(_testFinish)
    </script>
 </head>
 <body>
 <div id="canvas-container">
    <canvas id="mcanvas"></canvas>
 </div>
 </body>
--- a/examples/tweakpane-editor/index.html
+++ b/examples/tweakpane-editor/index.html
          "@threepipe/plugin-tweakpane-editor": "./../../plugins/tweakpane-editor/dist/index.mjs",
          "@threepipe/plugin-extra-importers": "./../../plugins/extra-importers/dist/index.mjs",
          "@threepipe/plugin-blend-importer": "./../../plugins/blend-importer/dist/index.mjs",
          "@threepipe/plugin-geometry-generator": "./../../plugins/geometry-generator/dist/index.mjs"
          "@threepipe/plugin-geometry-generator": "./../../plugins/geometry-generator/dist/index.mjs",
          "@threepipe/plugin-gaussian-splatting": "./../../plugins/gaussian-splatting/dist/index.mjs"
        }
    }
--- a/examples/tweakpane-editor/script.ts
+++ b/examples/tweakpane-editor/script.ts
 import {BlendLoadPlugin} from '@threepipe/plugin-blend-importer'
 import {extraImportPlugins} from '@threepipe/plugin-extra-importers'
 import {GeometryGeneratorPlugin} from '@threepipe/plugin-geometry-generator'
 import {GaussianSplattingPlugin} from '@threepipe/plugin-gaussian-splatting'
 async function init() {
        GeometryGeneratorPlugin,
        Object3DWidgetsPlugin,
        Object3DGeneratorPlugin,
        GaussianSplattingPlugin,
        ...extraImportPlugins,
    ])
--- a/plugins/gaussian-splatting/package-lock.json
+++ b/plugins/gaussian-splatting/package-lock.json
 {
    "name": "@threepipe/plugin-gaussian-splatting",
    "version": "0.1.0",
    "lockfileVersion": 3,
    "requires": true,
    "packages": {
        "": {
            "name": "@threepipe/plugin-gaussian-splatting",
            "version": "0.1.0",
            "license": "Apache-2.0",
            "dependencies": {
                "threepipe": "file:./../../src/"
            },
            "devDependencies": {
                "@types/emscripten": "^1.39.10",
                "comlink": "^4.4.1"
            }
        },
        "../../../three-gaussian-splat": {
            "name": "@zappar/three-gaussian-splat",
            "version": "0.0.1",
            "extraneous": true,
            "license": "MIT",
            "dependencies": {
                "@types/emscripten": "^1.39.10",
                "comlink": "4.4.1"
            },
            "devDependencies": {
                "@types/node": "^14.11.2",
                "@types/three": "0.150.1",
                "gts": "^3.1.1",
                "lil-gui": "^0.18.2",
                "parcel": "^2.10.3",
                "path-browserify": "^1.0.1",
                "process": "^0.11.10",
                "three": "^0.150.1",
                "typescript": "^4.9.4"
            },
            "engines": {
                "node": ">=18.0.0"
            },
            "peerDependencies": {
                "three": "*"
            }
        },
        "../../src": {},
        "node_modules/@types/emscripten": {
            "version": "1.39.10",
            "resolved": "https://registry.npmjs.org/@types/emscripten/-/emscripten-1.39.10.tgz",
            "integrity": "sha512-TB/6hBkYQJxsZHSqyeuO1Jt0AB/bW6G7rHt9g7lML7SOF6lbgcHvw/Lr+69iqN0qxgXLhWKScAon73JNnptuDw==",
            "dev": true
        },
        "node_modules/comlink": {
            "version": "4.4.1",
            "resolved": "https://registry.npmjs.org/comlink/-/comlink-4.4.1.tgz",
            "integrity": "sha512-+1dlx0aY5Jo1vHy/tSsIGpSkN4tS9rZSW8FIhG0JH/crs9wwweswIo/POr451r7bZww3hFbPAKnTpimzL/mm4Q==",
            "dev": true
        },
        "node_modules/threepipe": {
            "resolved": "../../src",
            "link": true
        }
    }
 }
--- a/plugins/gaussian-splatting/package.json
+++ b/plugins/gaussian-splatting/package.json
 {
    "name": "@threepipe/plugin-gaussian-splatting",
    "description": "Gaussian Splatting for Threepipe",
    "version": "0.1.0",
    "devDependencies": {
        "comlink": "^4.4.1",
        "@types/emscripten": "^1.39.10"
    },
    "dependencies": {
        "threepipe": "file:./../../src/"
    },
    "clean-package": {
        "remove": [
            "clean-package",
            "scripts",
            "devDependencies",
            "//",
            "markdown-to-html"
        ],
        "replace": {
            "dependencies": {
                "threepipe": "^0.0.23"
            }
        }
    },
    "type": "module",
    "main": "dist/index.js",
    "module": "dist/index.mjs",
    "types": "dist/index.d.ts",
    "files": [
        "dist",
        "src"
    ],
    "scripts": {
        "new:pack": "npm run prepare && clean-package && npm pack && clean-package restore",
        "new:publish": "npm run prepare && clean-package && npm publish --access public && clean-package restore",
        "prepare": "npm run build",
        "build": "rimraf dist && vite build",
        "dev": "NODE_ENV=development vite build --watch",
        "docs": "rimraf docs && npx typedoc"
    },
    "author": "repalash <palash@shaders.app>",
    "license": "Apache-2.0",
    "keywords": [
        "three",
        "three.js",
        "threepipe",
        "gaussian-splatting",
        "ml",
        "ai"
    ],
    "bugs": {
        "url": "https://github.com/repalash/threepipe/issues"
    },
    "homepage": "https://github.com/repalash/threepipe#readme",
    "repository": {
        "type": "git",
        "url": "git://github.com/repalash/threepipe.git"
    }
 }
--- a/plugins/gaussian-splatting/src/gaussian-splats-3d/GaussianSplatsPlugin.ts
+++ b/plugins/gaussian-splatting/src/gaussian-splats-3d/GaussianSplatsPlugin.ts
 // import {AViewerPluginSync, createStyles, IViewerEvent, ThreeViewer} from 'threepipe'
 // import styles from './SamplePlugin.css'
 // import * as GaussianSplats3D from '@mkkellogg/gaussian-splats-3d'
 //
 // // import * as GaussianSplats3D from 'gle-gs3d'
 //
 // export class GaussianSplatsPlugin extends AViewerPluginSync<string> {
 //     public static readonly PluginType: string = 'GaussianSplatsPlugin'
 //     enabled = true
 //     dependencies = []
 //     toJSON: any = null
 //
 //     constructor() {
 //         super()
 //     }
 //
 //     splats: any
 //     private _ready = false
 //     onAdded(viewer: ThreeViewer) {
 //         super.onAdded(viewer)
 //         createStyles(styles)
 //         this.splats = new GaussianSplats3D.Viewer({
 //             'selfDrivenMode': false,
 //             'renderer': viewer.renderManager.webglRenderer,
 //             'camera': viewer.scene.mainCamera,
 //             'useBuiltInControls': false,
 //             // 'ignoreDevicePixelRatio': false,
 //             // 'gpuAcceleratedSort': true,
 //             // 'halfPrecisionCovariancesOnGPU': true,
 //             'sharedMemoryForWorkers': false,
 //             // 'integerBasedSort': false,
 //             // 'dynamicScene': false,
 //             // 'webXRMode': GaussianSplats3D.WebXRMode.None,
 //         })
 //         this.splats.init()
 //         // this.splats.loadFile('https://generic-cors-proxy.repalash.workers.dev/https://zappar-xr.github.io/three-gaussian-splat-example/bonsai.5148b146.splat').then(()=>{
 //         this.splats.addSplatScene('https://generic-cors-proxy.repalash.workers.dev/https://projects.markkellogg.org/threejs/assets/data/garden/garden_high.ksplat').then(()=>{
 //             this._ready = true
 //         })
 //     }
 //
 //     protected _viewerListeners = {
 //         postFrame: (_: IViewerEvent) => {
 //             if (!this._ready) return
 //             console.log('postframe')
 //             this.splats.update()
 //             this.splats.render()
 //         },
 //     }
 // }
--- a/plugins/gaussian-splatting/src/global.d.ts
+++ b/plugins/gaussian-splatting/src/global.d.ts
 declare module '*.txt' {
    const content: string
    export default content
 }
 declare module '*.glsl' {
    const content: string
    export default content
 }
 declare module '*.vert' {
    const content: string
    export default content
 }
 declare module '*.frag' {
    const content: string
    export default content
 }
 declare module '*.module.scss' {
    const content: any
    export default content
    export const stylesheet: string
 }
 declare module '*.module.css' {
    const content: any
    export default content
    export const stylesheet: string
 }
 declare module '*.css' {
    const content: string
    export default content
 }
 declare module '*.css?inline' { // for vite
    const content: string
    export default content
 }
 // export {}
 // hack for typedoc
 // eslint-disable-next-line @typescript-eslint/naming-convention
 // declare type OffscreenCanvas = HTMLCanvasElement
--- a/plugins/gaussian-splatting/src/index.ts
+++ b/plugins/gaussian-splatting/src/index.ts
 import {ThreeGaussianSplatPlugin} from './three-gaussian-splat'
 export class GaussianSplattingPlugin extends ThreeGaussianSplatPlugin {}
 export * as threeGaussianSplat from './three-gaussian-splat'
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/ThreeGaussianSplatPlugin.css
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/ThreeGaussianSplatPlugin.css
 html{
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/ThreeGaussianSplatPlugin.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/ThreeGaussianSplatPlugin.ts
 import {AViewerPluginSync, createStyles, IGeometryEvent, ILoader, Importer, ThreeViewer} from 'threepipe'
 import styles from './ThreeGaussianSplatPlugin.css?inline'
 import {GaussianSplatMesh} from './index'
 import {AnyOptions} from 'ts-browser-helpers'
 import {SplatLoader} from './loaders/SplatLoader'
 import {SortWorkerManager} from './cpp-sorter/SortWorkerManager'
 import {GaussianSplatGeometry} from './geometry/GaussianSplatGeometry'
 export class ThreeGaussianSplatPlugin extends AViewerPluginSync<string> {
    public static readonly PluginType: string = 'ThreeGaussianSplatPlugin'
    enabled = true
    dependencies = []
    toJSON: any = null
    constructor() {
        super()
    }
    splats: GaussianSplatMesh[] = []
    private _ready = false
    onAdded(viewer: ThreeViewer) {
        super.onAdded(viewer)
        createStyles(styles)
        viewer.assetManager.importer.addImporter(this._importer)
        viewer.scene.addEventListener('mainCameraUpdate', this._activeCameraUpdate)
        viewer.scene.addEventListener('geometryUpdate', this._geometryUpdate)
        this._ready = true
    }
    onRemove(viewer: ThreeViewer) {
        viewer.assetManager.importer.removeImporter(this._importer)
    }
    private _activeCameraUpdate = () => {
        if (!this._ready || this.isDisabled()) return
        this.splats.forEach(async splat=>splat.update(this._viewer!.scene.mainCamera, this._viewer!.renderManager.webglRenderer))
    }
    private _geometryUpdate = (event: IGeometryEvent) => {
        if (!this._ready || this.isDisabled() || !(event.geometry as GaussianSplatGeometry)?.isGaussianSplatGeometry) return
        event.geometry!.appliedMeshes.forEach(async(splat: GaussianSplatMesh)=>splat.update ? splat.update(this._viewer!.scene.mainCamera, this._viewer!.renderManager.webglRenderer) : undefined)
    }
    private _sortWorkerManager = new SortWorkerManager() // todo: dispose?
    protected _importer = new Importer(class extends SplatLoader implements ILoader {
        onDispose: (mesh: GaussianSplatMesh)=>void = ()=>{return}
        onCreate: (mesh: GaussianSplatMesh)=>void = ()=>{return}
        transform(res: GaussianSplatMesh, _: AnyOptions): any {
            res.addEventListener('dispose', ()=>this.onDispose(res))
            this.onCreate(res)
            return res
        }
    }, ['splat'], [], true, (l)=>{
        if (!l) return l
        l.sortWorkerManager = this._sortWorkerManager
        l.onDispose = (mesh: GaussianSplatMesh)=>{ // todo: dispose should only remove from GPU?
            this.splats = this.splats.filter(splat=>splat !== mesh)
        }
        l.onCreate = (mesh: GaussianSplatMesh)=>{
            this.splats.push(mesh)
        }
        l.onGeometryLoad = (_)=>{
            // console.log('geometry loaded')
            // console.log(geometry.boundingBox)
            this._viewer?.setDirty()
        }
        return l
    })
    // protected _viewerListeners = {
    //     preRender: (_: IViewerEvent) => {
    //     },
    // }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/BufferPool.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/BufferPool.ts
 export class BufferPool {
  private pool: ArrayBuffer[];
  private bufferSize: number;
  constructor(bufferSize: number, initialPoolSize: number = 0) {
    this.bufferSize = bufferSize;
    this.pool = [];
    this.initPool(initialPoolSize);
  }
  private initPool(initialPoolSize: number): void {
    for (let i = 0; i < initialPoolSize; i++) {
      this.pool.push(new ArrayBuffer(this.bufferSize));
    }
  }
  public getBuffer(): ArrayBuffer {
    if (this.pool.length > 0) {
      return this.pool.pop()!;
    } else {
      return new ArrayBuffer(this.bufferSize);
    }
  }
  public returnBuffer(buffer: ArrayBuffer): void {
    if (buffer.byteLength === this.bufferSize) {
      this.pool.push(buffer);
    }
  }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/ISort.d.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/ISort.d.ts
 export interface MainModule {
  runSort(_0: number, _1: number, _2: number, _3: number): void;
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/SortWorkerManager.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/SortWorkerManager.ts
 import {Remote, transfer, wrap} from 'comlink'
 import type {WasmSorter} from './worker'
 export const SPLAT_ROW_LENGTH = 3 * 4 + 3 * 4 + 4 + 4
 function trimBuffer(_buffer: Uint8Array, _maxSplats: number, _vertexCount: number): {buffer: Uint8Array; vertexCount: number} {
    const actualVertexCount = Math.min(_vertexCount, _maxSplats)
    const actualBufferSize = SPLAT_ROW_LENGTH * actualVertexCount
    const buffer = _buffer.slice(0, actualBufferSize)
    return {buffer, vertexCount: actualVertexCount}
 }
 export class SortWorkerManager {
    private _workers: Remote<WasmSorter>[] = []
    private _maxWorkers = 8
    private _workerCtor: new (vertexCount: number, globalBuffer: Uint8Array) => Remote<WasmSorter>
    onError = (e: any) => {
        console.error(e)
        console.error(
            'ERROR: Line ', e.lineno, ' in ', e.filename, ': ', e.message
        )
    }
    constructor() {
        const worker = new Worker(new URL('../cpp-sorter/worker', import.meta.url), {type: 'module'})
        worker.addEventListener('error', this.onError, false)
        this._workerCtor = wrap(worker) as any
    }
    async createWorker(data: Uint8Array, maxSplats = 1000000) {
        if (this._workers.length < this._maxWorkers) {
            const vertexCount = Math.floor(data.length / SPLAT_ROW_LENGTH)
            const bufferInfo = trimBuffer(data, maxSplats, vertexCount)
            const globalBuffer = transfer(bufferInfo.buffer, [bufferInfo.buffer.buffer])
            const worker = await new this._workerCtor(vertexCount, globalBuffer)
            await worker.load()
            this._workers.push(worker)
            return worker
        }
        console.error('Max workers reached')
        throw new Error('Max workers reached')
    }
    async disposeWorker(worker: Remote<WasmSorter>) {
        const index = this._workers.indexOf(worker)
        if (index !== -1) {
            this._workers.splice(index, 1)
            await worker.dispose()
        }
    }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/build-in-docker.sh
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/build-in-docker.sh
 #!/bin/bash
 set -e
 DOCKER_IMAGE="emscripten/emsdk:3.1.50"
 docker run --rm -v "$(pwd)":/src -w /src $DOCKER_IMAGE bash build.sh
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/build.sh
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/build.sh
 #!/bin/bash
 set -e
 rm -rf sort.wasm sort.js sort.d.ts
 rm -rf pthread_sort.wasm pthread_sort.js pthread_sort.worker.js
 em++ -pthread -O3 -s EXPORT_ES6=1 -s ALLOW_MEMORY_GROWTH=1 -s MODULARIZE=1 -s ENVIRONMENT=web,worker -flto --embind-emit-tsd=ISort.d.ts sort.cpp -o pthread_sort.js -s WASM=1 -lembind -s EXPORTED_FUNCTIONS='["_malloc", "_free"]' -s AGGRESSIVE_VARIABLE_ELIMINATION=1 -s ELIMINATE_DUPLICATE_FUNCTIONS=1
 em++ -O3 -s EXPORT_ES6=1 -s ALLOW_MEMORY_GROWTH=1 -s MODULARIZE=1 -s ENVIRONMENT=web,worker -flto --embind-emit-tsd=ISort.d.ts sort.cpp -o sort.js -s WASM=1 -lembind -s EXPORTED_FUNCTIONS='["_malloc", "_free"]' -s AGGRESSIVE_VARIABLE_ELIMINATION=1 -s ELIMINATE_DUPLICATE_FUNCTIONS=1
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/sort.cpp
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/sort.cpp
 #include <emscripten/bind.h>
 #include <algorithm>
 #include <cstdint>
 #include <cmath>
 #include <vector>
 #include <numeric>
 #include <stdio.h>
 #include <cstring>
 union DepthIndex {
    struct {
        float depth;
        uint32_t index;
    };
    bool operator < (const DepthIndex& other) const {
        return depth < other.depth;
    }
 };
 uint32_t floatToUInt(float f) {
    uint32_t u;
    std::memcpy(&u, &f, sizeof(float));
    uint32_t mask = -int32_t(u >> 31) | 0x80000000;
    return u ^ mask;
 }
 void radixSort(std::vector<DepthIndex>& depthIndices) {
    const int n = depthIndices.size();
    std::vector<DepthIndex> temp(n);
    for (int shift = 0; shift < 32; shift += 8) {
        size_t count[256] = {};
        for (auto& di : depthIndices) {
            count[(floatToUInt(di.depth) >> shift) & 0xFF]++;
        }
        size_t pos[256];
        pos[0] = 0;
        for (int i = 1; i < 256; i++) {
            pos[i] = pos[i - 1] + count[i - 1];
        }
        for (auto& di : depthIndices) {
            int index = (floatToUInt(di.depth) >> shift) & 0xFF;
            temp[pos[index]++] = di;
        }
        depthIndices.swap(temp);
    }
 }
 using namespace emscripten;
 void runSort(int viewProjPtr, int bufferPtr, int vertexCount, int combinedPtr) {
    auto *viewProj = reinterpret_cast<float *>(viewProjPtr);
    uint8_t *buffer = reinterpret_cast<uint8_t *>(bufferPtr);
    auto *combined = reinterpret_cast<float *>(combinedPtr);
    // Calculate lengths based on vertexCount
    int quatLength = 4 * vertexCount;
    int scaleLength = 3 * vertexCount;
    int centerLength = 3 * vertexCount;
    int colorLength = 4 * vertexCount;
    // Calculate offsets for each array within the combined array
    int quatOffset = 0;
    int scaleOffset = quatOffset + quatLength;
    int centerOffset = scaleOffset + scaleLength;
    int colorOffset = centerOffset + centerLength;
    std::vector<DepthIndex> depthIndices(vertexCount);
    for (int i = 0; i < vertexCount; ++i) {
        auto *f_buffer = reinterpret_cast<float *>(buffer + 32 * i);
        depthIndices[i].depth = 10000 - (viewProj[2] * f_buffer[0] + viewProj[6] * f_buffer[1] + viewProj[10] * f_buffer[2]);
        depthIndices[i].index = i;
    }
    radixSort(depthIndices);
    for (int i = 0; i < vertexCount; ++i) {
        uint32_t index = depthIndices[i].index;
        auto *f_buffer = reinterpret_cast<float *>(buffer + 32 * index);
        combined[quatOffset + 4 * i + 0] = (buffer[32 * index + 28 + 0] - 128) / 128.0f;
        combined[quatOffset + 4 * i + 1] = (buffer[32 * index + 28 + 1] - 128) / 128.0f;
        combined[quatOffset + 4 * i + 2] = (buffer[32 * index + 28 + 2] - 128) / 128.0f;
        combined[quatOffset + 4 * i + 3] = (buffer[32 * index + 28 + 3] - 128) / 128.0f;
        combined[centerOffset + 3 * i + 0] = f_buffer[0];
        combined[centerOffset + 3 * i + 1] = f_buffer[1];
        combined[centerOffset + 3 * i + 2] = f_buffer[2];
        combined[colorOffset + 4 * i + 0] = buffer[32 * index + 24 + 0] / 255.0f;
        combined[colorOffset + 4 * i + 1] = buffer[32 * index + 24 + 1] / 255.0f;
        combined[colorOffset + 4 * i + 2] = buffer[32 * index + 24 + 2] / 255.0f;
        combined[colorOffset + 4 * i + 3] = buffer[32 * index + 24 + 3] / 255.0f;
        combined[scaleOffset + 3 * i + 0] = f_buffer[3];
        combined[scaleOffset + 3 * i + 1] = f_buffer[4];
        combined[scaleOffset + 3 * i + 2] = f_buffer[5];
    }
 }
 EMSCRIPTEN_BINDINGS(my_module) {
    function("runSort", &runSort);
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/sort.js
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/sort.js
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/sort.wasm
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/sort.wasm
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/watch.sh
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/watch.sh
 #!/bin/bash
 set -e
 ./build.sh
 fswatch -o sort.cpp | while read line
 do
  echo "File changed, rebuilding..."
  ./build.sh
  echo "Done."
 done
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/worker.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/cpp-sorter/worker.ts
 import {expose, transfer} from 'comlink'
 import type {MainModule} from './ISort'
 // @ts-expect-error no types
 import workerPromise from './sort'
 // import sharedArrayBufferWorkerPromise from './sort'
 import {BufferPool} from './BufferPool'
 export class WasmSorter {
    public module: EmscriptenModule & MainModule
    private _viewProjPtr: number
    private _globalBufferPtr: number
    private _combinedPtr: number
    private _bufferPool: BufferPool
    constructor(private _vertexCount: number, private _globalBuffer: Uint8Array) {
        const combinedLength = this._calculateCombinedLength()
        // we're double buffering, so 2 is the magic number here
        this._bufferPool = new BufferPool(combinedLength, 2)
    }
    public async load() {
        // const sharedABSupported = typeof SharedArrayBuffer !== 'undefined'
        // this.module = await (sharedABSupported ? sharedArrayBufferWorkerPromise() : workerPromise())
        this.module = await workerPromise()
        this._viewProjPtr = this.module._malloc(16 * Float32Array.BYTES_PER_ELEMENT)
        this._globalBufferPtr = this.module._malloc(this._vertexCount * 32)
        this._combinedPtr = this.module._malloc(this._calculateCombinedLength())
        this.module.HEAPU8.set(this._globalBuffer, this._globalBufferPtr)
    }
    public runSort(viewProj: Float32Array): ArrayBuffer {
        this.module.HEAPF32.set(viewProj, this._viewProjPtr / Float32Array.BYTES_PER_ELEMENT)
        this.module.runSort(this._viewProjPtr, this._globalBufferPtr, this._vertexCount, this._combinedPtr)
        const byteStart = this._combinedPtr
        const byteLength = this._calculateCombinedLength()
        const bufferToTransfer = this._bufferPool.getBuffer()
        new Uint8Array(bufferToTransfer).set(new Uint8Array(this.module.HEAPU8.buffer, byteStart, byteLength))
        return transfer(bufferToTransfer, [bufferToTransfer])
    }
    public returnBuffer(buffer: ArrayBuffer): void {
        this._bufferPool.returnBuffer(buffer)
    }
    private _calculateCombinedLength(): number {
        return 4 * 4 * this._vertexCount + 3 * 4 * this._vertexCount + 3 * 4 * this._vertexCount + 4 * 4 * this._vertexCount * Float32Array.BYTES_PER_ELEMENT
    }
    public dispose(): void {
        if (this.module) {
            this.module._free(this._viewProjPtr)
            this.module._free(this._globalBufferPtr)
            this.module._free(this._combinedPtr)
        }
    }
 }
 expose(WasmSorter)
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/geometry/GaussianSplatGeometry.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/geometry/GaussianSplatGeometry.ts
 import type {WasmSorter} from '../cpp-sorter/worker'
 import {Remote, transfer} from 'comlink'
 import {
    Box3,
    BufferAttribute,
    Camera,
    IGeometry,
    iGeometryCommons,
    InstancedBufferAttribute,
    InstancedBufferGeometry,
    type IObject3D,
    Matrix4,
    PerspectiveCamera,
    Sphere,
    Vector3,
 } from 'threepipe'
 export class GaussianSplatGeometry extends InstancedBufferGeometry implements IGeometry {
    constructor(
        private _worker: Remote<WasmSorter>,
        private _vertexCount: number,
        private _maxSplats: number,
        private _onLoad?: (geometry: GaussianSplatGeometry) => void,
    ) {
        super()
        iGeometryCommons.upgradeGeometry.call(this)
        this.initAttributes()
    }
    readonly isGaussianSplatGeometry = true
    assetType: 'geometry' // dont set the value here since its checked in upgradeGeometry
    setDirty = iGeometryCommons.setDirty
    refreshUi = iGeometryCommons.refreshUi
    appliedMeshes = new Set<IObject3D>()
    private _viewProj: number[] = []
    private _sortRunning = false
    private _initialized = false
    private _centersBuffer: Float32Array = new Float32Array(0)
    public async update(camera: PerspectiveCamera | Camera, meshMatrixWorld: Matrix4) {
        if (this._sortRunning || !this._initialized || !this._worker) {
            return
        }
        camera.updateMatrixWorld(true)
        this._viewProj = new Matrix4().multiply(camera.projectionMatrix).multiply(camera.matrixWorldInverse).multiply(meshMatrixWorld).elements
        this._sortRunning = true
        const viewProj = new Float32Array(this._viewProj)
        const result = await this._worker.runSort(viewProj)
        const {quat, scale, center, color} = this._extractViews(result)
        if (this._centersBuffer.length !== center.length) this._centersBuffer = new Float32Array(center)
        else this._centersBuffer.set(center)
        ;(this.attributes.color as InstancedBufferAttribute).array = color;
        (this.attributes.quat as InstancedBufferAttribute).array = quat;
        (this.attributes.scale as InstancedBufferAttribute).array = scale;
        (this.attributes.center as InstancedBufferAttribute).array = this._centersBuffer
        // (this.attributes.center as InstancedBufferAttribute).array = center
        const pms = Promise.all([
            new Promise<void>(resolve => (this.attributes.color as InstancedBufferAttribute).onUpload(resolve)),
            new Promise<void>(resolve => (this.attributes.quat as InstancedBufferAttribute).onUpload(resolve)),
            new Promise<void>(resolve => (this.attributes.scale as InstancedBufferAttribute).onUpload(resolve)),
            // new Promise<void>(resolve => (this.attributes.center as InstancedBufferAttribute).onUpload(resolve)),
        ])
        this.attributes.color.needsUpdate = true
        this.attributes.quat.needsUpdate = true
        this.attributes.scale.needsUpdate = true
        this.attributes.center.needsUpdate = true
        this.setDirty()
        await pms
        await this._worker.returnBuffer(transfer(result, [result]))
        this._sortRunning = false
    }
    async initAttributes() {
        const viewProj = new Float32Array(this._viewProj)
        const result = await this._worker.runSort(viewProj)
        const {quat, scale, center, color} = this._extractViews(result)
        this.setAttribute('color', new InstancedBufferAttribute(color, 4, true))
        this.setAttribute('quat', new InstancedBufferAttribute(quat, 4, true))
        this.setAttribute('scale', new InstancedBufferAttribute(scale, 3, true))
        this.setAttribute('center', new InstancedBufferAttribute(center, 3, true))
        this.setAttribute('position', new BufferAttribute(new Float32Array([1, -1, 0, 1, 1, 0, -1, -1, 0, -1, 1, 0]), 3, true))
        this.attributes.position.needsUpdate = true
        this.setIndex(new BufferAttribute(new Uint16Array([0, 1, 2, 2, 3, 0]), 1, true))
        this.instanceCount = Math.min(quat.length / 4, this._maxSplats)
        this.computeBoundingBox()
        this.computeBoundingSphere()
        this._initialized = true
        this.setDirty()
        this._onLoad && this._onLoad(this)
    }
    private _extractViews(receivedBuffer: ArrayBuffer): {quat: Float32Array; scale: Float32Array; center: Float32Array; color: Float32Array} {
        const combined = new Float32Array(receivedBuffer)
        const quatLength = 4 * this._vertexCount
        const scaleLength = 3 * this._vertexCount
        const centerLength = 3 * this._vertexCount
        const colorLength = 4 * this._vertexCount
        const quatOffset = 0
        const scaleOffset = quatOffset + quatLength
        const centerOffset = scaleOffset + scaleLength
        const colorOffset = centerOffset + centerLength
        const quat = combined.subarray(quatOffset, quatOffset + quatLength)
        const scale = combined.subarray(scaleOffset, scaleOffset + scaleLength)
        const center = combined.subarray(centerOffset, centerOffset + centerLength)
        const color = combined.subarray(colorOffset, colorOffset + colorLength)
        return {quat, scale, center, color}
    }
    computeBoundingBox() {
        if (!this.getAttribute('center')) return super.computeBoundingBox()
        const box = this.boundingBox ?? (this.boundingBox = new Box3())
        box.setFromBufferAttribute(this.getAttribute('center') as InstancedBufferAttribute)
        if (isNaN(this.boundingBox!.min.x) || isNaN(this.boundingBox!.min.y) || isNaN(this.boundingBox!.min.z)) {
            console.error('GaussianSplatGeometry.computeBoundingBox(): Computed min/max have NaN values. The "position" attribute is likely to have NaN values.', this)
        }
    }
    computeBoundingSphere() {
        if (this.boundingSphere === null) this.boundingSphere = new Sphere()
        const position = this.attributes.center
        if (position && (position as any).isGLBufferAttribute) {
            console.error('THREE.BufferGeometry.computeBoundingSphere(): GLBufferAttribute requires a manual bounding sphere. Alternatively set "mesh.frustumCulled" to "false".', this)
            this.boundingSphere.set(new Vector3(), Infinity)
            return
        }
        if (position) {
            // first, find the center of the bounding sphere
            const center = this.boundingSphere.center
            if (!this.boundingBox) this.computeBoundingBox()
            this.boundingBox!.getCenter(center)
            // second, try to find a boundingSphere with a radius smaller than the
            // boundingSphere of the boundingBox: sqrt(3) smaller in the best case
            let maxRadiusSq = 0
            const vector = new Vector3()
            for (let i = 0, il = position.count; i < il; i++) {
                vector.fromBufferAttribute(position, i)
                maxRadiusSq = Math.max(maxRadiusSq, center.distanceToSquared(vector))
            }
            this.boundingSphere.radius = Math.sqrt(maxRadiusSq)
            if (isNaN(this.boundingSphere.radius)) {
                console.error('THREE.BufferGeometry.computeBoundingSphere(): Computed radius is NaN. The "position" attribute is likely to have NaN values.', this)
            }
        }
    }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/index.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/index.ts
 /**
 * ThreeGaussianSplatPlugin: Threejs utilities and threepipe plugin and material extension for Gaussian Splatting
 *
 * @license
 * A port of -
 * https://github.com/zappar-xr/three-gaussian-splat
 * MIT License (c) 2023 Zappar Limited
 * Which is based on -
 * gsplat.js, MIT License (c) 2023 Dylan Ebert
 * antimatter15/splat, MIT License (c) 2023 Kevin Kwok
 */
 export {GaussianSplatMaterialExtension} from './materials/GaussianSplatMaterialExtension'
 export {GaussianSplatMaterialPhysical} from './materials/GaussianSplatMaterialPhysical'
 export {GaussianSplatMaterialRaw} from './materials/GaussianSplatMaterialRaw'
 export {GaussianSplatMaterialUnlit} from './materials/GaussianSplatMaterialUnlit'
 export {computeFocalLengths} from './materials/util'
 export {GaussianSplatGeometry} from './geometry/GaussianSplatGeometry'
 export {GaussianSplatMesh} from './mesh/GaussianSplatMesh'
 export {ThreeGaussianSplatPlugin} from './ThreeGaussianSplatPlugin'
 export {SplatLoader} from './loaders/SplatLoader'
 export {gaussianSplatShaders} from './shaders'
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/loaders/SplatLoader.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/loaders/SplatLoader.ts
 import {FileLoader, ILoader, IMaterial, Loader, LoadingManager} from 'threepipe'
 import {SortWorkerManager, SPLAT_ROW_LENGTH} from '../cpp-sorter/SortWorkerManager'
 import {GaussianSplatGeometry} from '../geometry/GaussianSplatGeometry'
 import {GaussianSplatMesh} from '../mesh/GaussianSplatMesh'
 import {GaussianSplatMaterialUnlit} from '../materials/GaussianSplatMaterialUnlit'
 export class SplatLoader extends Loader implements ILoader {
    sortWorkerManager: SortWorkerManager
    materialConstructor = (_: GaussianSplatGeometry): IMaterial|undefined => new GaussianSplatMaterialUnlit()
    constructor(manager?: LoadingManager) {
        super(manager)
    }
    onGeometryLoad = (_: GaussianSplatGeometry)=>{
        return
    }
    public load(url: string, onLoad?: (data: GaussianSplatMesh) => void, onProgress?: (event: ProgressEvent) => void, onError?: (event: ErrorEvent) => void): void {
        // const path = ( scope.path === '' ) ? LoaderUtils.extractUrlBase( url ) : scope.path;
        const loader = new FileLoader(this.manager)
        loader.setPath(this.path)
        loader.setResponseType('arraybuffer')
        loader.setRequestHeader(this.requestHeader)
        loader.setWithCredentials(this.withCredentials)
        loader.load(url, async(buffer) => {
            try {
                const data = new Uint8Array(buffer as ArrayBuffer)
                const maxSplats = 1000000
                const vertexCount = Math.floor(data.length / SPLAT_ROW_LENGTH)
                const worker = await this.sortWorkerManager.createWorker(data, maxSplats)
                const geometry = new GaussianSplatGeometry(worker, vertexCount, maxSplats, this.onGeometryLoad)
                const mesh = new GaussianSplatMesh(geometry, this.materialConstructor(geometry) as any)
                // const mesh = new GaussianSplatMesh(geometry, new UnlitMaterial() as any)
                mesh.rotation.x = Math.PI
                if (!url.startsWith('blob:') && !url.startsWith('data:'))
                    mesh.name = url.split('/').pop()!.split('?')[0]
                onLoad && onLoad(mesh)
            } catch (e) {
                if (onError) onError(e)
                else console.error(e)
                this.manager.itemError(url)
            }
        }, onProgress, onError)
    }
    public async loadAsync(url: string, onProgress?: (event: ProgressEvent) => void): Promise<GaussianSplatMesh> {
        return new Promise((resolve, reject) => this.load(url, resolve, onProgress, reject))
    }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/materials/GaussianSplatMaterialExtension.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/materials/GaussianSplatMaterialExtension.ts
 import {
    Camera,
    IMaterial,
    MaterialExtension,
    PerspectiveCamera,
    Shader,
    shaderReplaceString,
    Vector2,
    WebGLRenderer,
 } from 'threepipe'
 import {computeFocalLengths} from './util'
 import {gaussianSplatShaders} from '../shaders'
 export class GaussianSplatMaterialExtension implements MaterialExtension {
    readonly isGaussianSplatMaterialExtension = true
    extraUniforms = {
        viewport: {value: new Vector2()},
        focal: {value: new Vector2()},
        minAlpha: {value: 0.02},
    }
    parsFragmentSnippet = gaussianSplatShaders.pars_frag
    parsVertexSnippet = gaussianSplatShaders.pars_vert
    shaderExtender = (shader: Shader, material: IMaterial) => {
        shader.vertexShader = shaderReplaceString(shader.vertexShader,
            '#include <begin_vertex>',
            'vec3 transformed = vec3( center );')
        if (shader.vertexShader.includes('#include <beginnormal_vertex>')) {
            // todo: get correct normal by rendering to depth and then sampling the normal
            shader.vertexShader = shaderReplaceString(shader.vertexShader,
                '#include <beginnormal_vertex>',
                '\nobjectNormal = vec3(1,0,0);\n', {append: true})
        }
        shader.vertexShader = shaderReplaceString(shader.vertexShader,
            '#include <project_vertex>',
            gaussianSplatShaders.main_vert, {append: true})
        if (!material.isGBufferMaterial && !material.userData.isGBufferMaterial) {
            shader.fragmentShader = shaderReplaceString(shader.fragmentShader,
                '#include <map_fragment>',
                shaderReplaceString(
                    gaussianSplatShaders.main_frag,
                    /gl_FragColor\s*=/, // because of minification
                    'diffuseColor*='
                ), {prepend: true})
        }
        // eslint-disable-next-line no-constant-condition
        if (0)
            shader.fragmentShader = shaderReplaceString(shader.fragmentShader,
                '#include <output_fragment>',
                '\ngl_FragColor=diffuseColor;',
                // '\ngl_FragColor=vec4(reflectedLight.directDiffuse, 1.);',
                // '\ngl_FragColor=vec4(geometryNormal, 1.);',
                {append: true})
        return shader
    }
    isCompatible = (material: IMaterial) =>
        !!material.isPhysicalMaterial ||
        !!material.isUnlitMaterial ||
        !!material.isGBufferMaterial ||
        !!material.userData.isGBufferMaterial
    setDirty?: () => void
    private _currentCamera?: PerspectiveCamera | Camera
    private _renderer?: WebGLRenderer
    public set minAlpha(value: number) {
        this.extraUniforms.minAlpha.value = value
        this.setDirty && this.setDirty()
    }
    constructor() {
        window.addEventListener('resize', this._refresh)
    }
    dispose() {
        // todo: add again on added to mesh?
        window.removeEventListener('resize', this._refresh)
    }
    update(camera: PerspectiveCamera | Camera, renderer: WebGLRenderer): void {
        if (this._currentCamera === camera && this._renderer === renderer) return
        this._renderer = renderer
        this._currentCamera = camera
        this._refresh()
    }
    private _refresh = (): void => {
        if (!this._currentCamera) return
        const size = new Vector2()
        this._renderer?.getSize(size)
        const dpr = this._renderer?.getPixelRatio() || 1
        let fov = 75
        let aspect = size.x / size.y
        if (this._currentCamera instanceof PerspectiveCamera) {
            fov = this._currentCamera.fov
            aspect = this._currentCamera.aspect
        }
        this.extraUniforms.focal.value = computeFocalLengths(size.x, size.y, fov, aspect, dpr)
        this.extraUniforms.viewport.value = new Vector2(size.x * dpr, size.y * dpr)
    }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/materials/GaussianSplatMaterialPhysical.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/materials/GaussianSplatMaterialPhysical.ts
 import {PhysicalMaterial} from 'threepipe'
 import {GaussianSplatMaterialExtension} from './GaussianSplatMaterialExtension'
 export class GaussianSplatMaterialPhysical extends PhysicalMaterial {
    readonly isGaussianSplatMaterialPhysical = true
    gsplatExtension = new GaussianSplatMaterialExtension()
    constructor() {
        super({
            depthTest: true,
            depthWrite: false,
            transparent: true,
            vertexColors: false,
        })
        // this.userData.renderToGBuffer = true
        // this.userData.renderToDepth = true
        this.registerMaterialExtensions([this.gsplatExtension])
    }
    dispose() {
        this.gsplatExtension.dispose()
        return super.dispose()
    }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/materials/GaussianSplatMaterialRaw.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/materials/GaussianSplatMaterialRaw.ts
 import {Camera, PerspectiveCamera, ShaderMaterial2, Vector2, WebGLRenderer} from 'threepipe'
 import {computeFocalLengths} from './util'
 import {gaussianSplatShaders} from '../shaders'
 export class GaussianSplatMaterialRaw extends ShaderMaterial2 {
    private _currentCamera?: PerspectiveCamera | Camera
    private _renderer?: WebGLRenderer
    readonly isGaussianSplatMaterialRaw = true
    public set minAlpha(value: number) {
        this.uniforms.minAlpha.value = value
        this.needsUpdate = true
    }
    constructor() {
        super({
            uniforms: {
                viewport: {value: new Vector2()},
                focal: {value: new Vector2()},
                minAlpha: {value: 0.02},
            },
            fragmentShader: `${gaussianSplatShaders.pars_frag}
 void main () {
 ${gaussianSplatShaders.main_frag}
 }`,
            vertexShader: `${gaussianSplatShaders.pars_vert}
 void main () {
    gl_Position = projectionMatrix * modelViewMatrix * vec4(center, 1);
    ${gaussianSplatShaders.main_vert}
 }`,
            depthTest: true,
            depthWrite: false,
            transparent: true,
        }, true)
        window.addEventListener('resize', this._refresh)
    }
    private _refresh = (): void => {
        if (!this._currentCamera) return
        const size = new Vector2()
        this._renderer?.getSize(size)
        const width = size.x
        const height = size.y
        const dpr = this._renderer?.getPixelRatio() || 1
        let fov = 75
        let aspect = width / height
        if (this._currentCamera instanceof PerspectiveCamera) {
            fov = this._currentCamera.fov
            aspect = this._currentCamera.aspect
        }
        this.uniforms.focal.value = computeFocalLengths(width, height, fov, aspect, dpr)
        this.uniforms.viewport.value = new Vector2(width * dpr, height * dpr)
    }
    dispose() {
        // todo: add again on added to mesh?
        window.removeEventListener('resize', this._refresh)
        return super.dispose()
    }
    update(camera: PerspectiveCamera | Camera, renderer: WebGLRenderer): void {
        this._renderer = renderer
        this._currentCamera = camera
        this._refresh()
    }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/materials/GaussianSplatMaterialUnlit.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/materials/GaussianSplatMaterialUnlit.ts
 import {UnlitMaterial} from 'threepipe'
 import {GaussianSplatMaterialExtension} from './GaussianSplatMaterialExtension'
 export class GaussianSplatMaterialUnlit extends UnlitMaterial {
    readonly isGaussianSplatMaterialUnlit = true
    gsplatExtension = new GaussianSplatMaterialExtension()
    constructor() {
        super({
            depthTest: true,
            depthWrite: false,
            transparent: true,
            vertexColors: false,
        })
        // this.userData.renderToGBuffer = true
        // this.userData.renderToDepth = true
        this.registerMaterialExtensions([this.gsplatExtension])
    }
    dispose() {
        this.gsplatExtension.dispose()
        return super.dispose()
    }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/materials/util.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/materials/util.ts
 import {MathUtils, Vector2} from 'threepipe'
 export const computeFocalLengths = (width: number, height: number, fov: number, aspect: number, dpr: number) => {
    const fovRad = MathUtils.degToRad(fov)
    const fovXRad = 2 * Math.atan(Math.tan(fovRad / 2) * aspect)
    const fy = dpr * height / (2 * Math.tan(fovRad / 2))
    const fx = dpr * width / (2 * Math.tan(fovXRad / 2))
    return new Vector2(fx, fy)
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/mesh/GaussianSplatMesh.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/mesh/GaussianSplatMesh.ts
 import {GaussianSplatGeometry} from '../geometry/GaussianSplatGeometry'
 import {Camera, IMaterial, Mesh2, PerspectiveCamera, WebGLRenderer} from 'threepipe'
 import type {GaussianSplatMaterialRaw} from '../materials/GaussianSplatMaterialRaw'
 import {GaussianSplatMaterialUnlit} from '../materials/GaussianSplatMaterialUnlit'
 import {Matrix4, Ray, Raycaster, Sphere, Vector3} from 'three'
 import {GaussianSplatMaterialExtension} from '../materials/GaussianSplatMaterialExtension'
 export class GaussianSplatMesh extends Mesh2<GaussianSplatGeometry, IMaterial> {
    readonly isGaussianSplatMesh = true
    constructor(geometry: GaussianSplatGeometry, material: IMaterial) {
        super(geometry, material)
        this.frustumCulled = false
    }
    public async update(camera: PerspectiveCamera | Camera, renderer: WebGLRenderer) {
        if ((this.material as any as GaussianSplatMaterialRaw)?.isGaussianSplatMaterialRaw) {
            (this.material as any as GaussianSplatMaterialRaw).update(camera, renderer)
        } else if (this.material) {
            const ext =
                (this.material as GaussianSplatMaterialUnlit).gsplatExtension ??
                this.material.materialExtensions.find(e=>
                    (e as GaussianSplatMaterialExtension).isGaussianSplatMaterialExtension) as GaussianSplatMaterialExtension
            ext && ext.update(camera, renderer)
        }
        this.updateMatrixWorld(true)
        return this.geometry.update(camera, this.matrixWorld)
    }
    raycast(raycaster: Raycaster, intersects: any[]) {
        const geometry = this.geometry
        const matrixWorld = this.matrixWorld
        // const threshold = raycaster.params.Points?.threshold ?? .01
        const threshold = .02
        // Checking boundingSphere distance to ray
        if (geometry.boundingSphere === null) geometry.computeBoundingSphere()
        const sphere = new Sphere()
        sphere.copy(geometry.boundingSphere!)
        sphere.applyMatrix4(matrixWorld)
        sphere.radius += threshold
        if (!raycaster.ray.intersectsSphere(sphere)) return
        //
        const inverseMatrix = new Matrix4()
        const ray = new Ray()
        inverseMatrix.copy(matrixWorld).invert()
        ray.copy(raycaster.ray).applyMatrix4(inverseMatrix)
        const localThreshold = threshold / ((this.scale.x + this.scale.y + this.scale.z) / 3)
        const localThresholdSq = localThreshold * localThreshold
        const attributes = geometry.attributes
        const positionAttribute = attributes.center
        const position = new Vector3()
        for (let i = 0, l = positionAttribute.count; i < l; i++) {
            position.fromBufferAttribute(positionAttribute, i)
            const rayPointDistanceSq = ray.distanceSqToPoint(position)
            if (rayPointDistanceSq < localThresholdSq) {
                const intersectPoint = new Vector3()
                ray.closestPointToPoint(position, intersectPoint)
                intersectPoint.applyMatrix4(matrixWorld)
                const distance = raycaster.ray.origin.distanceTo(intersectPoint)
                if (distance < raycaster.near || distance > raycaster.far) return
                intersects.push({
                    distance: distance,
                    distanceToRay: Math.sqrt(rayPointDistanceSq),
                    point: intersectPoint,
                    face: null,
                    object: this,
                })
            }
        }
    }
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/gsplat.main.frag.glsl
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/gsplat.main.frag.glsl
 vec2 d = (vCenter - 2.0 * (gl_FragCoord.xy/viewport - vec2(0.5, 0.5))) * viewport * 0.5;
 float power = -0.5 * (vConic.x * d.x * d.x + vConic.z * d.y * d.y) + vConic.y * d.x * d.y;
 if (power > 0.0) discard;
 float alpha = min(0.99, vColor.a * exp(power));
 if(alpha < minAlpha) discard;
 gl_FragColor = vec4(vColor.rgb, alpha);
 //vec2 d = (vCenter - 2.0 * (gl_FragCoord.xy/viewport - vec2(0.5, 0.5))) * viewport * 0.5;
 //
 //float power = -0.5 * (vConic.x * d.x * d.x + vConic.z * d.y * d.y) + vConic.y * d.x * d.y;
 //
 //if (power > 0.0) discard;
 //float alpha = min(0.99, vColor.a * exp(power));
 //if(alpha < minAlpha) discard;
 //
 //diffuseColor *= vec4(vColor.rgb, alpha);
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/gsplat.main.vert.glsl
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/gsplat.main.vert.glsl
 vec3 cov2d = compute_cov2d(center, scale, quat);
 float det = cov2d.x * cov2d.z - cov2d.y * cov2d.y;
 vec3 conic = vec3(cov2d.z, cov2d.y, cov2d.x) / det;
 float mid = 0.5 * (cov2d.x + cov2d.z);
 float lambda1 = mid + sqrt(max(0.1, mid * mid - det));
 float lambda2 = mid - sqrt(max(0.1, mid * mid - det));
 vec2 v1 = 7.0 * sqrt(lambda1) * normalize(vec2(cov2d.y, lambda1 - cov2d.x));
 vec2 v2 = 7.0 * sqrt(lambda2) * normalize(vec2(-(lambda1 - cov2d.x),cov2d.y));
 vColor = color;
 vConic = conic;
 vCenter = vec2(gl_Position) / gl_Position.w;
 vPosition = vec2(vCenter + position.x * (position.y < 0.0 ? v1 : v2) / viewport);
 gl_Position = vec4(vPosition, gl_Position.z / gl_Position.w, 1);
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/gsplat.pars.frag.glsl
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/gsplat.pars.frag.glsl
 // https://github.com/vincent-lecrubier-skydio/react-three-fiber-gaussian-splat
 precision mediump float;
 varying vec4 vColor;
 varying vec3 vConic;
 varying vec2 vCenter;
 uniform vec2 viewport;
 uniform vec2 focal;
 uniform float minAlpha;
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/gsplat.pars.vert.glsl
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/gsplat.pars.vert.glsl
 // https://github.com/vincent-lecrubier-skydio/react-three-fiber-gaussian-splat
 precision mediump float;
 #ifndef SHADER_NAME // isRawShaderMaterial
 attribute vec3 position;
 uniform mat4 modelViewMatrix;
 uniform mat4 projectionMatrix;
 mat3 transpose(mat3 m) { return mat3(m[0][0], m[1][0], m[2][0], m[0][1], m[1][1], m[2][1], m[0][2], m[1][2], m[2][2]); }
 #endif
 attribute vec4 color;
 attribute vec4 quat;
 attribute vec3 scale;
 attribute vec3 center;
 uniform vec2 focal;
 uniform vec2 viewport;
 //uniform vec3 sphereCenter;
 //uniform vec3 planeNormal;
 //uniform float planeDistance;
 varying vec4 vColor;
 varying vec3 vConic;
 varying vec2 vCenter;
 varying vec2 vPosition;
 //varying float vDistance;
 //varying float vPlaneSide;
 mat3 compute_cov3d(vec3 scale, vec4 rot) {
    mat3 S = mat3(
    scale.x, 0.0, 0.0,
    0.0, scale.y, 0.0,
    0.0, 0.0, scale.z
    );
    mat3 R = mat3(
    1.0 - 2.0 * (rot.z * rot.z + rot.w * rot.w), 2.0 * (rot.y * rot.z - rot.x * rot.w), 2.0 * (rot.y * rot.w + rot.x * rot.z),
    2.0 * (rot.y * rot.z + rot.x * rot.w), 1.0 - 2.0 * (rot.y * rot.y + rot.w * rot.w), 2.0 * (rot.z * rot.w - rot.x * rot.y),
    2.0 * (rot.y * rot.w - rot.x * rot.z), 2.0 * (rot.z * rot.w + rot.x * rot.y), 1.0 - 2.0 * (rot.y * rot.y + rot.z * rot.z)
    );
    mat3 M = S * R;
    return transpose(M) * M;
 }
 vec3 compute_cov2d(vec3 center, vec3 scale, vec4 rot){
    mat3 Vrk = compute_cov3d(scale, rot);
    vec4 t = modelViewMatrix * vec4(center, 1.0);
    vec2 lims = 1.3 * 0.5 * viewport / focal;
    t.xy = min(lims, max(-lims, t.xy / t.z)) * t.z;
    mat3 J = mat3(
    focal.x / t.z, 0., -(focal.x * t.x) / (t.z * t.z),
    0., focal.y / t.z, -(focal.y * t.y) / (t.z * t.z),
    0., 0., 0.
    );
    mat3 W = transpose(mat3(modelViewMatrix));
    mat3 T = W * J;
    mat3 cov = transpose(T) * transpose(Vrk) * T;
    return vec3(cov[0][0] + 0.3, cov[0][1], cov[1][1] + 0.3);
 }
--- a/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/index.ts
+++ b/plugins/gaussian-splatting/src/three-gaussian-splat/shaders/index.ts
 import fragmentParsShaderSource from '../shaders/gsplat.pars.frag.glsl'
 import vertexParsShaderSource from '../shaders/gsplat.pars.vert.glsl'
 import vertexShaderSource from '../shaders/gsplat.main.vert.glsl'
 import fragmentShaderSource from '../shaders/gsplat.main.frag.glsl'
 export const gaussianSplatShaders = {
    main_frag: '\n' + fragmentShaderSource + '\n',
    main_vert: '\n' + vertexShaderSource + '\n',
    pars_frag: fragmentParsShaderSource,
    pars_vert: vertexParsShaderSource,
 }
--- a/plugins/gaussian-splatting/tsconfig.json
+++ b/plugins/gaussian-splatting/tsconfig.json
 {
    "compilerOptions": {
        "baseUrl": "./src",
        "rootDir": "./src",
        "allowJs": false,
        "checkJs": false,
        "skipLibCheck": true,
        "allowSyntheticDefaultImports": true,
        "experimentalDecorators": true,
        "isolatedModules": true,
        "module": "es2020",
        "noImplicitAny": true,
        "declaration": true,
        "declarationMap": true,
        "declarationDir": "dist",
        "outDir": "dist",
        "noImplicitThis": true,
        "noUnusedLocals": true,
        "noUnusedParameters": true,
        "removeComments": false,
        "preserveConstEnums": true,
        "moduleResolution": "node",
        "emitDecoratorMetadata": false,
        "sourceMap": true,
        "target": "ES2021",
        "strictNullChecks": true,
        "lib": [
            "es2020",
            "esnext",
            "dom"
        ]
    },
    "include": [
        "src/**/*"
    ],
    "exclude": [
        "node_modules",
        "**/*.spec.ts",
        "dist"
    ]
 }
--- a/plugins/gaussian-splatting/typedoc.json
+++ b/plugins/gaussian-splatting/typedoc.json
 {
    "extends": [
        "../../typedoc.json"
    ],
    "entryPoints": [
        "src/index.ts"
    ],
    "name": "Threepipe Gaussian Splatting Plugin",
    "readme": "none"
 }
--- a/plugins/gaussian-splatting/vite.config.js
+++ b/plugins/gaussian-splatting/vite.config.js
 import {defineConfig} from 'vite'
 import json from '@rollup/plugin-json';
 import dts from 'vite-plugin-dts'
 import packageJson from './package.json';
 import license from 'rollup-plugin-license';
 import replace from '@rollup/plugin-replace';
 import glsl from 'rollup-plugin-glsl';
 import path from 'node:path';
 const isProd = process.env.NODE_ENV === 'production'
 const { name, version, author } = packageJson
 const {main, module, browser} = packageJson
 const globals = {
    'three': 'threepipe', // just incase someone uses three
    'threepipe': 'threepipe',
 }
 export default defineConfig({
    optimizeDeps: {
        exclude: ['uiconfig.js', 'ts-browser-helpers'],
    },
    base: '',
    // define: {
    //     'process.env': process.env
    // },
    build: {
        sourcemap: true,
        minify: false,
        cssMinify: isProd,
        cssCodeSplit: false,
        watch: !isProd ? {
            buildDelay: 1000,
        } : null,
        lib: {
            entry: 'src/index.ts',
            formats: isProd ? ['es', 'umd'] : ['es'],
            name: name,
            fileName: (format) => (format === 'umd' ? main : module).replace('dist/', ''),
        },
        outDir: 'dist',
        emptyOutDir: isProd,
        commonjsOptions: {
            exclude: [/uiconfig.js/, /ts-browser-helpers/],
        },
        rollupOptions: {
            output: {
                // inlineDynamicImports: false,
                globals,
            },
            external: Object.keys(globals),
        },
    },
    plugins: [
        isProd ? dts({tsconfigPath: './tsconfig.json'}) : null,
        replace({
            'from \'three\'': 'from \'threepipe\'',
            delimiters: ['', ''],
        }),
        replace({
            'process.env.NODE_ENV': JSON.stringify(isProd ? 'production' : 'development'),
            preventAssignment: true,
        }),
        glsl({ // todo: minify glsl.
            include: 'src/**/*.glsl',
        }),
        json(),
        // postcss({
        //     modules: false,
        //     autoModules: true,  // todo; issues with typescript import css, because inject is false
        //     inject: false,
        //     minimize: isProduction,
        //     // Or with custom options for `postcss-modules`
        // }),
        license({
            banner: `
        @license
        ${name} v${version}
        Copyright 2022<%= moment().format('YYYY') > 2022 ? '-' + moment().format('YYYY') : null %> ${author}
        ${packageJson.license} License
      `,
            thirdParty: {
                output: path.join(__dirname, 'dist', 'dependencies.txt'),
                includePrivate: true, // Default is false.
            },
        }),
    ],
 })