VkCV Framework issueshttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues2023-08-11T19:15:28Zhttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/146Rework of asset loader module2023-08-11T19:15:28ZTobias FrischRework of asset loader moduleThe current asset loader (https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/79) works but it's far from perfect. In multiple cases the API is inefficient or gets in the way by not providing helpful options.
Here's a list ...The current asset loader (https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/79) works but it's far from perfect. In multiple cases the API is inefficient or gets in the way by not providing helpful options.
Here's a list of issues I collected over time which needs to be addressed:
- https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/118
- https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/121
- https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/135
- https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/136
- https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/138
- https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/140
- https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/141
- https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/145
I suggest an abstraction layer with a class for each type of asset (Texture and Mesh) which can be retrieved by Loader-instances depending on the format of the underlying resource (memory or file). Then Converter-instances can be linked directly to the Loader-instances to execute in asynchronous manner. If possible such an API could be extended to allow loading any kind of resource via a directed graph structure (potentially adding shader compilation and pipeline creation later on).
Such a directed graph would allow processing independent paths in parallel via multiple threads without running into race conditions.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/145Improve texture loading2023-08-11T19:15:29ZTobias FrischImprove texture loadingOne of the major bottlenecks in initial startup performance is the loading of all textures. These textures are mostly PNG images loaded via stb. That's not extremely fast if you look at comparisons online. Obviously we could address this...One of the major bottlenecks in initial startup performance is the loading of all textures. These textures are mostly PNG images loaded via stb. That's not extremely fast if you look at comparisons online. Obviously we could address this issue by loading the textures via multiple threads or potentially in an asynchronous way. However I think we should first make sure there isn't an easier solution to apply.
So this is what I found which could improve loading speed quite a bit:
- https://github.com/google/wuffs/
Here is a mirror of the release only provided as a single source file (this would make sense to use as git submodule):
- https://github.com/google/wuffs-mirror-release-c/tree/main/release/c
One example how to use it for loading images can be found here:
- https://github.com/google/wuffs/blob/v0.3.0-beta.1/example/imageviewer/imageviewer.cchttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/144Slang Compiler2023-09-19T18:23:44ZTobias FrischSlang CompilerSince the HLSL support in glslang is still improving, slang might be another option to compiler some shaders. The project is also licensed under MIT license which seems convenient. Here's the repository:
https://github.com/shader-slang/...Since the HLSL support in glslang is still improving, slang might be another option to compiler some shaders. The project is also licensed under MIT license which seems convenient. Here's the repository:
https://github.com/shader-slang/slanghttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/143AR/VR Support via OpenXR2023-04-27T11:33:13ZTobias FrischAR/VR Support via OpenXRFor a framework made for projects and applications around computer visualistics there should be support for augmented or virtual reality interfaces. One of the most wide-spread APIs in that field is OpenXR which could be supported by the...For a framework made for projects and applications around computer visualistics there should be support for augmented or virtual reality interfaces. One of the most wide-spread APIs in that field is OpenXR which could be supported by the VkCV. There are also open-source implementations available for reliable testing purposes.
Here is the SDK (source and header files) to implement OpenXR support:
https://github.com/KhronosGroup/OpenXR-SDK/
Here is an open-source implementation of OpenXR for Linux, Windows and Android:
https://gitlab.freedesktop.org/monado/monadohttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/142Add optimized algorithms2023-03-08T11:25:57ZTobias FrischAdd optimized algorithmsFor the sake of building more optimized applications, it would be useful to have libraries like following as module integrated in the framework:
- https://github.com/intel/x86-simd-sortFor the sake of building more optimized applications, it would be useful to have libraries like following as module integrated in the framework:
- https://github.com/intel/x86-simd-sorthttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/141DDS support for loading images2023-08-11T19:15:29ZTobias FrischDDS support for loading imagesDDS is a file format which supports mipmaps internally. It would be neat to have support in the asset loader module. There's also a header-only library on Github under MIT license: https://github.com/redorav/ddspp
Looks quite fitting!DDS is a file format which supports mipmaps internally. It would be neat to have support in the asset loader module. There's also a header-only library on Github under MIT license: https://github.com/redorav/ddspp
Looks quite fitting!https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/140Integrate MikkTSpace for calculating tangents2023-08-11T19:15:29ZTobias FrischIntegrate MikkTSpace for calculating tangentsThe implementation of MikkTSpace is relatively common to calculate tangents automatically using the UV-space. It would be a great enhancement to have in the asset-loader module.
https://github.com/mmikk/MikkTSpaceThe implementation of MikkTSpace is relatively common to calculate tangents automatically using the UV-space. It would be a great enhancement to have in the asset-loader module.
https://github.com/mmikk/MikkTSpacehttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/139DLSS Support2023-02-10T19:46:07ZTobias FrischDLSS SupportTechnically someone could implement DLSS support in the upscaling module for NVIDIA GPUs on Windows and Linux. [Here](https://github.com/NVIDIA/DLSS) is their Github repository offering the required headers and libraries.
I personally d...Technically someone could implement DLSS support in the upscaling module for NVIDIA GPUs on Windows and Linux. [Here](https://github.com/NVIDIA/DLSS) is their Github repository offering the required headers and libraries.
I personally don't think it's ideal to have binaries in place with proprietary code but maybe NVIDIA will release the source code one day as with PhysX for example. It definitely provides a high quality option for upscaling with Vulkan support.
Theoretically there's also another option from Intel with [XeSS](https://github.com/intel/xess). But currently it only supports DirectX 12.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/138Index buffer type mismatch2023-08-11T19:15:29ZTobias FrischIndex buffer type mismatchCurrently an index buffer uses the exact type as defined in the scene file when processed by the asset loader. This causes quite a bit manual adjustments when indices are provided directly to a shader. So it would be good if there was an...Currently an index buffer uses the exact type as defined in the scene file when processed by the asset loader. This causes quite a bit manual adjustments when indices are provided directly to a shader. So it would be good if there was an option to automatically convert indices for a targeted pipeline.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/137Proper game settings as module2023-01-11T19:30:28ZTobias FrischProper game settings as moduleFor PC games it could be useful if all the points described in the following video are implemented as module:
https://www.youtube.com/watch?v=Kr7RGkFuPdQ
This would make the framework more versatile in usage.For PC games it could be useful if all the points described in the following video are implemented as module:
https://www.youtube.com/watch?v=Kr7RGkFuPdQ
This would make the framework more versatile in usage.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/136FBX support for loading scenes2023-08-11T19:15:29ZTobias FrischFBX support for loading scenesGLTF is obviously the preferred format of choice when it comes to loading scenes. But there might be assets available as FBX file for example. It would be neat if the framework would just support those files as well and not require a con...GLTF is obviously the preferred format of choice when it comes to loading scenes. But there might be assets available as FBX file for example. It would be neat if the framework would just support those files as well and not require a conversion upfront. There is a project which can be used as dependency for that:
- https://github.com/nem0/OpenFBXhttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/135Support for compressed texture formats2023-08-11T19:15:29ZTobias FrischSupport for compressed texture formatsCurrently there is only limited texture format support by the supported formats stb can handle. So maybe this should be extended in the asset loader module with some other dependencies like this for example:
- https://github.com/inequati...Currently there is only limited texture format support by the supported formats stb can handle. So maybe this should be extended in the asset loader module with some other dependencies like this for example:
- https://github.com/inequation/detexhttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/134Shader re-compilation2022-12-08T23:28:10ZTobias FrischShader re-compilationThis is a long time issue because of the problems supporting all those different platforms at once. We never got file tracking into the framework and therefore shaders won't get automatically re-compiled if necessary for debugging shader...This is a long time issue because of the problems supporting all those different platforms at once. We never got file tracking into the framework and therefore shaders won't get automatically re-compiled if necessary for debugging shaders during runtime (this would be extremely useful though).
So in some applications there needs to be a button triggering re-compilation of certain shaders and creating new pipelines replacing the old ones. However this could be automated using file tracking from this here:
- https://github.com/SpartanJ/efsw
It works across all major operating systems, needs zero extra dependencies and it's published under MIT license. Pretty amazing!
The aftermath of implementing automatic re-compilation is obviously re-creating and replacing pipelines... this could be a bit more work though. ^^'
Still this could be a starting point for changing the API into a more asynchronous framework (optionally of course). So for example the shader compiler already takes a callback for the event of the finished compilation. If creating a pipeline and rendering could all be chained with own callbacks. This would create an automatic pipe/graph to react on changes in previous steps of itself.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/132Denoising Module2022-12-10T10:44:31ZTobias FrischDenoising ModuleSince ray tracing is now available for bigger scenes and meshes, it might be useful to look into denoising the image. For this there are multiple sources to potentially look into:
- https://github.com/GPUOpen-Effects/FidelityFX-Denoiser
...Since ray tracing is now available for bigger scenes and meshes, it might be useful to look into denoising the image. For this there are multiple sources to potentially look into:
- https://github.com/GPUOpen-Effects/FidelityFX-Denoiser
- https://github.com/OpenImageDenoise/oidn
- https://github.com/NVIDIAGameWorks/RayTracingDenoiser
- https://github.com/nvpro-samples/vk_denoise
I assume the FidelityFX-Denoiser shaders should be most interesting because they are published under MIT license matching the framework.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/131Need to track DeviceAddress for secure ref-counting2022-11-29T23:14:59ZTobias FrischNeed to track DeviceAddress for secure ref-countingAs mentioned in the PR [here](https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/merge_requests/112#note_151590), we need to track device addresses from functions like `getBufferDeviceAddress()`. Otherwise it might happen that a b...As mentioned in the PR [here](https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/merge_requests/112#note_151590), we need to track device addresses from functions like `getBufferDeviceAddress()`. Otherwise it might happen that a buffer gets destroyed and its memory freed because of the handles ref-counting despite its address being in use. In worst case this can corrupt memory accesses and reset a GPU.
So the framework should protect users/developers from such occurrences by providing a new type of handle for device addresses.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/130MaterialX support2022-11-20T22:29:53ZTobias FrischMaterialX supportEs gibt aktuell bereits ein Modul für vereinfachte Materialien im VkCV-Framework. Dieses könnte ergänzt bzw. erweitert werden mit Support für das MaterialX-Format. Dies sollte sich mittels CMake recht einfach einbinden lassen und der Vor...Es gibt aktuell bereits ein Modul für vereinfachte Materialien im VkCV-Framework. Dieses könnte ergänzt bzw. erweitert werden mit Support für das MaterialX-Format. Dies sollte sich mittels CMake recht einfach einbinden lassen und der Vorteil gegenüber der aktuellen Implementation wäre eine breite Unterstützung diverser Materialien:
Das Repository von MaterialX:
https://github.com/AcademySoftwareFoundation/MaterialX
Mehr Informationen zu MaterialX:
https://materialx.org/
Eine Liste von verfügbaren Materialien in diesem Format:
https://matlib.gpuopen.com/main/materials/allhttps://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/129Projekte und Assets auslagern2022-10-08T20:26:42ZTobias FrischProjekte und Assets auslagernDie Assets werden zwar hauptsächlich über Git-LFS gemanaged, aber es zieht trotzdem relativ viel Speicher. Das ist allerdings völlig unnötig für den Fall, dass man das Framework ohne Beispiele oder nur mit den Modulen verwenden möchte.
...Die Assets werden zwar hauptsächlich über Git-LFS gemanaged, aber es zieht trotzdem relativ viel Speicher. Das ist allerdings völlig unnötig für den Fall, dass man das Framework ohne Beispiele oder nur mit den Modulen verwenden möchte.
Hier wäre deutlich sinnvoller, die Projekte in ein separates Repository auszulagern und dieses als Submodul einzubinden.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/127Moving reflection from ShaderProgram into a separate module2022-09-05T13:35:52ZTobias FrischMoving reflection from ShaderProgram into a separate moduleIt might make change to move the code for shader reflection into a separate module because of licensing reasons. It would also reduce the core dependencies of the framework quite a bit.It might make change to move the code for shader reflection into a separate module because of licensing reasons. It would also reduce the core dependencies of the framework quite a bit.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/126Custom border color in sampler2022-09-05T13:33:28ZTobias FrischCustom border color in samplerThe feature to set a custom border color for a sampler is not yet supported. It shouldn't take huge changes though.The feature to set a custom border color for a sampler is not yet supported. It shouldn't take huge changes though.https://gitlab.uni-koblenz.de/vulkan2021/vkcv-framework/-/issues/124DescriptorSets optimieren2022-07-22T09:44:02ZTobias FrischDescriptorSets optimierenEs gibt inzwischen eine Extension (VK_KHR_descriptor_update_template) in Vulkan 1.3, welche das Beschreiben von DescriptorSets etwas optimieren soll, gerade wenn man diese einzeln und vollständig beschreibt.
Hier könnte man überlegen da...Es gibt inzwischen eine Extension (VK_KHR_descriptor_update_template) in Vulkan 1.3, welche das Beschreiben von DescriptorSets etwas optimieren soll, gerade wenn man diese einzeln und vollständig beschreibt.
Hier könnte man überlegen das ins Framework einzubauen. Das könnte potentiell die Performance deutlich verbessern, je nach Anwendungsfall.
https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_descriptor_update_template.html