Year summary

As the year concludes I wanted to give a small update on what I’ve been working on. From a technology perspecitve this has been one of the most intense years in a long time! I’ve had the opportunity to spend most of my time researching new technologies and algorithms for our next voxel engine. Here’s a summary:

New renderer

The first half of this year I spent researching advancements in path tracing, learning about Vulkan and hardware raytracing. I implemented my findings as a prototype renderer for the new engine. It relies heavily on hardware raytracing (using intersection shaders, still no triangles!), importance sampling and removes a lot of the limitations we’re seeing with our current engine. For instance it no longer uses a shadow volume, which enables unlimited world size, sharp shadows, no light leakage and vast amounts of moving objects at a very small cost. It also enables true reflections and global illumination, since all scene information is available on the GPU. For denoising I’ve been using NVIDIA DLSS Ray Reconstruction, and overall I’ve been very pleased with the results. It tends to be a bit aggressive in some scenarios, but still very impressive technology. The prototype I made has since been taken over by my colleague Gabe Rundlett, who has done a lot of optimizations, improvemenents and cleanup.

Screenshot from original prototype

Deformable voxels

I’ve had this idea for some time, that voxels wouldn’t necessarily need to conform to a perfectly orthogonal box. I remember doing some experiments for Teardown to enable trees swaying in the wind, but I could never really get it to work properly with lighting beacuse of the shadow volume. Raytracing voxels inside an arbitrarily skewed bounding box was a much harder problem than I anticipated. The only related information on this problem that I could find was this paper by David Eberly of Geometric Tools. If you haven’t yet checked out Geometric Tools, it’s an invaluable source of information for all kinds of geometric problems. After e-mail conversations with Mr Eberly we came to the conclusion that an invertible affine transformation between a unit cube and an arbitrarily deformed cuboid does not exist, even if faces are co-planar (paper in link now updated). However, I managed to come up with an approximate solution that is good enough for my use case. It has limitations, but I’m quite happy with the results and can’t wait to integrate it properly into the new engine!

A static image doesn’t really do these sausages any justice. Link to video.

Framework update

The base framework, including vector math, containers, IO streams, threading, etc has gotten a major overhaul and cleanup. We had long internal discussions on wether we should reduce it in some areas and rely more on STL. I did some comparisons and the implementations in STL are for the most part really good these days (at least on msvc), but the long compile times, terrible debug mode performance and lack of control over memory allocations made us stick to our own implementations also going forward. One area I spent a lot of time with is the task manager, which distributes tasks on multiple threads. The Teardown engine is based around data parallelism, which I think has served us well and is something we want to keep, but in order to be efficient on modern machines it requires a task manager with fast synchronization and low overhead. In order to improve on this and fully embrace data parallelism going forward, I rewrote the task manager to use atomics and lock-less programming for fine grained task distribution and synchronization (instead of relying on operating system primitives, such as mutexes and condition variables). Lock-less programming is hard, and you never really know if you got it right, but it does make a big difference when used correctly.

New engine architecture

We have a couple of really talented programmers on the team who speaks very fondly of ECS and convinced me to read up on it and give it a try. For a long time I’ve not been that keen on ECS and I still can’t say that I am, but I have really appreciated learning about it. It does have some very compelling properties, both from an architectural and performance perspective, and while the new engine is not a “pure ECS” implementation, it does use the concept of components and systems where it makes sense. I have a lot to say about ECS and my experiences with it, but that’s not the topic of this post. The new engine architecture is also centered around a new “virtual property” system that connects entities to the editor UI, serialization, undo/redo and the scripting system in a relatively clean and unified manner.

Sparse voxel objects

Teardown uses a dense voxel format, where every shape stores voxel data in an uncompressed, regular 3D grid. This is very fast to access, but since it’s just a grid, regions of empty space within an object take up the same amount of memory as regions that actually contain voxels. This requires assets to be split up into multiple smaller shapes in order to get a better bounding box fit. The downside of this is of course that you end up with a lot of shapes, which also has an overhead. The new engine features a sparse voxel format that splits each shape into 8x8x8 voxel “chunks”, which are tracked with a 3D bitmap. This saves a lot of memory and enables shapes to put voxels everywhere inside the shape without worrying too much about emtpy space. From a technical perspective, the chunk-based format is more complex to work with, but it offers a lot advantages over the dense format - updates are fast, most algorithms can be implemented more efficiently, and it removes a lot of restrictions on voxel content. A welcome side effect is that shapes can now have voxels on negative coordinates, which greatly simplifies both modeling and procedural modification to existing objects.

Physics overhaul

Most recently and still ongoing I’ve been working on improving the physics simulation in the new engine. There has been several advancements in the field of game physics since I designed the physics engine for Teardown and I’ve been incorporating a selection of these improvements into the new engine. Erin Catto, author of Box 2D has recently done very extensive research in the field and his findings have been extremely valuable. If you are interested, you can read about his work in great details on the Box2D blog. The new physics engine now uses substepping instead of solver iteration (a method described here and sometimes refered to as “Temporal Gauss Seidel”) and features a parallel solver that can solve large piles of objects on multiple threads. There have also been improvements to contact generation and broad phase.

A large pile of voxel objects solved on multiple threads

If there is any particular area you’d like to know more about, let me know and I’ll go over it in a future blog post. As always, you can reach me on X @voxagonlabs