Reality Capture Basics

As a part of the curriculum I’m taking through Studio Arts, I’m taking a survey course in virtual production and the use of Unreal Engine for previsualization. The class is taught by Damian Allen, a vfx supervisor and previs artist.

Key Software

The primary tool we used was Reality Capture, which was acquired by Epic Games and is now free for use with Unreal. Reality Capture is primarily for photogrammetry, and excels at generating 3D models based on reference images.

By comparison, another popular technique is gaussian splatting, which generates a point cloud much like photogrammetry, but then each point is used to render the final image. Gaussian splatting is also unique in that it’s capable of parsing transparency, and can render fine detail far beyond what photogrammetry can typically capture. Chaos Group has an excellent blog post about this technique. NeRF Studio is one of the primary tools used to train neural networks based on point cloud data to create a gaussian splat. NeRF in this case stands for Neural Radiance Fields. Gaussian splatting does not explicitly require a neural network to be used, but can benefit from it. If you’ve used a real estate website like Zillow and used their virtual tour features, you’ve already encountered a simple version of a gaussian splat, where they use a low detail gaussian splat to create an interactive tour of a room.

In addition to Reality Capture, it’s worth mentioning Meshroom, a popular open source tool for photogrammetry that relies on the AliceVision network for generating meshes based on input photographs.

Building a Desert Landscape in UE5

As a part of a class I’m taking through Studio Arts here in LA, taught by Milton Mariscal, I’m working on a cinematic in Unreal. The first key step to this is to setup a landscape environment to use as the backdrop for the animation and short film.

I ended up doing several of these, in part through trial and error and also because I had several ideas for environments I wanted to experiment with, each with their own biome and geology.

I have a few of these which I’ll document, the first of which is a desert landscape that I’ve been working on. I took inspiration in part from a trip I took to Uluru/Kata Tjuta National Park in Australia. Much like the American southwest, Northern Territory and Western Australia have some of the most remarkable ‘desertscapes’ and geology in the world.

To start I put together a mood board, shown below:

The center photos show Kata Tjuta, which is my primary reference for rock texture. I also included several photos of Monument Valley in Utah, which I used primarily for structural reference.

Painting the Landscape

Using only UE’s Landscape mode I was able to paint a complex organic landscape, including ground transitions and buttes much like the sandstone formations in Utah. I primarily relied on the Sculpt, Smooth, and Flatten tools to achieve the shape, as well as the Erosion and Noise brushes to introduce some indication of water erosion and general weathering. The landscape system isn’t a substitute for something like ZBrush or Houdini for procedural landscapes, but is great for quickly prototyping a base mesh.

Setting the Mood Through Lighting

My goal was to create a classic western sunset feel, drawing on reference from my time in Australia, as well as Los Angeles. Both Sydney and Los Angeles have very similar landscapes and sunset colors, though LA’s are much more influenced by its pollution levels. Air temperature, position relative to the equator, and climate all play a role in the appearance of the sky. In researching this I found that in fact Los Angeles and Sydney are roughly similar distance from the equator, at around 2,300 miles. While the seasons are opposite, because of their similar relative distance from the equator they may experience a similar angle of sunlight in the same season. In the image above you can see where I was experimenting more with the key light angle and intensity, as well as the atmospheric scattering.

A key control for determining the look of the sky in Unreal is the Rayleigh and Mie Scattering color. The Mie scattering color will determine the absorption for short wavelengths of light, while the Rayleigh scattering color determines the absorption of longer wavelengths of light. Put simply, when the sun is low in the sky it’s emitted light is scattered through a thicker layer of atmosphere. Therefore, the primary light that reaches the viewers eye is red, since that’s the longest wavelength of light in the visible spectrum. The result is a warmer look at the horizon at sunset. We can further adjust this by tinting the Rayleigh scattering color as well as the Mie scattering color on the Sky Atmosphere. Scratchapixel has a good overview of the history and science behind these algorithms.

Setting the rayeigh and mie scattering to blue and a slightly purple tone respectively results in a consistent warm tone on the ground and a warmer horizon.

The default clouds feel somewhat pixelated. I wanted a lighter, more abstract sky with softer cirrus type cloud. These were the settings I ended up with, including the scatter color settings.

Changing the sky fill level and reducing it as well as adjusting the sun intensity can yield a more dramatic higher contrast look.

Layered Materials for a Landscape

I pulled in a number of Quixel assets at raw quality to start generating a tileable, paintable material. I wanted to use a single shader for the landscape and blend between multiple materials to a convincing desert biome ground. One minor error I ran into is common in UE5, where it warns you about running out of texture streaming memory. To fix this I entered the following console command:

r.Streaming.PoolSize 3000

Prior to setting up my materials I also made a few quality changes. To work compromise a bit on render quality so I could work quicker I set my render settings to Epic, so that I could still preview the volumetric clouds. I also turned off auto exposure compensation so I could get a better sense of the landscape color and have a more consistent view while exploring the environment in play mode.

I clicked Settings > Project Settings, then searched for Auto Exposure to find this setting.  Leaving Auto Exposure Bias at 0.1 allows for some exposure compensation when looking at very bright highlights, but prevents the camera from constantly adjusting as I roam in and out of shadowed areas.

The next step is to put together a custom material network. The process is pretty simple - create a new material, right click, and edit. From there we can drag in our textures from the content drawer. In this case I’m starting with a rough desert rock texture as the basis. One important map to take note of is the ORM map - Occlusion, Roughness, and Displacement. To enable displacement we need to enable Nanite in UE5. To enable Nanite, we need to enable it in the Project Settings, as well as enabling the plugin.`

Enabling the Nanite plugin requires an engine restart.

We then need to close the engine, and edit some parameters in the .ini file under the config folder in our project director. Once we’ve done that we can then restart UE5 and work with Nanite displacement. There are 2 lines we need to add, shown below:

Allow Tessellation needs to be set to 1, or on, for Nanite tessellation to work.

UE5: Hotkeys

Navigation:

• RMB + WASD - navigate the camera, pan up/down/left/right

• F - frame on the selected object

• LMB - rotate view

• Alt + LMB - rotates view around the current view point

• Alt + LMB after F - rotates about the object once its framed

• Alt + RMB - dolly camera

• RMB + WASD - Game style movement

• LMB/RMB - Pan and rotate from current viewpoint

• Scroll wheel - zoom in and out

• CTRL + ` - toggles between world space and object/local space for transforms and rotation

Selection and Viewing:

• CTRL + LMB Click - select multiple objects in the outliner

• Q - selection key

• Alt + LMB drag - duplicate selection and move

• Double click on object in the Outliner - selects, frames on the object and locates it in the scene

• Alt + G - Perspective view

• Alt + H - Front view

• Alt + K - Side view

• Alt + J - Top view

• CTRL + 0-9 - Book mark camera view. Press the corresponding number to jump to that view

Level Editing:

• CTRL + Space - Opens the drawer

• Shift + 1-8 - switch between editing modes, ie. Selection, Landscape

• H - Hide selected

• CTRL + H - Unhide all hidden objects

• F11 - Immersive

• G - Game view

• CTRL + K - search in blue print

• End, or fn + End - Snap to floor, depending on your keyboard

• Alt + End - Snap pivot to floor

• CTRL + End - Snap origin to grid

• Shift + End - Snap bounds to floor

• ESC - clear selection

• ~ - Open command console

Viewport Shortcuts:

• F - focus on selection

• Alt + 2 - Wireframe view

• Alt + 3 - Unlit view

• Alt + 4 - Lit view

• Grid size decrease - [

• Grid size increase - ]

• Rotation size decrease - Shift + [

• Rotation size increase - Shift + ]

• Show mouse cursor in play mode - Shift + F1

Modes

• Shift + 1 - Select/Placement

• Shift + 2 - Landscape painting mode

• Shift + 3 - Foliage, paint foliage placement interactively using source assets

• Shift + 4 - Mesh paint

• Shift + 5 - Modelling toolset

• Shift + 6 - Fracture

• Shift + 7 - Brush editing, for editing box brushes, etc.

• Shift + 8 - Animation toolset

UI

• CTRL + Shift + W - opens the widget reflector, allowing you to globally change the application scale including icons and font

Stylized Shading in UE5: Part Three

In the previous Part Two installment I documented each component of the stylized material I made. By combining and layering together a lot of simple concepts you can easily make a more complex network that’s visually interesting.

In this section I want to expand upon that further and demonstrate the parameters, document the post process materials I developed to enhance the look, and finally bring everything together in a simple test level using the Third Person template, where we’ll run around a stylized woodland.

The models I used are from Quaternius • Free Game Assets. I could make some simple assets of course, but my feeling was it’s even more valuable to test a shader on assets that you didn’t model yourself, to ensure that it performs well regardless of the geometry you give it. I also like the style of the models he makes. They’re CC0 Creative Commons license, but I want to give credit as he was nice to give his permission for me to use them in this piece.

Post Process Shaders

In addition to the base stylized material, I wanted to experiment with some post process materials. In Part Two I showed a turn table of the shader where the outline width can vary slightly depending on the surface topology and the viewing angle using a fresnel effect. In addition to this I wanted to experiment with the following:

• General outline shader

• Desaturation based on player distance from an object

• Desaturation of the ground base on player location, where the ground is desaturated after a certain falloff distance from the player

• Alternative texturing of the shadows, where the shadows can have a hatched effect

For the alternative texturing of shadows, the goal was to not only have the object self shadowing be hatched, but also the ground shadows, or allow the user to feed in a custom texture. I got the idea for this from a post by Tom Looman: Textured Shadows Trick in Unreal Engine - Tom Looman. In this post he demonstrates how to access the Light Attenuation buffer and query an objects UVs to project a texture into the shadow regions. The Light Attenuation buffer is technically no longer accessible in UE5 to my knowledge, at least not easily. Render Bucket, an excellent YouTube channel details an alternative workflow for creating an equivalent ‘shadow catcher’ pass: Unreal Engine 5+ Shadow Catcher / Shadow Pass (youtube.com). Effectively

Desaturation of distant object and areas on the ground is easy to capture by sampling the depth buffer and applying an Overlay material to relevant objects. Overlay materials can be prohibitive in terms of performance, but in this case the operation is simple, where I’m just desaturating the primary color.

An outline material relies on basic edge detection. There are a number of algorithms that allow for this, generally all relying on kernel convolution. A notable operator is the Sobel filter: Finding the Edges (Sobel Operator) - Computerphile (youtube.com). Laplachian style filtering generates a more complex edge with more interior details. Edge Detection Using Laplacian | Edge Detection (youtube.com) In this case I really only wanted the external edge, so relied on a Sobel style operator.

A key source for ideas on how to implement these effects in UE5 came from a GDC 2024 talk that actually released while I was in the midst of working on this. Chris Murphy of Epic Games covered a huge amount of information in just 30 minutes: Simple Stylization Techniques in Unreal Engine | GDC 2024 - YouTube. This talk shed light on how to access the correct frame buffers to use in my post process shaders.

Outline Shader

As mentioned the main additional post process effect I wanted was an outline material, to compliment the edge detection in the base material. By creating a more uniform black outline my feeling was this would enhance the read of objects and improve visual clarity.

Creating an outline material is fairly simple. We can change the material assigned to the Post Process Volume to overlay an outline over the entire scene by querying the incoming color and implementing an edge detection algorithm. It sounds complicated, but it’s actually very easy to implement in UE’s shading network.

Stylized Shading in UE5: Part Two

Translating the Gooch shading equation to HLSL

In Part One of this short series I’m putting together I talked about the basis for stylized shader, which is essentially a Gooch shading model. Part One

Gooch style shading, developed primarily by Amy Gooch and Bruce Gooch, has origins in technical illustration, where visual clarity is critical. Unlike Lambertian shading, there is no falloff to black in the shadow areas, but rather a predetermined color value. This creates a feeling of indirect bounce lighting, and is inherently similar to a cel shaded or two tone color look. Many stylized games seem to make use of this style of shading, where colors in the midtones and shadow regions are carefully controlled to create a softer look that mimics global illumination, without the use of expensive GI calculations or even baked solutions - it can all be achieved in shader.

Acerola has an excellent breakdown of this shading model on his tech art focused YouTube channel. I took some inspiration from this video, along with the book Non Photorealistic Rendering that I mentioned in Part One.

Below I show 2 samples of the Gooch component of the shader, which creates the soft gradient color transition on the ground plane. This serves 2 purposes - as mentioned, it’s aesthetically somewhat similar to global illumination, and it also adds visual clarity, distinguishing the curvature of the ground plane. Normals pointing upward get a lighter color green, while normals pointing away from the light vector direction of the keylight get a darker teal/green tone rather than black in Lambertian shading. To me this is a hallmark of stylized games, where the player has visual clarity and shadows are generally defined by darker monochromatic colors.

Preview image showing the Gooch shader applied to a ground plane, showing a transition from cool to warm colors in this case.  In the background a ground plane shows a transition from a teal to light green in a more traditional monochromatic color scheme.

To create this effect, I needed to translate the original equation into format that would work with Unreal Engine’s node based shading workflow. Effectively, with the node based implementation of HLSL in UE5, you need to do operations in reverse. So our goal is get color final as the output.

The ‘cool’ color or top color equation is: 1 + the dot product of the incoming light vector, dotted with the surface normal, all divided by 2, multiplied by the kcool RGB color value. This results in a gradient, where as the normal direction is further away from the sun vector, the influence of the cool color decays to zero, and we see more of the ‘warm’ color.

The ‘warm’ color or shadow side color equation is effectively: 1 - (1 + the dot product of the incoming light vector dotted with the surface normal, divided by 2) multiplied by kwarm. The result is then where the surface normal is pointing away from the sun, we transition to the warm color. The result of this and the corresponding HLSL nodes are shown below.

Preview of the Gooch shading algorithm on a sphere, showing the soft transition from the cool color to warm color in the shadows.  In this case I made the warm color quite bright to illustrate the creative control it allows over the shadow appearance.

There are many components to this shader. These are the key features I wanted to implement:

• Cross hatching effect to emphasize shadowed areas and form of the object

• A stylized outline similar to cel shading that would vary in thickness based on the object controu

• A secondary post process outline, uniform in thickness to help increase visual clarity

• A stylized specular response, similar to a hand drawn highlight, with a clean circular look

• Posterization, where the user can introduce a posterize or banding effect in the shader, to make it more similar to a graphic cel shaded look

• An overall specular component that will allow the user to differentiate material types while still looking stylized. Similar to the effect seen in Breath of the Wild, where many object have a broader specular layer.

The complete shader graph.  By combining many simple components, I was able to come up with something that had all the features and artist control I wanted to implement to start.

Cross hatching in HLSL using a simple sine wave

Above I show the complete shader network. Highlighted in red is the Gooch component I just explained in the previous section. Below to the left is the next key component, the cross hatch effect.

The goal of this as mentioned was to allow the user to emphasize the shadow regions of an object with a hatching effect that actually follows the geometry contours. There are many implementations of a hatch shader as a post process effect that look excellent. But temporal stability is poor generally, with obvious swimming and other artifacts as the player moves. Because of this, I chose to implement this effect in shader, using the surface normals and luminance.

I got the basics of the idea from this thread on the Nuke forums, where users were discussing how to achieve a Spiderverse style hatching effect: Halftone Gizmo.

I’ve implemented this in Nuke in the past. The concept is simple, but can be expanded upon to create some interesting artistic effects. Effectively all I’m doing is feeding a Linear Gradient node into a Sine wave, and multiplying it by a Constant in HLSL to control the frequence of the wave, as shown below:

Here I’m previewing the Linear Sine node applied to a sphere, which generates a hatched texture.  Hatch Up Angle controls the direction of the hatching.  I later use the surface normal and pixel normal to isolate the darker midtone areas where I want the hatch effect to appear.

The next step is to restrict the hatching effect. In this case I didn’t want excessively dark, dense shadowing, but mainly wanted to use the hatching to highlight the contours of an object and add definition to the mid tones. I use the nodes labeled Black Point and White Point to effectively control the contrast. Below I show how I used the dot product of the Vertex Normal and Sun Direction and multiply that against the Pixel Normal green component(the shadow direction) to isolate the midtones. The result is a black and white mask with a falloff that I multiply against the incoming hatch texture to create what’s shown below.

A mask isolating the mid tone shadow regions.

Previewing the hatch color.  This will be multiplied against the original to allow the user to tint the color of hatching.  In this case I’m keeping it monochromatic and just choosing a darker, less saturated version of the input. 

Cel shading banding effects, edge detection, and stylized specular

The final portion of the shader applies a banding effect to the incoming Gooch shading color, an edge outline, false stylized spec highlight, and a general spec layer. Adding these on top of the incoming result enhances the look, and gets a bit closer to

Combining the hatched effect with banding and the incoming color yields a more complex toon shaded result.

Creating the banding effect is fairly simple. Again I take the dot product of the incoming Sun Vector compared with the Pixel Normal in World Space, rather than the Vertex Normal. I then multiply that by a Step Number, in this case 8, for 8 bands and divide by the Step Number and Clamp. The end result is a stair stepped color value. Rather than a smooth gradient, it clamps to 8 different color value, yielding a stylized ‘lofi’ shaded result. This contributes to more of a ‘low poly’ feel that simplifies the look of objects in the scene.

Above is the result of my edge detection setup, which is really just a slight modification of a Fresnel node.

To create an edge detection effect, we can use a Fresnel node. However like Lambertian and Gooch shading, the result is relatively smooth initially. To create a hard edge outline, I again implement the same Clamp, Divide, and Round to create a banding effect to isolate the Fresnel to a restricted area. The Exponent Edge Thickness is just a float constant that I use to control the thickness of the outline. On a basic sphere shape, the outline is consistent. But on an object with more surface variation, the Pixel Normal will influence the apparent edge thickness. The banding effect in this case really only has 1 step, because we only want to return a black outline and a white interior, rather than multiple steps of gradation in value. Color Tint Outline is a color parameter I’m using to tint the outline so that’s its not fully black. Much like in Breath of the Wild, I chose the outline color based on the base color, in this case a brown tone to compliment the original orange value.

Adding a round toon style specular highlight and overall specular.

To create this spec hit I again add the Sun Vector to the Camera Vector, then Normalize it and take the Dot product compared with the Pixel Normal. A Saturate node ensures that the value range stays within 0 to 1. I then use a Power node and feed it a float constant I labeled Specular Power, then Round and Clamp the output. Taking the power to 12 restricts the highlight to a small area.

The stylized spec result.

Combining all these elements together yields an interesting stylized result. The final component I add on top is a stylized fresnel rim. Again I use the Fresnel node, clamp it and restrict to the top edge in the direction of the sun, and multiply it against a preset color.

Adding a separate edge fresnel rim to define the edges of the sphere.

Putting all together, in the video below I preview the result of the material applied to a 3d headscan to test the parameters.
In Part Three I’ll demonstrate using this material to look dev a stylized environment for a game, and also go into some post process materials to complete the effect.

Virtual Production - On Set for Europa

Recently I had the opportunity to reconnect with a former colleague, Eric Roth, who is serving as VFX Supervisor on a short film being made in collaboration with USC Film School. He invited me to participate in post production vfx work as well as assisting in data acquisition on set. For me it was a great excuse to get to see Pixomondo’s new LED volume. The volume utilizes Sony’s Crystal LED displays which you can read more about here: Crystal LED BH displays

The volume is located at Sony Pictures in Culver City on a sound stage managed by Pixomondo and built/financed by Sony Innovation Studios. Sony Pictures Virtual Production Stage

The shoot also provided an opportunity to test out Cooke cinema prime lenses, as well as the latest Sony Venice digital cameras, with revised firmware and features, including LED volume sync to mitigate flickering issues associated with filming a digital display.
Sony Venice cameras

The stage is being used to create the virtual locations for Europa. If you’re a bit of an astronomy/space buff you’ll know this is one of the moons of Jupiter, which has been theorized to have a liquid ocean beneath its icey surface. As a result the focus of the environment centers around creating a large ice cave environment, as well as the lunar surface.

In addition to visiting set, I also got to take some of the props to be scanned by The Scan Truck as a part of my informal duties as ‘vfx data wrangler’. They’re known for their work on For All Mankind in scanning and photogrammetry of assets, so of course they were the right choice to scan and capture data on the costumes, which were replica astronaut suits. Work will be done in post to replace and/or augment the helmet, particularly the visor to reflect the surrounding environment. Over the course of a few hours they completed full lidar scans of the helmets as well as texture acquisition. They supplemented this data with a full 360 capture of the helmets in their photogrammetry setup, comprised of dozens of Canon slr cameras mounted on a cage setup inside their truck. The setup is fully portable, intended to drive on to shooting locations.

Snapshot of the helmet inside the photogrammetry capture cage.

More to come as production wraps over the next few weeks and the team moves into post production work.