Depthmaps in the Source Filmmaker

 

Did you know that depth of field maps are present in certain Source games by default, such as Half-Life 2 (2004)? By using a console command 'r_depthoverlay 1', your in-game view changes to 'depthmap view', or how Valve likes to call it: a depthbuffer. Not all Source games contain depth of field information (like Left 4 Dead for instance) and so far I have only gotten it to work in Half-Life 2. There is some interesting information on the Valve Developer Wiki about this topic, yet I must say the information remains highly technical. It would take me quite some time to decipher everything that is on offer on the Wiki. Luckily, I don't have to decipher anything, as much of this information is obsolete (at least in my eyes if you take the purposes I use depthmaps for into account, as depth of field maps are implemented by default in the Source Filmmaker.

So, in a nutshell, what is the Source Filmmaker? First I'd like to give a bit of context of Source machinima-making. Let's wind the clock back 10 years. Pre-2012, machinimators had a couple of options to make machinima with the Source engine:

• Create animations in Garry's Mod. Garry's Mod is a physics sandbox game, originally released as a half-life 2 mod, and uses much of the source engine's assets and assets from valve's games. There is not so much a goal to Garry's Mod. You can build to your hearts content using custom content, or assets from various games by using many building tools or play together with people in countless different game servers. It is a toolbox that allows you to practically do anything you want. There is a plethora of workshop items, custom props, characters, npc's, weapons, vehicles, maps and scripts for this game that allowed you to make stopmotion style animations as well, place camera, lights and props. For accessible storytelling, Garry's Mod was often the place to go for machinimators. Think of the highly inspirational Garry's Mod series: 'Elliot Goes to School' by kitty0706 that took place in a school setting, made with assets from the game Counter Strike Source. Through voice-acting and stop motion animation, the viewer would embark on Elliot's often silly and exaggerated daily adventures at school, during field trips or during a pep rally. 
• In-game demo recording. In any source game, you are able to record demo's using the game console. Once done the demo UI (or demo smoother as it is officially called) could be triggered with the console command 'demoui'. This allowed you to play back a demo, change various settings such as FOV or activate console commands on a specific point in time (unfortunately this feature was heavily broken and a lot of people complained to never being able to get it to work, saw posts online as early as 2006), and not completely unimportant: to change the camera point of view and to establish camera tracks. Once the camera track was completed and smoothed out, your footage was then rendered with the same demoui to targa image sequences with a few additional console commands.
• Source SDK Tools. In-game demo recording within Source is and remains highly powerful. As a matter of fact, ONRIDE was made with only the in-game demo recording tools found within Source. But unfortunately you can only do so much with demo recording, as mentioned in a previous post, the player becomes the camera in the form of a camera-override. This is where the Source SDK tools come in. Hammer, for instance, allows you to build custom maps and edit existing ones. You can also set up entire scenes in Hammer and with face poser (a programme that allows you to create so-called choreography scenes and assign animations/actions to characters) and make it so that the entire scene automatically starts a few seconds after spawning the map. This gives you almost unlimited creativity. You could make a scene of two NPC's talking, or create a busy atmosphere of a hectic crowd within a shopping mall. Once in-game, you would still need to capture a demo containing all of the data and to edit the camera positions using the demo smoother.

During the summer of 2012, Valve released a beta of the Source Filmmaker, which is a tool that Valve used for their in-house cutscenes, marketing materials and trailers for each of their games. It allowed them to spawn a map, place characters, capture gameplay, adjust gameplay, place props, lights, particles and make characters talk with high quality lipsync. As Valve called it, it narrowed down the entire pipeline of a complete animation studio into one single piece of software. If you search for various Valve films such as 'meet the spy', or 'meet the heavy', or 'portal 2 investment opportunity' or 'left 4 dead 2 opening cutscene', you are in fact looking at marketing materials Valve made with this exact piece of software (or at least a similar build). It's learning curve is a bit steep, and to be fair I had a lot of trouble understanding how to actually use this software until I had a bit of 3D experience and experience with Source SDK. Now that I am up to speed, I find it such a remarkable machinima tool and highly suitable for the experimental films I work on. There are quite a few shots made in DUSK for instance in where the lighting in combination with the camera work worked very well in my opinion.

Now, let's talk depthmaps. I forgot about this feature until I was well underway with production for DUSK. That is why only a few of the shots contained depthmaps, such as the shots of the clown zombie standing in the fire pit. For some reason, the depth of field maps require a lot of exposure in order to be properly visible. I am still unsure why this is. I created two examples, which can be seen underneath. The first scene is a completely lit shot in a Team Fortress 2 map with some custom props and a camera containing depth of field. The second scene is just a part of another Team Fortress map where I simply placed a camera, added some depth of field parameters and called it a day. I did not change the exposure of the camera. As I did turn down the exposure for the first shot of the camera by a lot in order to enhance the lighting, I think the toned down exposure of the camera was what affected the depth of field renders as they ended up being completely black. In fact, I should have done it the other way around. I should have turned down the lighting and turned up the exposure of the camera. However, then the emphasis would be placed more on the standard in-game lighting that was present in the map, which was something I did not want for my result as you would want to step away from the standard ambient lighting and create contrast inside of your shot.

 The second scene contained shots with standard exposure values and the depth of field map corresponded by giving standard values as well. Therefore a careful first conclusion is that depth of field maps in the Source filmmaker do not work well in highly stylized scenes. If I were to create a DEPTHMAP Source edition of some kind, I would need to use the map 'as-is', and only adjust the exposure of the camera in order to get the end result I want. If I were to make it in a DEPTHMAP NEXT kind of style, I would need to create two instances of the shot: the stylized version and the depthmap-friendly version as DEPTHMAP NEXT combined both versions of the shot: the original and the depthmap.

 

Depthmaps are rendered in the Source Filmmaker by toggling the 'Write PFM Depth file' in the export movie tab. Of course, you would still need to enable progressive refinement in order for all secondary effects (motion blur, dof) to show up and to set up the settings accordingly. Thirdly, you would need to set the aperture parameter in order to define how heavy the depth of field would be. Once you click on export movie, the film will both render a wav file, the standard image sequence and the PFM depth sequence. PFM files can easily be converted using Photoshop to TGA/PNG using the image processor.

Example 1

Although I won't share the final depth of field map result, as the PFM file was completely black (due to the exposure of the camera being significantly turned down to create contrast and to get rid of the in-game ambient lighting), I wanted to share a few screenshots of the first test scene. I chose a corner in a TF2 map and added a couple of lights and props:

Setup with a picnic table and a small medic suitcase standing on the table.

Model browser: adding a locker in front of the camera to create depth

locker is placed in front of the camera

First light is roughly placed

Second light is roughly placed and DOF parameters are set. You can see faint depth of field in the shot.

Example 2

I opened up a second TF2 map, placed a camera in there without any additional props, lights or adjusting any of the settings. The end result was a standard depth of field map which then was enhanced in After Effects

Original file

Enhanced file

Original shot - also added the depthmap as a camera blur effect to blur the rocks in the front.

Cheers and until the next update!