Heavenly Virtual Production. – fxguide
“Philly Heaven” was recently shot on the Pixomondo’s and William F. White International’s Virtual Production stage. There are two stages at the studio, one is in heavy use by Star Trek (see below) and the other is being increasingly booked out for commercial and shorter duration productions.
The director was Leigh Marling and David Whiteson was the Creative Director / VFX Supervisor. Phil Jones was the Pixomondo’s (PXO) VFX supervisor. “Philly Heaven” also brought DOP Christopher Mably back to shoot with PXO. He had previously filmed a number of environments at the larger Toronto stage with Star Trek: Discovery. The spot was shot on Alexa. The Alexa and Sony Venice have been the most used platforms in the Pixomondo volumes, but the team has shot with other cameras such as the Blackmagic line of cinema cameras and various Red cameras.
Philadelphia Cream cheese was shot on the smaller stage, which has an adjustable ceiling that is separated into 4 sections so that the DOP can raise, lower, and tilt the individual panels to adjust the lighting and reflections cast from the ceiling. This has been particularly useful when shooting with very reflective practical surfaces such as cars and is also useful in helping to direct the light from the LEDs a bit more than when the ceiling is flat.
The wall was run from a project that was created and presented in the volume with Epic’s UE 4.27. Pixomondo is currently testing UE 5 in environment creation as well as presentation in their volumes, and the team is keen to get the new features of UE5 onto their stages and they would be doing this in the near future!
The Philadelphia Cream cheese commercial was shot over a 2-day period. Day one consisted of a 1/2 day traditional live-action shoot on a small practical kitchen set that was built in the same studio, adjacent to the LED volume. In the second half of day one, the team worked with the DOP to blend the practical set to the environment in the volume. The main shoot in the volume was on day two, where they shot the “Philly Heaven” heaven environment portion of the commercial.
This is on the smaller volume known as Stage 6. It is a permanent installation with 915 total panels. The panels are – 2.84mm (BP2v2)- 62ft/ø, 20ft tall, 31ft deep- 16,100 sq/feet. The tracking was done with Optitrack6.
David Whiteson is the Creative Director / VFX Supervisor at post-production house alter ego. When the team was approached by the agency with this script and they saw that the entire commercial takes place in the heavens above the clouds, and immediately knew this would be a perfect job for the LED Wall for many reasons. “The first being the creative opportunity to have the interactive lighting from the wall beautifully wrap around our characters and our angel’s wings,” David explains. The second reason was due to the schedule. “We were tasked with creating entire backgrounds for two : 30-second commercials with a post-delivery schedule of only two weeks from the day the edit was locked.” Using the LED wall allowed the team 4 weeks to create the environment before the shoot, but they could then rely on more than 60% of the shots to be final pixels captured in-camera. The VP approach naturally results in less tracking, keying, and rotoscoping, making the shortened timeline manageable.
The director Leigh Marling and the production company Animals chose Chris Mably as the DP. Chris already had a shorthand created from his prior work with the PXO team, which ultimately helped him to blend the lighting, camera movement, and the overall performances with the content on the LED wall.
The content itself started out as a matte painting high above the clouds complete with Philly Cheese mountains and bagel planets. While the PXO team was working out the cloud formations and their animations, golden sunset horizon, and the overall composition, alter ego was physically sculpting and shaping huge amounts of Philly cheese into mountains and smearing bagels to take and use via high-resolution photos. These photos were incorporated into the matte painting for client approval.
Bringing in an experienced food stylist and treating the “real food” items as a product stills shoot gave the agency and client an opportunity to sculpt them exactly the way they wanted. “It was much faster than having to create them inside of a CG program. I think it took the time frame of creating these assets from 4 days to 1. A huge time saver,” comments Whiteson.
Once the matte painting was client approved the PXO team then had the task of breaking it apart into layers and rebuilding it inside of UE4 so that that the creatives could have realistic parallax when moving the camera. “Of course, you could only move the camera so far before you would see empty spaces in the background, so we had to fill the negative holes with more clouds. The intent was never to move far enough that the clouds would need to be created volumetrically,” Whiteson explains.
Slow dolly moves in or out and small crane moves were agreed upon in the storyboarding process, which was then followed on set. “Early on we knew that we could save some time by not needing to fully blend the floor into the LED wall because alter ego was going to create a cloud pass that would cover that up,” he adds. “We did have some in camera smoke wafting through the frame to help post but in the end, that smoke elements moved too fast and did not resemble the thickness of real clouds.” The key feature to the LED wall was that the team could instantly run black or blue as a source on the wall giving the effects team the perfect key for the bright white smoke element shoot. There was no need for the grip department to assemble giant 24 x 24 frames of black cloth or drape a giant fabric on the ground. When needed, within minutes the entire studio became a black void to shoot the smoke passes.
With the schedule being a two-day shoot and with only one day using the wall, the production company built a swing set of a kitchen in the studio for day one, allowing the DOP to easily jump back and forth to monitor the final stages of the blend on the wall and give real time feedback. “Once the actual LED Wall shoot began and Chris started framing up shots the PXO team were able to easily move mountains or clouds as the director art directed them into place exactly where he wanted them to go,” Whiteson recalled. “That’s something you would never get the opportunity to do on location.” The familiar translation, sizing and depth tools were all able to happen live with individual items adjusted instantly in the environment on set instantly. “That’s a powerful tool,” he concludes.
Phil Jones was the Virtual Production Supervisor at PXO we sat down and asked him a few key questions:
FXGUIDE: It would be great to hear more about real-time updates on set and how you adjusted the environment
PHIL JONES: As in traditional VFX, the notes and change requests are continuous throughout the environment build process. This continues on the day of the shoot when the camera and practical elements in the volume come together to create the final image. To ensure this final image is to the client specs, we are constantly updating and enhancing the assets to ensure we retain the definition of the image, and at the same time, ensure flexibility in the look of the individual assets on the wall. Our asset team works diligently before the shoot to create dynamic, flexible shaders and assets that enable the on-set artists to tweak and mould the environment on the day. On top of this per-environment customization, we have also built a library of generic elements and assets that we can bring into the environment to enhance its depth and life on a shot-by-shot basis.
These adjustments are happening constantly throughout the shoot, applying slight tweaks to composition, subtly changing a shader, and when possible, adjusting the lighting in the environment. When the bake denies us these opportunities, we utilize colour correct regions to, for example, “paint in” some additional highlights or swing the colours slightly to achieve the requested look. Our team has customized the defaults quite extensively, giving the on-set team more accurate controls over the placement, affected area and other fine controls that allow for the very quick changes that are requested just before the camera rolls.
FXGUIDE: I wondered if the smoke in the stage made getting the correct levels on the screens an issue?
PHIL JONES: Practical SFX smoke is always a concern when shooting in the volume, since too much will lift the blacks on the wall, flattening out the overall image. However, this same smoke helps to give more depth and “life” to the practical elements in the volume. Its density needs to be constantly monitored and measured to avoid any issues with the image in camera. For this commercial, the smoke was used to represent the tops of the clouds in “Philly Heaven” and helped to blend the practical cloud “floor” into the cloud environment on the wall. To keep the smoke low to the stage, the SFX team mixed theatrical smoke with liquid nitrogen and used low speed fans to create the slow roiling clouds near the floor. We also shot many cloud elements within the volume for the traditional VFX team for use in the post work required for the commercial.
FXGUIDE: Where the assets created in UE4? Or imported from, Maya?
PHIL JONES: This environment provided us with quite a few challenges, number one being completely immersed within a cloud environment. We have used volumetric clouds created in Unreal for previous shoots, but these clouds were kept quite far in the distance as the voxels quickly became apparent when they got too close to camera. With Philly, we are completely immersed in the clouds which were front and centre in the volume and we required more detail and control of the clouds. To achieve this detail and realism required, we created numerous layers of clouds both from real elements as well as carefully composed DMP elements. Our UE artists then took these elements and created custom shaders that ensured that the clouds were kept alive and dynamic, as well as ensuring we had a great deal of real-time control over the colour, position, detail and movement during the shoot.
Other challenges that presented themselves in the Cream cheese “mountains” and the bagels with cream cheese “planets” up in the sky. The client is understandably concerned that their product is presented appropriately and looking its best. To ensure these elements were up to spec on the wall, we did an early pre-shoot of these elements with a food stylist and took these elements into Unreal as the base for our dynamic assets. Our artists created geometry that allowed us to add more shape to the lighting pre-bake. In addition, they built complex shaders that allowed us to tweak the asset’s colour globally and also in pre-determined areas on each asset. The combination of these techniques allowed us to respond quickly to requested changes during the shoot.
Star Trek on the other VP “AR” stage
As mentioned above the other larger ‘VR’ LED Volume has been in heavy use by the Star Trek creative teams. Below is a Paramount+ view of its use on Star Trek: Discovery.