So you want to try real-time chroma keying? 1

So you want to try real-time chroma keying?

Spread the love


We got our hands on a Blackmagic Design Ultimatte 12, and tested it out.

Blackmagic Design Ultimatte 12 is a hardware real-time compositing processor, capable of handling up to Ultra HD resolution signals. We’re going to take a look at what it entails to set up the hardware and use it to get a good quality chroma key. For this purpose, we’ve created a simple ‘set’ in our greenscreen studio and selected certain objects to test the limits of what’s possible.

The hardware setup

So you want to try real-time chroma keying? 2
Smart Remote 4 at the top, Ultimatte 12 at the bottom.

The main unit is a 1U rack mount enclosure which does all the processing. It doesn’t have any buttons or controls, though. That’s why you also need the Ultimatte Smart Remote 4. This is essentially a small computer with a touch screen and physical knobs that allows you to control one or more Ultimatte 12 chroma keyers. Having the physical knobs makes adjusting settings quick and intuitive.

Here’s the complete list of hardware used for our test:

  • Blackmagic Design Ultimatte 12
  • Blackmagic Design Smart Remote 4
  • Blackmagic Design URSA Mini Pro 12K
  • Blackmagic Design Video Assist 5″ 12G HDR

The URSA camera was used to provide a live SDI signal of a view of the greenscreen studio. The Video Assist was used to playback a pre-recorded second input to be used as a background.

So you want to try real-time chroma keying? 3
The back of the Ultimatte 12 showing all inputs and outputs.

Connecting the signals to the Ultimatte is pretty straightforward. It does have many inputs and outputs for advanced usage, but in our case we only need three: the FG (foreground) and BG (background) inputs, and the Monitor SDI Out to see the result.

Step 1 – Sampling the greenscreen

When you turn on the Ultimatte, it will automatically detect the green color in your input image and automatically create a key straight away. If you’re lucky, this might be good enough for certain cases, but in our case we’re going to try and manually create the best key we can.

So the first step is to select the color of our wall. Pressing the ‘Wall Cursor Position’ button enables a small square on the monitor to view the Cursor Position. Remember, the Smart Remote 4 is only a controller to the Ultimatte image processor, so it has no way of showing the image in its own interface, which could have been nice to be able to select the green color.

So you want to try real-time chroma keying? 4
The Smart Remote 4 is controlled through a touch screen.

The process of selecting the wall color is a little bit awkward. It’s very much like an etch-a-sketch. You have two control knobs, one for vertical and one for horizontal movement of the sample box. There is a small delay in the output response, though, which can make you overshoot. The selection box is small and gray, so you might briefly lose sight of it.

So you want to try real-time chroma keying? 5
The process of moving the sample selection box.

It’s best to select a color of the greenscreen close to your foreground object, rather than somewhere on the edge of the screen. As we’ll see in a minute, no greenscreen is completely uniform in color. Make sure to not select any part of the foreground, though, as that will ‘contaminate’ our sample.

The result of this initial color picking looks as follows.

So you want to try real-time chroma keying? 6
All inputs and outputs shown together.

The top left shows our input image, directly from our camera, connected to the FG input of the Ultimatte. The top right is an image we created earlier and recorded onto the Video Assist, which is connected to the BG input of the Ultimatte. In the bottom left corner we see the matte that was created. Anything that’s green should be completely white in this image. Any foreground object should be completely black. As you might be able to see, that’s not entirely the case. And because of this, our composite image on the bottom right isn’t perfect. You can see some gray ‘dirt’ on top of the floor and the wooden desk.

First of all, the greenscreen floor is somewhat dirty from people walking on it with their dirty shoes. Secondly, there is a slight color variation in the floor, compared to the wall. This is inevitable and due to the lighting and the angle of both surfaces towards the camera. And lastly, the lens has vignetting, which makes everything darker towards the edges of the image.

Step 2 – Refining the color selection

The first thing we can address is that difference in color between the wall and the floor. Conveniently, Blackmagic has made it an option to select a second chroma sample to help with this.

So you want to try real-time chroma keying? 7
Select ‘Floor Cursor Position’, position the sample box, then press ‘Sample Floor’.

After sampling the floor, the result improves considerably. Blackmagic doesn’t fully explain how this works, but it seems likely that this increases the amount of different shades of green that are considered as ‘background’.

So you want to try real-time chroma keying? 8
Before and after result of picking an additional floor sample.

Step 3 – Improving the black glossy table

Our small black table is quite glossy and thus reflects quite a lot of the green behind it. This results in the keyer ‘thinking’ this is part of the background, rather than the foreground and consequently makes the top of the table a bit transparent.

So you want to try real-time chroma keying? 9
Table top matte and resulting transparency.

On the left we see the top of the table is a light gray color in the matte. On the right we see the horizontal line in the background visible through the table.

There is a special control called ‘Black Gloss’, which was specifically made for this purpose. It’s a trade-off, however, and should be used with caution. When you increase the value of this parameter, it tries to make the top of the table less transparent.

So you want to try real-time chroma keying? 10
‘Black Gloss’ feature to help with shiny surfaces.

The table was picked to deliberately create an extreme case, and as you can see below, it’s actually too shiny for this feature to work well. The top of the table stays pretty much the same, even when pushed to the extreme. If you go too far with this setting though, you can see the noise in the floor comes back as well.

So you want to try real-time chroma keying? 11
Adjusting the ‘Black Gloss’ setting from 0% to 100%.

It does, however, work quite well for the less shiny Gobo Grip heads and the magazines.

Step 4 – Sampling the whole image

We can take advantage of the fact that our camera is static for this shot. Instead of just sampling one or two colors, we can sample the entire frame, including all its imperfections and effects of the lens such as vignetting. This process is part of Ultimatte’s ‘Screen Correction’ feature. First we have to clear our ‘set’ of any foreground objects.

So you want to try real-time chroma keying? 12
Clearing the frame to be able to sample the entire image.

Once the frame is clear, it’s a simple press on the ‘Screen Capture’ button (you’ll need to have previously clicked on ‘Screen Correct’ to enable this), to store this information in the memory of Ultimatte. The improvement is quite dramatic.

So you want to try real-time chroma keying? 13
Before and after of using the ‘Screen Capture’ feature.

As you can see, all of the imperfections in the floor are gone and the whole image appears brighter at the edges. We might not have even noticed before, but this was the vignetting coming through in the composite.

There are many more controls to fine tune things, such as ‘Matte Density’, ‘Clean Up Level’, ‘Shadow Threshold’, ‘Veil Correction’, etc. But with the steps described above, it’s possible to create a pretty good key.

Step 5 – adjusting the background color

One last thing to improve is the balance between the foreground and background. Our background is a bit too dark and the color temperature is a bit too cool overall. Ultimatte has color correction controls to fix that. Below is the result of changing the grade of the background input.

So you want to try real-time chroma keying? 14
Before and after of adjusting the background color.

There’s one more feature of the Ultimatte that’s worth mentioning, which can best be shown with a different studio setup.

A different subject

For our next setup, we happen to have a background with a strong light coming from the back. To match the direction of that light on our foreground subject, we’re going to have to put our light stand almost behind our subject. This means it will be in frame.

So you want to try real-time chroma keying? 15
A new subject and background.

If we do nothing about it, the light will be considered a foreground object and will be visible in the composite image. Luckily there’s a feature to cut off any unwanted parts of the screen.

So you want to try real-time chroma keying? 16

The Window feature allows you to create a simple garbage matte, cutting off things from the top, bottom, left or right. It also has a softness setting to soften the transition of the cut off line, to better hide it.

Combining this feature and performing the same steps as before, leads to final result below.

So you want to try real-time chroma keying? 17
Steps: original FG, initial key, garbage matte edges, black gloss adjust, sample whole screen.

Conclusion

As we were able to demonstrate, it’s possible to create quite a good key with Ultimatte 12, but the usual caveats of using a greenscreen should always be taken into consideration. You can’t have any foreground elements with green colors in them. Reflective objects should not be too shiny. Keeping your greenscreen clean and the lighting of it nice and even, will all help to create a better key.

Note: Blackmagic Design loaned us the Ultimatte 12 and Smart Remote 4 hardware for this test. You can find more information on the Ultimatte 12 here.

About the author: Marijn Eken is a freelance visual effects supervisor with a background in many aspects of visual effects. Most notably as a digital compositor on feature films such as Interstellar, The Avengers, Looper and The Maze Runner. He has worked on a variety of other types of projects as well for theatre, musea, and others where real-time graphics were involved. He also teaches part-time at the Netherlands Film Academy and is working on introducing virtual production into the curriculum. You can check out his IMDB page here. Or follow him on Twitter here.

Get bonus and early VFX content via a befores & afters Patreon membership





Source link

You may also like...

Leave a Reply

Your email address will not be published.