Keyer is the most essential part of any Green screen Virtual Studio production. Reality Keyer Sections under Reality Keyer describe steps on how to work with various Keying properties. Reality Keyer is an internal keyer feature in Reality Engine which is one of the world’s most advanced and true-life keyers which provides spectacular results for keying of live camera video input to preserve real environment details such as contact shadows, transparent objects, and sub-pixel details like hair. Reality Keyer also works with 4:4:4 RGB camera sources. The keyer also supports 3D masks for “force-fill” and “spill bypass” buffers besides garbage masks. These masks enable hybrid virtual studio operation which lets you combine virtual and real environments.
Reality Keyer Node
Reality Keyer node is used for image-based keying technology. The pins are:
Input: This should be either direct or delayed output from AJAIN which is the SDI Input
CleanPlate: This can either be a color from COLOR node or distorted RENDER output of CYCLORAMA node.
Mask: This can either be a combination of 3D Geometry masks or distorted MASK output of CYCLORAMA node.
Output: This is the keyed image OUTPUT of the REALITY KEYER node.
HardMask: This is the Core Matte view for fine-tuning the keyed image.
HardMaskHorzBlur: When the lighting, cyclorama background color and camera is optimized, Reality Keyer will only require small adjustments. While working on different modes you can use OUTPUT property of the REALITY KEYER node as below:
After setting up the clean plate, here is the suggested workflow for fine-tuning Reality Keyer. These are the main steps in the keying process:
Fine-tuning the Soft and Core Mattes
Creating a key for the spilled areas
Screen subtraction and Spill suppression
Color Correction (optional)
Capturing Cyclorama Projections for Clean Plate
To expect the best result from Reality Keyer which is an Image-Based Keyer, it is important to take a good projection of the cyclorama. It is important to take the wider screenshot as possible like the one below:
To add a projection of the Cyclorama as the one above, please go to the Functions tab of the CYCLORAMA node and click ADD PROJECTION
Once you add this projection, this Clean Plate is added to CAPTURE section of the CYCLORAMA node.
Remember that if the whole of the cyclorama cannot fit in one shot, you should start taking projections from the center and then you can click on ADD PROJECTION while you are taking projections of the sides of the Cyclorama and you will see that these projections add as stitching to the shots.
Reality Keyer Properties
Now going over different Output Types, it is time for fine tuning the Keyer output.
We should first start looking at the result of COMBINED SOFT MATTE. In the final, we want this matte to be soft and provide fine details for the edges of the foreground subject. It is acceptable to have some transparency and/or holes in the middle of the soft matte. Do not try to fill in these holes using soft matte settings, that will be job of the CORE MATTE to fill in these holes.
Choose the OUTPUT TYPE as Combined Soft Matte.
For an ideal green screen, these weights should be at “0.5”, however in some situations we might need to tweak these values.
If you adjust SOFTMATTEREDWEIGHT, you will expect that those pixels will become pinker.
We combine SOFTMATTEGAMA1 and SOFTMATTEGAMA1 which are in opposite directions to work on softness details like shadows. Modifying these values will affect the visibility of the shadows and transparency.
Do not try to compensate all the holes with Soft Matte settings. Try to make edges look good with Soft Matte.
After having a good COMBINED SOFT MATTE, we should look into fine-tuning the hard matte. We can view the hard matte using the CORE MATTE viewing mode. This matte`s job is to fill in the holes of the COMBINED SOFT MATTE. Since this matte will be harder and provides a harsh matte, we will need to “clip”, “erode” and “soften” it, using the hard-matte controls. While tuning the hard-matte controls, we should aim to not to see any holes or transparencies. We should create a rough matte for the foreground subject, and try to not to include any portion of the green screen cyclorama by more than a few pixels in size.
Choose the OUTPUT TYPE as Core Matte.
Ideally, you shouldn`t use any Core Matte at all. The pixels which reflect green light will go softer or more transparent.
Any indirect light coming in will make it transparent on Core Matte so please check if the fabric for Cyclorama you are using is not a reflective material.
The holes in the center of the talent are handled by Core Matte. Make sure that the talent inside the scene has no black points on Core Matte.
Tune it by clipping the “blacks” and “whites” of the hard matte. You can try to remove any wrong pixels in the green screen by increasing the CORE MATTE BLACK POINT value. You can also fill the holes and transparencies in the foreground subject by lowering CORE MATTE WHITE POINT value.
When you need to adjust the hair details, please switch the OUTPUT TYPE to SHRINKED MATTE and modify SOFTNESS and find the optimum value to show the hair detail.
After generating the matte, the keyer needs to suppress for the green colored edges of the subject and the indirect green light reflected off the green screen walls and floor. By looking at the scene you can see the spill and decide how much to manipulate the spill parameters.
Choose the OUTPUT TYPE as Spill Matte.
When SCREEN SUBTRACK EDGE is 1, it means that subtracting all the greens from the edges. Likewise, SCREEN SUBTRACK CORE refers to subtracting the greens from the core. It is another way of suppressing green reflections and edges.
We can manipulate the UNPREMULTGAMA value rather than remodify the Core Matte. This value will also affect the softness of the edges.
Do not overuse EDGE SPILL REPLACE COLOR, you can slightly modify this to match the color of your scene.
Reality Keyer pipeline generates the “Combined Soft Matte” and “Shrinked Core Matte” and then they are combined into the final matte. We can view it using the FINAL MATTE viewing mode.
Do very little adjustment to FINAL MATTE BLACK POINT and FINAL MATTE WHITE POINT to fill the holes if necessary. It has to be no more than 0.1, if you need to adjust to a value more than this, please consider rearranging the lights, checking the camera sensor and its properties or the color properties of your physical cyclorama.
Use FINAL MATTE GAMA to control the final softness of the matte.
Adding a Color Correction Node
As different sets require different color properties of the talents, to correct the color properties of the talents in the set, you might need to do color correction each time you switch to a different set. Rather than using the COLOR CORRECTION section in the REALITY KEYER node, it is a better practice to add a COLOR CORRECTION node after REALITY KEYER node as shown below: which can also be seen on the templates:
And change only these properties when you switch to a different set:
If you are using a broadcast camera; check the CLASS setting. You can find detailed information here.
If you see Black Lines around the talent: You might have a bad sensor on the camera, and also check the sharpness of the camera.
If you see Red Lines around the talent: You might have a bad camera and lens, you can use dilateRGBA node's properties to overcome this.
Saturation caused by dirt on the cyclorama will surely affect keying quality as it is a color-difference keyer. Make sure the area is always clean.