Virtuality in modern workflow
The TV production market today has to provide ever higher quality with increasing flexibility and smaller budgets. Virtuality is a great help here. Modern technologies and workflows bring the design and use of virtual worlds for television production within reach and reach. By using the Unreal Engine from Epic Games, a high degree of photorealism can be achieved despite real-time rendering. The advantages are obvious. Virtuality opens new possibilities, which would not be possible or too expensive in the real world. It is no longer necessary to build elaborate, real stage sets and the size of the real studio is no longer related to the size of the space visible to the audience.
Virtuality can also be a great help in automating the workflow. This saves money and increases the safety, repeatability and quality of production.
A strong partnership for the best result
The Unreal Engine, developed by Epic Games, is basically a rendering and physics engine for video games. But since modern video games have almost the same use case as virtuality in the studio world, namely to render as photorealistic and live as possible, the Unreal Engine is also used as a basis for studio graphics systems. A big advantage is the enormous development power and know-how of Epic Games from meanwhile decades of engine development and the large community that creates content, ideas and designs. The Unreal Engine can be used to simulate realistic rooms as well as entire outdoor worlds with flora and fauna and realistic light and weather effects.
Zero Density provides the interface between the Unreal Engine and the studio. With the product Reality Engine, Zero Density has created a studio graphics system based directly on the source code of the Unreal Engine, which is easy to handle but offers maximum performance and flexibility. It is only with Reality that the Unreal Engine can be used easily, quickly and in a sensible workflow for the studio world. With the Reality Keyer, Zero Density has created a unique and revolutionary keying technology. Unlike the conventional chromakey, this is the first, so-called difference keyer. Another important core technology of the interface is the application of camera tracking data to the graphics to synchronize the perspective of the virtual and real world. Here too, Zero Density performs absolutely essential work by implementing all common protocols and calibrating the camera optics.
With our core competences as system integrator and project house, KST Moschkau is able to advise the customer optimally in his decision and to carry out the project from the basic planning to the turnkey studio as well as the user training completely from one source. In such an overall project, all subdivisions of the Moschkau group of companies work hand in hand to achieve the perfect and individually tailored result for the customer.
The strategic partnership with Zero Density and Epic Games not only enables KST to distribute and conduct certified training throughout Europe. The KST Innovation Center also enables the active further development of the workflow and the product. The Innovation Center is an important development, demo and research environment for all partners.
Fields of application
The classic use case of virtuality is a virtual studio. In this process, a large part of the real background is cut out and replaced by a virtual one. A virtual studio makes it possible to realize complex and fantastic rooms in a relatively small studio. Furthermore, the different environments can be changed quickly and easily. A studio can switch between different magazine situations or similar within seconds. In order to make it easy to cut out the objects in the studio, we usually work with a green area, i.e. a green wall.
Virtual Production / Portal Production
A special application of a Virtual Studio is the so-called Virtual Production. One wall in the studio, in complex cases even all walls, including the ceiling, are filled with LED walls or a large-screen projection. Instead of cutting out the presenter as in a green-screen studio and placing him or her in a virtual background, the virtual world is now placed directly on the screens. It is essential that the perspective of the virtual world matches the camera currently in use. From the camera’s perspective, the walls disappear and they become windows or portals into the virtual world. For the external viewer, this distorted representation may feel confusing. With this production method it is possible to bring the virtual environment into real reflections in the action area and also realize the complete lighting of the set via virtual lights.
Of course, all described applications can be combined in any form to hybrid applications. The individual application areas do not differ at all from the software, workflow and design used. Depending on the situation, it may therefore make sense to combine green area, projection or LED wall.
Three-dimensional virtual design
One of the most important components of modern virtuality is three-dimensionality. All virtual objects and also the environment are three-dimensional. This allows both the camera and the presenter to move freely in this world and around these objects. The three-dimensionality is essential for a correct perspective and general realism. The design side is given simple tools by using the Unreal Engine and there are almost no limits. Thanks to the large community and the Content Marketplace, it is possible to obtain expertise for little money and use already designed objects or environments.
For the perfect key, the green area must be illuminated as homogeneously as possible. A frequently seen misjudgement here is that this requires the use of expensive studio lights that can be adjusted in all possible parameters. By using the Reality Keyer this is explicitly not the case. Adjustable light only needs to be used in the Zero Density Workflow to provide additional lighting or effects for the presenter. The optimal basic illumination for the keyer is a homogeneous daylight, i.e. lamps with a temperature of about 5800k.
Camera tracking is essential to enable camera movement. Tracking in this case is a data stream containing all spatial and rotational coordinates of the camera. Also all data about the current state of the optics are included in this stream. With this data, the rendering machine is able to synchronously adjust the perspective of the virtual camera to that of the real camera at any time. For the illusion of a virtual studio, precise tracking is mandatory.
But tracking is not only used for camera tracking. With Talent-Tracking it is also possible to track the moderator and his movement. This allows the moderator to move naturally in the virtual world and interact with virtual objects.
Keying describes the releasing of the moderator or objects in the Green Area in order to use them in another environment. It is the most important technology in the virtual studio.
With the Realitykeyer, Zero Density has developed the currently most impressive keyer on the market. Unlike the normal chromakey, this difference keyer does not work with a green color palette that is cut out, but with a reference image. Under optimal conditions, this method enables a pixel-perfect key without loss of detail such as, for example, in the case of a key that is not visible on the screen. individual hairs.
Rendering machines and composition
Most of the workload in the graphics workflow is carried by the so-called render engine. This is a high performance workstation that is responsible for one rendered camera image at a time. For performance and renunciation reasons it is absolutely recommended to use one renderigen per rendered camera string.
A particular advantage of the zero density workflow is that both the keying process and the rendering of the graphic take place on the same machine. This real-time composition allows the parts to react with each other, creating realistic interactions between camera image and virtuality. For example, the moderator appears in the purely virtual reflection of the mirroring virtual floor.
The interface to the control room in live operation is also important. Using the extensive software systems reality HUB, action and setup, it is possible to configure the entire system down to the smallest detail from a software interface and to handle, control and fill it with content in live operation using a simple interface prepared for the operator. Connections to automation systems and databases are also possible.