Realception^®

Fraunhofer IIS has developed algorithms for multi-camera content processing and implemented them in form of plugins for different software suites.

• Realception^® for Foundry´s Nuke provides several scripts and nodes that enable the creation of disparity maps from multi-camera footage. These disparity maps can be used in compositing to simplify grading or more advanced effects like virtual camera movements or synthetic defocus.
• Realception^® for Unreal Engine allows for real-time rendering of light-fields in VR experiences for a seamless composition of CGI and live-action footage.

If you would like to know more about light-field technology, please check the following link:

• Light-field Technology

Realception^® is based on process picture data from light-field or multi-camera array recordings. The outcome is a so called disparity map – a description of the distance of two corresponding points in adjacent images. Having this profound understanding of the captured scene opens all kind of new opportunities manipulating images in a post process like the generation of depth maps.

The optimized algorithms developed by Fraunhofer offer a wide range of editing features such as:

• Generation of depth maps and point-clouds for the creation of impressive VFX
• Selection of virtual camera positions, orientation and focus points
• Usage in depth-based color grading
• Creation of 6DoF live-action content for Virtual and Mixed Reality
• Re-lighting of objects within scenes

The plug-in suite is available for professional pilot users on request.

This video explains the rectification process based on the Realception^® plug-ins. It shows how the vertical and horizontal alignment of the footage can be achieved. That forms the basis for further disparity estimation and computational imaging.

Notice: Starting the video transfers usage data to vimeo.

Realception^® for Unreal Engine enables real-time playback of live-action video light-fields in VR worlds on different head-mounted displays. Its main application is to provide a windowed 6DoF experience with a mix of live-action footage and VR footage.

Realception^® for Unreal Engine imports any light-field created by Realception^® for Foundry´s Nuke. Capturing the footage in front of a green screen with a camera array permits seamless interaction between live-action footage and CGI elements providing correct perspective and stereo depth cues when watching with head-mounted displays.

Thanks to the use of light-field technology, Realception^® for Unreal provides the following benefits:

• Real-time playback of live-action videos without needing to convert the data into meshes
• Maintenance of quality of original footage by pixel-based rendering
• Support of complex scenes with many occlusions
• Interaction between actors and objects across depth layers

Realception^® for Unreal is available for professional pilot users on request.