Fraunhofer Institute for Open Communication SystemsTo support advanced user interfaces and assisting systems, Fraunhofer FOKUS' researchers are developing real-time-capable algorithms for visualization, tracking, data fusion and interaction. Besides allowing the output of razor-sharp, brilliant, high-resolution images, those also supply the necessary intelligence for the analysis and fusion of data from imaging sensors. Our technologies are used in projection systems in more than 100 installations worldwide - flight simulators, planetariums, surround cinema and theme parks, for calibration and playout. Our medical-related research focuses on enhancing medical visualization with additional information, on positioning and motion tracking in minimally-invasive surgery assistance systems, on developing medical simulators needed for the development of the surgical devices of the future, and on the creation of the new standards for the real-time capable operation room network
Immersive Visualization
Tabbed contents
Projector Auto-Alignment
Projector Auto-Alignment for arbitrary shaped screens
A multi-projector system is often necessary when projecting content on curved screens, such as panoramic or dome projections. The challenge lies in calibrating the projectors with reasonable expenditures of time and effort in such a way that a homogeneous image is created.
Fraunhofer FOKUS has developed an automatic camera-based projector calibration procedure that stands out with its great flexibility and speed. Test patterns are projected onto the screen and automatically captured by digital cameras.
Using imaging processing algorithms, the image parts sent by each projector are then distorted and adapted to the screen in such a way that a high-resolution, seamless image emerges. The projectors' partial images are accurately adapted to one another in the areas of overlap. Automatic projector re-calibration can easily and quickly be run whenever the position or the parameters of one or more projectors change. The projection screens can be of any shape: domes, spheres, cylinders, and curved screens are just some examples.
The system thus covers all necessary functions for an automatic multi-projector calibration: geometry correction (warping), automatic softedge blending, brightness correction and projector auto-alignment. Fraunhofer FOKUS holds several patents on camera-based projector autocalibration.
Privacy warning
With the click on the play button an external video from www.youtube.com is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link: https://policies.google.com/privacySurround Vision – Perfectly synchronized projections on surfaces of any shape
Fraunhofer FOKUS is licensing the software (warping, blending, and projector auto-calibration) and integrates it in existing and new systems, such as media servers, warping boxes and projectors.
In the last ten years, more than 130 multi-projector installations with more than 600 projectors in 13 different countries worldwide have been created using the Fraunhofer FOKUS software for projector autoalignment, warping and edgeblending. Most of the installations are planetarium domes, amusement parks, flight simulators and 3D dome cinemas.
In 2013, Fraunhofer FOKUS disclosed the details of their automatic projector calibration technology in the book “Fullspace-Projektion. Mit dem 360°Lab zum Holodeck” that was published by Springer Publishing.
Warping and Blending
Warping and blending for any application
In cooperation with the producer of graphic processors NVIDIA, researchers from Fraunhofer FOKUS have developed a process for desktop warping. Laying at the graphic card driver level, it enables any application to benefit from warping and blending. The process is already successfully in use in multimedia-control rooms.
Media server for multi-projector systems
Media server for multi-projector systems
Without specialized equipment, large-format or wide-angle projections can only be realized by distributing the image across several projectors. The media server that Fraunhofer FOKUS has developed in close cooperation with Carl Zeiss facilitates auto-calibrating projections onto randomly shaped surfaces and consists of the show player, screen configurator, and show manager.
The player displays the media (e.g. films, videos, sounds, and texts) and provides partial images to each projector. No specialized hardware is needed because image processing takes place on ordinary PCs.
The screen configurator is the planning tool. With it, the projection geometry is defined in advance and an appropriate projector setting considering the respective projector types and determining projector number, position, and orientation is determined. It is responsible for auto-calibration and calculates the distortion rules needed by the player to compose a seamless overall image regarding geometry, brightness, and color from the projectors’ partial images.
Different media types are combined to shows and replayed in real-time with a resolution of 8K x 8K. Similar to a video editing program, various media types such as film and still images can be combined and external devices can be integrated. Gamepads or joysticks can be connected to the system to provide an immersive gaming experience. Additionally, device-free control of multi-media content is possible through gesture commands. Live input can easily be integrated through the combination of different media in real-time (Realtime Compositing).
The technology is suitable for active and passive stereo projection.
The software can be licensed in two ways: the software can either control multi-projector systems directly as a stand-alone model. Alternatively, individual segments of the system can be integrated into existing visualization systems. Fraunhofer FOKUS is available to license holders as a competent partner in the further development of the system. Carl Zeiss is using the system as the basis for its PowerDome system (digital planetarium).
XR-Technologies
Extended-Reality (XR)-Technologies
Today, computer-aided extensions are increasingly affecting our perception of the environment. Augmented reality (AR) and mixed reality applications enable the integration of additional information into a picture of the environment. They are used, for example, in Apps for navigation or for visiting historical sites. The images generated in this way can take the user back in time and provide him with additional information. Augmented reality technologies are also used to research inaccessible locations or production processes in the factory.
On the other hand, virtual reality (VR) applications create entirely virtually generated environments into which the viewer can delve with the help of special glasses. For example, they enable the representation and exploration of complex structures or objects in 3D. The user can view such objects from all sides and move freely in virtual environments.
Depending on the application, the applications run on various smartphones, tablets, AR glasses (e.g. Microsoft HoloLens, Meta2) or VR glasses (e.g. Oculus Rift, HTC Vive).
By means of projections, AR can even be implemented without any additional equipment on the part of the user. The technology has already been used in various projects:
- ARMARE - Augmented Reality Telepresence System for the Deep Sea
- Visualization of complex graph structures using VR technologies
The scientists of Fraunhofer FOKUS are working on technologies for augmented, mixed and virtual reality applications. These are summarized under the term Extended Reality (XR). They advise their customers on current technologies for XR applications and on suitable devices for customer-specific use. In addition, they develop and implement prototypes as well as Apps and programs for the control of VR, AR and MR technologies. Their services cover the entire value-added process.