Live events are an integral part of our society. From classical concerts to festivals, people want to gather impressions with their own senses. Not only the current COVID-19 pandemic but also ecological and practical life factors, such as traveling for too long or the lack of childcare in the evening, mean that many people are not able to physically attend events. This is precisely where technical solutions come in, which make an audience experience possible through audiovisual immersion that comes much closer to “being there live” than previously common formats.

The rapid development of digital media formats and offerings in combination with increasingly powerful playback devices such as smartphones or tablets offers a wide range of opportunities for digitally supplementing traditional performance practices, for example at concerts, plays, or other event formats such as congresses or trade fairs. In addition, digitization also offers extensive opportunities for developing new, independent formats and addressing new target groups.

The complexity of new technologies and the often associated high implementation costs for their online platforms make it difficult for cultural professionals to offer their services at the current state of the art. The three institutes Fraunhofer FOKUS, Fraunhofer HHI and Fraunhofer IIS have launched the project Virtual LiVe. In this context, Virtual LiVe aims to develop a platform that offers applicable and scalable high-end solutions for the event streaming needs of different industries in the form of a modular system. From this modular system, SMEs can put together a customized program for their respective event needs and thus obtain a technology-based solution for the realization of their formats. Existing offerings should thus be significantly surpassed in terms of quality, accessibility (payment barriers, hardware integration, the prerequisite of high Internet speed, etc.), and legal aspects (especially concerning the topics of data sovereignty and EU data protection directives).

In the project, Fraunhofer FOKUS's Visual Computing business unit is working on the immersive visualization of the novel media formats in omnidirectional projections, using VR/AR glasses or on mobile devices. In this way, concerts can be broadcast as a 360° live stream in planetariums. Viewers not present on-site can interactively control and move 360° cameras on camera cranes. The spectators themselves can select the viewing angle. 

Viewers equipped with VR glasses can "virtually" immerse themselves in the middle of the action during a live concert and thus, for example, take part in the show in the middle of the stage. Through a 3D 360° camera on stage, they receive the comprehensive view streamed into the VR glasses in real-time at home. They have the feeling of actually being on stage. Through a webcam or the sensors of the VR glasses and a VR controller, the movements and gestures from the spectator at home can be transmitted to the stage so that the spectator is visualized on a monitor on stage (via live video or 3D avatar). In this way, he can dance or play along (e.g. virtual drums, (air) guitar, etc.) and interact directly with the musicians (e.g. headbanging, high-fiving, etc.).

Virtual LiVe is hosted at the 3IT Innovation Center and funded under the Fraunhofer-Gesellschaft's “KMU-akut” program “Forschung für den Mittelstand (Research for SMEs)”.