Media Streaming

Fraunhofer FOKUS FAME is dedicated to addressing these trends and challenges through targeted research and development. Our expertise spans the critical domains of media streaming, enabling us to offer innovative solutions and contribute to a more efficient, engaging, and sustainable media future. Our holistic approach ensures that advancements in one area often complement and enhance capabilities in others, leading to comprehensive and impactful solutions for our partners.

• FAMIUM DASH (Dynamic Adaptive Streaming over HTTP): Fraunhofer FOKUS FAME provides FAMIUM DASH, a comprehensive, fully standard-compliant end-to-end solution for MPEG-DASH and HLS streaming. This solution supports cutting-edge features for both content creation and playback, facilitating faster start-up times and delivery in the highest possible quality for any given network condition. The active contribution of Fraunhofer FOKUS to standardization bodies like the DASH Industry Forum (DASH-IF) (now part of the Streaming Video Technology Alliance, SVTA) ensures that our solutions are not only state-of-the-art but also promote interoperability and drive the industry forward. Fraunhofer FOKUS also co-chairs the SVTA Players & Playback Working Group.
• FAMIUM Player: Based on the open-source dash.js player, to which Fraunhofer FOKUS is an active contributor and maintains for the DASH Industry Forum (now SVTA), the FAMIUM Player leverages HTML5 Media Source Extensions (MSE) and Encrypted Media Extensions (EME) for interoperable premium video consumption in browsers without proprietary plug-ins. Key features include adaptive 360° video streaming, dynamic ad insertion capabilities, multi-screen media synchronization, and metrics/analytics reporting using FAMIUM SAND. Recent versions of dash.js (v5 and later) include improved manifest parsing times, support for AdaptationSet Switching, native handling of DRM key status changes, improved ABR and throughput calculation logic, CMCD signaling via MPD and CMCD v2 keys, flexible URL parameters for Common Access Tokens (CAT), integration of the SVTA Common Media Library for CMCD, CMSD and ID3 parsing, and support for forced-subtitle tracks. FAME offers services including consulting, testing and validation (utilizing tools like the ABR Testbed and Device Playback Capability Test Suite, part of the FAMIUM Streaming Media Test Suite), streaming analytics integration with FAMIUM SAND, player customization, and training workshops.
• FAMIUM SAND (Server and Network Assisted DASH): Specified in ISO/IEC 23009-5, MPEG-SAND enables standardized communication between servers, clients, and network elements in MPEG-DASH streaming. Fraunhofer FOKUS has developed the “SANDLibrary” (JavaScript-based, integrates with players like dash.js, Shaka, hls.js, ExoPlayer, and supports iOS, tvOS, HbbTV) for SAND metric reporting and a Node.js-based “SANDServer” for storing metrics and managing shared resource allocation, deployable on platforms like Google Cloud Platform (GCP). This technology is crucial for content providers to monitor streaming performance (e.g., throughput, buffer levels, QoE metrics) and for network components to manage bandwidth allocation efficiently, especially in shared network environments. FAMIUM SAND supports standards like CMCD (CTA-5004) and CMSD (CTA-5006) and enables AI-driven streaming analytics for viewership prediction, data analysis, and content steering. It facilitates real-time analytics, debugging, alerting, and troubleshooting for VoD/Live sessions, including ad-insertion reliability. FAMIUM SAND provides detailed playback test reports within the FAMIUM Media Streaming Test Suite.
• Low Latency Streaming & FAMIUM MoQ (Media over QUIC): FAME is actively researching and developing solutions for low latency streaming, crucial for live events and interactive applications. This includes investigations into the use of modern transport protocols such as HTTP/2, HTTP/3, and QUIC, as well as the emerging Media over QUIC (MoQ) specifications. Our FAMIUM MoQ solution leverages WebTransport, HTTP/3, and QUIC to minimize connection overhead, avoid head-of-line blocking, and ensure reliable retransmission, making ultra-low-latency media streaming more achievable. Core features include faster startup times, server push capabilities, improved throughput estimation for ABR algorithms, and integration with Metaverse and cloud gaming/remote rendering platforms (Unreal Engine, Unity). FAMIUM MoQ is aligned with the IETF MoQ draft and supports MSE & EME clients. Fraunhofer FOKUS offers an MoQ Testbed for performance measurement and validation, a demonstrator for practical low-latency scenarios, and ongoing R&D in this area. End-to-end latencies below 40ms have been demonstrated in cloud gaming scenarios using this technology.
• Quality of Experience (QoE): Our research in QoE focuses on understanding and optimizing the end-user's perceived quality. This involves developing and evaluating Adaptive Bitrate (ABR) algorithms, studying the impact of network conditions and player behavior on QoE, and conducting subjective and objective assessments. Our research contributes significantly to this area, with publications on ABR algorithms and QoE in various streaming scenarios.

FAME is pioneering technologies that transform passive viewing into active, engaging experiences, pushing the boundaries of storytelling and user interaction.

• FAMIUM Metaverse Framework: Addressing the growing interest in Metaverse experiences, the FAMIUM Metaverse Framework provides a robust solution for rendering and streaming compute-intensive 3D applications to a wide range of devices, including those with limited graphical processing power like TVs. This is achieved by offloading the 3D graphics processing to remote servers (on-premise, 5G edge, or cloud) and streaming the rendered views. The framework supports popular 3D engines like Unity, Unreal Engine, and NVIDIA Omniverse, packaged as Docker containers for flexible deployment. A critical component is the 1ClickMetaverse Testbed, which allows for comprehensive testing and evaluation under various network conditions (Ethernet, WiFi, LTE, 5G via the 5G Media Testbed), communication protocols (WebRTC, WebTransport), and video encodings, with performance metrics collected via FAMIUM SAND. This future-oriented research ensures FAME is prepared for the next wave of immersive digital experiences.

Leveraging Artificial Intelligence and Machine Learning, FAME develops intelligent solutions to optimize media workflows, personalize content delivery, and create new revenue opportunities.

• Dynamic Ad Insertion (DAI): The FAMIUM DAI Service is an end-to-end solution for dynamic ad insertion, supporting both Server-Side Ad Insertion (SSAI) and Client-Side Ad Insertion (CSAI) for MPEG-DASH and HLS streams. Built on a flexible microservices architecture, it includes an Aggregation Service (AGS) for managing media feeds and ad assets, and a Manifest Stitcher for on-the-fly, per-client manifest creation. It supports industry standards like VAST, VPAID, VMAP, and SCTE35 for ad signaling and reporting, and covers MPEG-DASH ad-insertion techniques like XLink and DASH inline/inband events. Management is facilitated through the FOKUS Stream Scheduler, and the service targets platforms including HTML5 browsers, HbbTV devices, Android, iOS, FireTV, and Chromecast. A key feature is its VoD-to-Live workflow, enabling the monetization of large VoD libraries by creating personalized, ad-enabled linear channels. Fraunhofer FOKUS collaborates with Nowtilus, whose “serverside.ai” product licenses and implements FAME's SSAI technology stack, and integrates with VAST-compliant ad servers like Google Ad Manager, SpotX, and Adition.  
• Machine Learning in Media: FAME broadly applies machine learning for various aspects of media processing, including the personalization of services and enhancing interaction across multiple end devices. This commitment to AI ensures our solutions are intelligent, adaptive, and efficient.  

Protecting valuable media assets and ensuring the authenticity of content are critical in today's digital environment.

• FAMIUM DRM (Digital Rights Management) / FAMIUM Multi-DRM: FAMIUM Multi-DRM offers robust, interoperable content protection for internet-delivered video using DASH and HLS. It utilizes the Common Encryption (CENC) standard to support multiple DRM systems, including Microsoft PlayReady®, Google Widevine®, and Apple FairPlay®, with key exchange facilitated by the DASH-IF CPIX specification. For browser-based playback, FAME provides solutions implementing the W3C Encrypted Media Extensions (EME) specification, including the FAMIUM Content Decryption Module (CDM) & Content Decryption Module interface (CDMi). A key component is the Open Content Decryption Module (OCDM), an open-source project maintained by Fraunhofer FOKUS that promotes DRM interoperability and has been integrated with browsers like Chromium, Opera, and Microsoft PlayReady. This approach enables plugin-free HTML5 playback of protected content.  
• Content Provenance and Authenticity: In an era of increasing misinformation and deepfakes, establishing trust in media content is vital. FAME is developing technologies for authenticating digital content and physical products, proving their provenance via trust chains. These technologies aim to combat counterfeiting, fraud, and fake news, thereby protecting brand integrity and ensuring regulatory compliance.

Recognizing the environmental impact of digital technologies, FAME is committed to research that promotes energy-efficient media streaming.

• Research Focus: The “Green Streaming” project investigates and develops solutions for a more energy-efficient video streaming value chain, covering A/V encoders, packaging, content delivery networks (CDNs), A/V players, and end devices. This involves defining measurement points and parameters to collect data for optimizing energy consumption, often employing machine learning and AI models. The development of “Green Digital Twins” aims to model and optimize media workflows for sustainability. Fraunhofer FOKUS FAME has also contributed to “Guidelines on evaluating the energy consumption of video streaming”.  
• FAMIUM GreenView: A key outcome of this research is FAMIUM GreenView, an AI-driven, client-side solution for energy-efficient streaming on SmartTVs (HbbTV, CTV) and other devices. This technology dynamically adjusts the presentation of streaming content for energy-saving playback – potentially saving up to 20 percent energy on streaming devices – without altering the original content or reducing perceived image quality. It uses an AI model trained with data from FAME's Green Streaming Measurement Framework and FAMIUM Stream Analytics, considering context parameters like content, device, and display type. FAMIUM GreenView is available for Android-based SmartTVs (via Exoplayer) and HbbTV, and its benefits can extend to linear broadcast TV.  

This comprehensive research agenda, supported by practical application and a forward-looking perspective, allows Fraunhofer FOKUS FAME to make significant contributions to the evolution of media streaming, addressing both current industry needs and future societal challenges.

The FAMIUM suite represents Fraunhofer FOKUS FAME's ecosystem for future web technologies and the streaming media ecosystem. It is an end-to-end prototype implementation framework designed for early technology evaluation, interoperability testing, and the development of market-ready solutions. FAMIUM bundles a wide range of functionalities related to adaptive bitrate content delivery, Digital Rights Management (DRM), interactive media, and multi-device experiences.   

Key FAMIUM components central to our media streaming research include:

• FAMIUM DASH: For adaptive streaming content creation and delivery using MPEG-DASH and HLS.   
• FAMIUM SAND: For streaming analytics, Quality of Experience (QoE) monitoring, and content steering.   
• FAMIUM DRM: For robust, interoperable content protection using CENC, EME, and supporting multiple DRM systems.   
• FAMIUM DAI: For dynamic ad insertion in live and VoD content across various platforms.   
• FAMIUM Metaverse Framework: For rendering and streaming 3D and Metaverse applications to a wide range of devices.   
• FAMIUM GreenView: For energy-efficient and sustainable streaming on SmartTVs and other devices.   
• FAMIUM MoQ: For next-generation, ultra-low-latency streaming using Media over QUIC.   
• FAMIUM Streaming Media Test Suite: For comprehensive validation and testing of media content, players (DASH/HLS), and streaming technologies
• FAMIUM Provenance and Authenticity: For authenticating digital content and physical products, and proving their provenance via trust chains.   
• FAMIUM 5G Media Testbed: A laboratory environment for testing and evaluating 5G media applications and services, including those utilizing the FAMIUM Metaverse Framework.

These solutions are developed and advanced within Fraunhofer FOKUS FAME, which combines the latest expert knowledge with experience gained from numerous projects to build prototypes, demonstrators, and proof-of-concept solutions aligned with the latest standards. The integrated nature of the FAMIUM ecosystem allows us to address complex media streaming challenges holistically, offering comprehensive solutions that span the entire media lifecycle.

Fraunhofer FOKUS FAME is committed to sharing its research findings and technological advancements with the broader community.

• For a comprehensive list of our scientific papers, white papers, and studies, please visit the general FAME Publications page.   
• FAME Video Dev Blog: Stay updated on our latest research and prototyping activities in areas like Video Streaming Standards, Dynamic Ad Insertion, and Video Encoding by visiting our blog: 
• FOKUS Media Web Symposium (MWS): This annual international conference, hosted by Fraunhofer FOKUS, is a premier event for industry leaders and media tech enthusiasts to explore advancements in Internet-based media delivery. Topics regularly include streaming innovations, AI in media, sustainable media technologies, and advanced advertising. It serves as a vital platform for knowledge exchange and networking.   
• FOKUS Media Streaming Course: Enhance your understanding of media streaming with the “Media Streaming, Encoding and Video Player” course offered by the FOKUS Akademie.
• Subscribe to the FAME Newsletter to stay informed about our upcoming events, news, and publications.