The Importance of the File Format for Adaptive Streaming

When it comes to adaptive bitrate streaming over HTTP, file format really does matter. The fragmented MP4 (fMP4) file format, first used by Microsoft, is the basis of IIS Smooth Streaming, Adobe HTTP Dynamic Streaming, and two industry streaming standards. fMP4 files have some distinct advantages over adaptive streaming solutions based on MPEG-2 Transport Streams, such as the proprietary HTTP Live Streaming (HLS) format developed by Apple.

   Microsoft and Adobe

HLS is based on a derivative of the MPEG-2 Transport Stream (M2TS), which was developed nearly two decades ago for broadcast TV. On the other hand, the fMP4 file format was specifically designed to address the needs of modern streaming to computers, televisions, and mobile devices. fMP4 offers a number of key benefits over M2TS solutions, such as:

  • Trickplay capabilities (e.g., fast-forward, pause, instant replay)
  • Seamless stream adaptation to local conditions
  • Reduced storage requirements
  • Backwards compatibility with M2TS-based solutions
  • Integrated digital rights management (DRM)

When Microsoft first developed IIS Smooth Streaming, we built the fMP4 format on Part 12 of the MPEG-4 standard, which defines the ISO Base Media File Format. We released the specification for the fMP4 format under the name "Protected Interoperable File Format" (PIFF). In addition to IIS Smooth Streaming, this format has become the basis for streaming for Netflix, Comcast XFINITY, and two industry standards: the UltraViolet digital-rights management and delivery standard, which based their Common File Format on PIFF; and the upcoming MPEG-DASH (Dynamic Adaptive Streaming over HTTP) standard.

 Netflix, XFINITY, UltraViolet, MPEG

Tim Siglin, a well-known digital media consultant and writer with Transitions, Inc., has just released an excellent white paper that explores the fMP4 file format and what it means for the future of streaming video. A unique aspect of this paper is that teams from Microsoft and Adobe worked together with Tim on it, a first-time collaboration that provides hope for a standardized media streaming format in the near future. To get the Adobe perspective on this, please refer to Kevin Towes' blog post.

To learn more about fMP4 and read the white paper, titled "Unifying Global Video Strategies - MP4 File Fragmentation for Broadcast, Mobile and Web Delivery," please see Tim's blog post.

10 Comments

  • I completely agree with you. But it seems that the HLS will be the widely adopted one in the future .
    There are several actions that adobe and Microsoft can do if they really believe in the fragmented MP4 solution but it needs to be done now.
    1.Leave the ago aside and Merge the two adaptive formats to a one standard that will be delivered via FMS WOWZA and IIS
    2.develop a player framework that support The fragmented MP4 format for IOS and android so that IOS and android developers can easily offer their customers a better video experience .
    3.Microsoft and Adobe P.R would really need to work hard to fix the Silverlight/flash is going to die talks.
    4.collaborate with the major browsers vendors to support fragmented MP4 adaptive video on them.

    As it seems for now none of the above will happen so. We will be stuck (at least for the near future) with HLS.
    So be real writing articles in the benefits of the format won't do the work. you guys need to be PRO active on it.

  • "Leave the ago aside and Merge the two adaptive formats to a one standard that will be delivered via FMS WOWZA and IIS"

    That's already done. It's an ISO standard called MPEG-DASH.

    "collaborate with the major browsers vendors to support fragmented MP4 adaptive video on them."

    We hope that MPEG-DASH will be the adaptive streaming standard adopted by browser manufacturers as part of their HTML5 Video implementation, but neither Microsoft nor Adobe are in a position to mandate that beyond our own products. As you're probably aware, W3C has not defined any protocols as part of the HTML5 specification either. That said, DASH has been a collaborative industry effort and has a lot of support among studios, content providers, software companies and consumer electronics manufacturers - so it's really our best bet for having a universal, functional adaptive streaming standard.

    Alex Zambelli
    Microsoft Corp.

  • I am starting-up a video streaming business, right now based on IIS and Silverlight. I think the VC-1 codec in SmoothStreaming fragments is the best video streaming protocol right now. Unfortunately it looks like there will never be an open version of Silverlight running on Linux, iOS, Android devices. Should I switch to HTML5?

    It's starting to look like the only solution is to encode videos in multiple formats, MP4, WebM... far from ideal.


    Thanks!

  • It would be nice if I could play my fragmented mp4 files locally in Windows media player, or any other media player. Does anyone know if this is something that is currently available or will be available soon?

  • Alex,

    Smooth Streaming is a streaming technology; it's not dependent on Silverlight, though we do provide players based on Silverlight. Because Smooth Streaming just uses a combination of HTTP requests, XML parsing, MP4 parsing and common video codecs (H.264, VC-1, AAC, WMA) it is possible to build a Smooth Streaming player on any platform which provides those decoders and an HTTP stack. We provide a Silverlight player which works on PC and Mac, as well as Linux (using Moonlight), and a native Smooth Streaming Porting Kit for implementing C/C++ based players for other platforms.

    HTML5 isn't really applicable here as it is largely a UI markup standard with limited video capabilities.

    Alex

  • Michael,

    If you install Expression Encoder (even the free version) that will give you the ability to play ISMV files in any DirectShow-compatible player.

    If any open-source developers are interested in building a more universal PIFF (or UltraViolet CFF) source filter for DirectShow or Media Foundation, I'd be more than happy to provide them with technical advice.

    Alex

  • Alex,
    I see a lot of talk about the Smooth Streaming Porting Kit... is sit available to download? I cannot find it anywhere. We have some uses for it.

    Thanks,
    Adam Schildmeyer

  • Adam,
    If your business has a need for licensing the SSPK, please send us an e-mail to sspk at microsoft dot com with information about your project and company. It's a source code product which means we license it directly to customers.
    Alex

  • Alex,

    Is there anything you can share about plans on when IE will support MPEG-DASH / Smooth Streaming? Given that Windows 8 and Metro will not support plugins like Silverlight, we are wondering Microsoft's recommendation for playback of smooth streaming content on these devices.

    Thanks
    Dharmesh

  • Alex,

    Is there anything you can share about plans on when IE will support MPEG-DASH / Smooth Streaming? Given that Windows 8 and Metro will not support plugins like Silverlight, we are wondering Microsoft's recommendation for playback of smooth streaming content on these devices.

    Thanks
    Dharmesh

Comments have been disabled for this content.