IIS Smooth Streaming encoding for Windows Phone 7
We recently announced the release of IIS Smooth Streaming Client 1.1 for Windows Phone 7 (link). Shortly after, Microsoft also announced release of client’s companion Silverlight Media Framework (SMF) 2.2 (link). SMF utilizes the client and offers easier app development for Windows Phone 7 through rich controls. I would not get into this more at this point of time. You can always find more documentation on that online.
Using the tools above you can have an awesome app on an awesome phone :). Here is a quick snapshot of what Silverlight Media Framework (SMF) based default player looks like.
What you want in addition is an awesome experience for the end users streaming and that is more than building a great app. Windows Phone 7 requires you to encode your videos in a special way. In this post I wanted to cover that topic and provide some tips on how you can ensure that the experience is awesome too.
Windows Phone 7 (WP7) Encoding requirements
The article on MSDN titled ‘Supported Media Codecs for Windows Phone’ outlines the various codecs, resolution, profile, etc. that WP7 supports. I would encourage you to look at it if you haven’t already. As this post talks about IIS Smooth Streaming for WP7, I have compiled a short list (cheat sheet) of things you need to care about.
- Supported Codecs
- Device: VC1 / H.264 for video, WMA Pro/HEAAC/AACL for audio. You can refer to link mentioned above for more details on profiles, etc.
- Emulator: As of now only VC1/WMA is supported in the emulator. This means you would need a device to test playback of H.264 content.
- Maximum Resolution (H.264)
- 720 x 480 pixels at 30 fps
- 720 x 576 pixels at 25 fps
- Support for Variable resolution streams – NO.
- Let me explain this further. If you have done Smooth Streaming, you will see that for most content, most bitrates have a different encoded resolution. This is not supported on WP7 today and all bitrates should have the same resolution. I have covered this more in FAQ section below as well.
Recommended Profile
Disclaimer: We strongly encourage you to test this profile in your network and app conditions before you go out an deploy this. We have tested the below and found it works well in most conditions but you can never be too sure.
The reason I start with a disclaimer here is that it is hard to find the best profile that meets everyone’s needs and encoding is a really specialized field. The profile mentioned below should work well in most conditions. Also, the profile mentioned below is if you are targeting WP7 only. You can mix is it with other profiles to find a super profile that works multi screen for you. Super profile is a topic for another post as that is a much longer conversation.
There are quite a few things to take into consideration while designing your profile. Some of these are:
- Be aware of the device’s constraints in terms of bitrate, resolution, etc. You totally don’t want to stream something the device can’t play or can’t play well.
- There will be some users who would stream over WiFi but at the same time there will be some who try to stream over 3G. 3G networks would have higher latencies involved. Having a good range of bitrates helps.
- The device will be in motion, think of a person travelling with the phone in his car and streaming. This does not affect encoding profiles directly but it does have some bearing. Having a good range of bitrates helps here too.
- Another thing to keep in mind is are there other devices you want to target with the same content. If yes, what are the restrictions of those devices.
Video
Keeping the above considerations in mind, here is a profile that we recommend you use:
| Video Codec | Profile @ Level | Bitrate | Width | Height | FPS |
| H.264 | Main @ 3.0 | 1000 | 640 | 352 | Full <= 30 |
| H.264 | Main @ 3.0 | 800 | 640 | 352 | Full <= 30 |
| H.264 | Main @ 3.0 | 600 | 640 | 352 | Full <= 30 |
| H.264 | Main @ 3.0 | 400 | 640 | 352 | Full <= 30 |
| H.264 | Main @ 3.0 | 200 | 640 | 352 | Half <= 15 |
| H.264 | Main @ 3.0 | 100 | 640 | 352 | Half <= 15 |
Audio
You could pick either AAC-LC or HEAAC:
- AAC-LC audio at 64 kbps 48 kHz mono.
- HE-AAC audio at 64 kbps 48 kHz mono.
Frequently Asked Questions (FAQ)
- I already have content encoded with variable resolutions, what do I do to make it work on WP7?
Ans: You can use the RestrictTracks API provided by the IIS Smooth Streaming Client on Windows Phone 7 to restrict the set of video tracks (or QualityLevels) to a set that has the same resolution. In the worst case this would be just a single bitrate. Of course, the other option would be to re-encode. Note: RestrictTracks should be called in the event handler for ManifestReady for it to work as intended. - I am encoding content for multiple screens, are there any recommendations for me?
Ans: There are various ways to solve this. One of them is to use a profile that is a superset of bitrates each screen requires (e.g., Windows Phone and Silverlight on desktop). Once you have this set (a.k.a. super profile) use APIs in the client to restrict playback to the intended subset of bitrates for that device (you can use RestrictTracks on Phone and SelectTracks for desktop). Other options include keeping separate assets for each non overlapping screen. Note: RestrictTracks should be called in the event handler for ManifestReady for it to work as intended. - Can I have a mix of baseline and main profile content in a single stream for H.264 Smooth content?
Ans: No, this is not currently supported. - Does emulator support H.264 content playback?
Ans: No, emulator does not support H.264 playback. You would need a device to test H.264 Smooth Streaming playback. - Does Windows Phone 7 support H.264 Profile Level 3.1?
Ans: Only H.264 profile level 3.0 is guaranteed to work as of today. Detailed codec specific limitations are available here. - Is the suggestion to always use for IIS Smooth Streaming Client for Smooth Streaming content playback on Windows Phone 7? Can I write my own client?
Ans:IIS team strongly discourages writing your own client as the intention of the client is to ease development and provide consistent support across server versions. To ease development you can also use Silverlight Media Framework (SMF) 2.2 or higher. SMF internally uses IIS Smooth Streaming Client but facilitates faster app development with ready-made controls.