New Live Smooth Streaming UI Explained

In the new IIS Media Services 4.0 release, the most obvious change for Live Smooth Streaming is probably the new UI pages that show a lot more detailed information about the publishing points. Here let me walk through some interesting pieces of the new UI and show you some examples about how it could help you with server monitoring and troubleshooting.

1. The “Publishing Point Summary” section:

The first thing that you’ll notice is the new “Publishing Point Summary” section that’s located below the publishing point list. When the publishing point is running, it shows high level of information about the publishing point including the number of streams, tracks (with bitrates), options enabled and my personal favorite: the archive path for the current session (so that I can simply click on the link to navigate to the archive location!).

2. The “Details” page

From the main page as shown above, if you click on the “Details” link on the right hand side panel, you will navigate to the publishing point detail page. This page contains three sections: the “Stream Summary” section, the “Track Details” section and the “Connection Details” section. Before we jump into the details about this page, let’s first define some important terms here:

Track: The definition of “track” is the same as used in the MPEG-4 spec: “A collection of related samples (q.v.) in an ISO base media file. For media data, a track corresponds to a sequence of images or sampled audio.” For example if the presentation has a few video bitrates, each one of them is a separate video track. Likewise, the audio part of the presentation has its own track(s).

Stream: For Live Smooth Streaming publishing points, “stream” is defined as a group of “tracks” (as defined above) that are combined in a single Fragmented MP4 formatted bit stream. It’s the same bit stream that’s transmitted over a single TCP connection between the encoder and the server and between servers. So comparing to MPEG-4 file format, the definition of “stream” here is at the same level of the MPEG-4 file container. The grouping of tracks into streams is determined by the encoder. For example, some encoder may choose to output one track per stream so you would have as many streams as the number of tracks. Another encoder may choose to output all tracks in one stream. Or you may have any combinations in between.

Stream State: Each stream can have two states: started or stopped. When the stream is in started state, it is either actively receiving media data or ready to receive media data from new sources. It’s important to understand that a “started” stream does NOT necessarily mean that it’s currently receiving live stream. A stream can stay in started state even if all connections from encoders have dropped. A stream can only be stopped by either an End-Of-Stream (EOS) signal from the encoder (or upstream server), a manual publishing point stop/shutdown command, or some internal errors. Without having any one of those three conditions, the stream will stay in started state. This is useful in the sense that the stream is still open and ready to receive from new backup/failover sources to continue streaming. Now you may ask, how would I be able to tell if the stream is still receiving data? Keep reading. :)

Source: “Source” is where the publishing point is getting data from. Each source provides one or more streams to the publishing point. In most cases, each stream just has one source. But in some scenarios, like active-active failover, you could have multiple sources from different locations sending in the same stream (identified by the Stream Name) to the same publishing point. In this case, the publishing point will do the proper job to consolidate the identical streams into one logical stream. The types of sources that IIS Media Services 4.0 support are: HTTP Push and HTTP Pull.

Destination: “Destination” is where the media data goes to and the type of destination shows how the publishing point is making use of the media data that’s coming from the source. The types of destinations that you can expect to see in IIS Media Services 4.0 are: IIS Smooth Streaming, Apple HTTP Live Streaming, HTTP Push and HTTP Pull. The last two types of HTTP destinations are for server-to-server distribution.

You can also refer to my previous blog here to see how streams, sources and destinations work together.

2.1 “Stream Summary” section

This section lists the same high level information about the streams and tracks as in the “Publishing Point Summary” section of the main page.

2.2 “Track Details” section

This section contains a table listing detailed information for each track. For the definition of each column, you can refer to the online doc here. Here let me show you a few examples about how information in this table can help you in troubleshooting.

Question 1: How do I know the grouping of tracks into streams?

Check the “Stream Name” column for the track-to-stream mapping. “Stream Name” is an unique identifier of the streams. In this example, there are two streams. One is called “2716-stream0” which contains a 48kbps audio track and a 1950kbps video track. The other stream is called “2716-stream1” which just has a 950kbps video track.

Question 2: How do I know if a stream is still receiving data?

You can check the “Incoming Bit Rate” to see if it has dropped to zero. You could also use the “Fragment Timestamp” field to check if the track’s timestamps stopped progressing. The “Connection Details” section can also tell you if all the sources are still connected (explained below).

Question 3: How can I tell if the tracks are in-sync?

Compare the values in the “Fragment Timestamp” column and see if they’re mostly in-sync across all tracks. For audio and video tracks, they should always be very close to each other. For things like command/text/ads tracks, a gap in timestamp is probably fine due to their sparse nature.

Question 4: Certain tracks/bitrates seem to stop working, where do I start?

First check the state of the stream to see if it’s in started or stopped state. If it’s still started, check if the stream is still receiving data (as described above). If the stream is stopped (while the publishing point is still running), it’s either due to EOS or internal errors. Go to the Windows Event log (under “Event Viewer” –> “Windows Logs” –> “Application”) which should have log entries describing the reason for stream stop.

Question 5: Why my failover encoder streams can’t connect to the publishing point?

This question came up a lot especially during the initial integration testing with new encoders or deployments. The scenario is that you are trying to simulate an encoder failover by bringing down one encoder stream and bringing up a new one. Somehow the new streams can not connect to the publishing point. The most common problem has been that the first encoder sent EOS signal(s) to the publishing point before the streams went down, which caused the streams on the publishing point going into “Stopped” state. Failover can only properly happen when the streams are in “Started” state because EOS to the server means that the live presentation is over. This problem is typically caused by improper simulation of the failover (e.g. by clicking some button on the encoder UI rather than, say, just yanking the network cable). How to check if this is the case? Simply take a look at the “Stream State” for all the streams. If all streams has gone to “Stopped” state, the publishing point will automatically go into “Stopped” state as well.

Question 6: How can I tell from the server side which bitrates are popular?

Check the “Request Rate” for each track. This is the rate, in request/sec,  of the successful requests for each track. Note that it might not give you a full picture if there are cache servers between the IIS server and the clients.

2.3 Connection Details

This section shows the source and destination information for all the streams grouped by machine IP address. In this particular example, this publishing point is receiving a source from machine “sam-oob” (this is the local machine) and the source is of type “HTTP Push” pushing in two streams: 2716-stream0 and 2716-stream1. From the destination list, you can tell that these two streams are in turn used for “IIS Smooth Streaming”. Destinations like “IIS Smooth Streaming” and “Apple HTTP Live Streaming” will always have the local machine IP address.

Now let me walk through some more examples to show you how the connection details view looks in some other scenarios:

Example 1: Source disconnected unexpectedly

In this example, the encoder connections were dropped and the encoder didn’t come back to reconnect. Because the encoder didn’t properly send EOS to stop the publishing point, the publishing point and the streams are still in started state. But the “live streaming” part of the processing has essentially stopped (though DVR is still working). You can identify this situation from the UI by the fact that all the sources are gone.

Example 2: Redundant Streams

In this example, there are two encoders pushing redundant streams to the same publishing point. One is from “sam-oob” and the other is from “sam-ms”. The stream names  (“test-stream0” and “test-stream1” in the screen shot)are the same between the two encoders. So for each stream, there are two connections serving as redundant stream to each other.

Example 3: Server-to-server distribution

In this case, there are two publishing points on machine “sam-ms” that are sourcing data from this publishing point. One of them uses HTTP Pull and the other uses HTTP Push. And as usual, this publishing point is also doing IIS Smooth Streaming on local machine (“sam-oob”).

Example 4: Error case of Apple HTTP Live Streaming

This is a case when the publishing point is doing both IIS Smooth Streaming and Apple HTTP Live Streaming. Do you see anything wrong in here?

You can see from the “Destination” section that while stream “test1-stream1” is doing both IIS Smooth Streaming and Apple HTTP Live Streaming which is expected, somehow stream “test1-stream0” is only doing IIS Smooth Streaming but not the Apple one. This is an indication that something has gone wrong with the Apple HTTP Live Streaming for this stream. Then if I switch over to the Windows event log and I would find an error event with the following message:

Track '1' in stream 'test1-stream0' for publishing point '/test.isml' contains unsupported codecs and cannot be converted to the MPEG-2 TS container format. Only Advanced Audio Coding (AAC) audio and Advanced Video Coding (AVC) video codecs are supported.

Mystery solved. :)

3. Summary

As you can probably tell, the reason why we created this new UI is to expose more detailed information about the internals of the publishing point so that it’s no longer a black box. It’s also worth noting that all these information is also exposed through IIS RSCA configuration which can be accessed programmatically. If you already know how IIS configuration and RSCA works, you can take a look at the schema file of IIS Media Services (IISMedia_schema.xml in the IIS schema directory) to get started. Otherwise, I’ll do another blog sometime later to show you how to do with with a sample script.

Thank you.