How to Troubleshoot Live Smooth Streaming Issues? – Part 2 (Publishing Point UI and States)

In Part one, we discussed basic data flow and protocols being used in IIS Live Smooth Streaming. In this blog, we will discuss where all the diagnostic information for live smooth streaming is available and how to make sense out of it.

1. Publishing Point UI

The publishing point UI is the first place that you should look at when troubleshooting live smooth streaming issues. This is especially true with IIS Media Services 4.0 release which has a new detailed view about streams, tracks, fragments, etc. Please refer to my other blog here for this new UI in 4.0 release. In general, we highly recommend our customers to upgrade to IIS Media Services 4.0 release for a number of improvements. The screen capture below was done with IIS Media Services 4.0.

 

image

Publishing Point State and Summary Page

 

image

Publishing Point Detail Page

One piece of important information about publishing point is its state which is available on both 3.0 and 4.0’s UI. Understanding the publishing point state is very important for any live smooth streaming troubleshooting. Below is a diagram describing key publishing point states:

 

image

Publishing Point State Diagram

Note: In the following discussion, the term “encoder” is interchangeable with “upstream server”. They both mean where the publishing point is sourcing live data from.

  • Idle/Shutdown – This is the default state for any publishing point that was just created or whose worker process just restarted. Shutting down a publishing point will also put it into “idle” state. In this state, the publishing point is doing nothing – it’s not receiving incoming streams nor is it responding to any client requests. You can edit any publishing point settings as you wish and they will be picked up when the publishing point next starts.
  • Starting – Once the publishing point is waken up, either by manual start command or automatically by incoming streams if you have the option enabled, it enters the “starting” state. In this state, the publishing point is open for accepting any encoder stream but it’s not ready to serve any client request yet (manifest or fragment). Only after the publishing point successfully received the initial information from all streams would it transit into the next state “started”.
  • Started – This is the state where the publishing point is fully functioning, from both the encoder and the clients’ point of view. It receives live data from the encoder, archive the content (if enabled), generating manifest and fragments to satisfy client’s requests and relaying the streams to any configured downstream publishing points. When the publishing point is in this state, normally you should be able to see constantly updating fragment information in the detailed view on the publishing point UI. However, as I described in this blog, the fact that a particular stream or the whole publishing point is in “started” state does NOT necessarily mean that it’s actively receiving data from the encoders. In Live Smooth Streaming, the state of the publishing point is decoupled from the state of each individual incoming data connections. This design allows full flexibility in redundancy and failover scenarios. The only ways to transit from “started” state to “stopped” state are manual stop or EOS (End Of Stream) signals from the encoders. If the encoders simply dropped the connections without sending EOS or reconnect with new data, the publishing point can stay in “started” state for ever.
  • Stopped – This is a state that often confuses some users especially who have long time experience with traditional live streaming servers. I have a full blog here just to discuss why this state is important. When a publishing point is in “stopped” state, it’s still “half functional”. It no longer receives any data from the encoder but it will still happily serve any manifest/fragment requests from the clients. Basically this is the pure “DVR” mode. To re-iterate what’s stated above, the only two ways to get into “stopped” state are: (1) Manual stop command (either from UI or automated RSCA calls) or (2) Encoder sending EOS for all the streams.
  • Shutdown – This is the same state as “idle” state but I just want to describe a bit more here about how a publishing point can transit from “stopped” state to this state. The most common way is to do it through manual shutdown command. Another option is to automate the shutdown and restart process (only for push publishing point) with the “restartOnEncoderReconnect” option. You can know more about this option from here and here.

There are actually some other states that I didn’t discuss above just to keep the logic simpler. Those are either transitional or error states. For example, there is an “error” state that the publishing point can get into if any unexpected error happened and the operation of the publishing point cannot continue (e.g. wrong archive path set). There are two transitional states “stopping” and “shutting down” that you should rarely see unless for some reason the stop/shutdown operation is taking a very long time.

To be continued……

Published Monday, March 7, 2011 4:46 PM by samzhang

Comments

# re: How to Troubleshoot Live Smooth Streaming Issues? – Part 2 (Publishing Point UI and States)

Tuesday, March 8, 2011 4:17 PM by oldayn

Thank you for your great blog, this posts helps us very much.

Will you cover issues with "auto start" feature: we always run in small problems with cascaded pull publishing points - first client who connects and start publishing point receive 404 error (seems it happen because publishing point is in "starting" and not in "started" mode), and only all other clients after that first receive correct Manifest. However, if this first client use IE it often caches that 404 answer and doesn't show content before force reload (Ctrl-F5). So we must use only manual start, and this behaviour of autostart makes in no so good...

But maybe we do something wrong?

# re: How to Troubleshoot Live Smooth Streaming Issues? – Part 2 (Publishing Point UI and States)

Tuesday, March 8, 2011 8:34 PM by samzhang

Are you using 4.0 release? The server has logic to hold the initial requests to wait for the publishing point to start in this case. But there might be some race conditions that we need to look into. Or you could consider using cascading Push publishing points which can automatically manage publishing point start/stop and shutdown (if configured) among themselves.

# re: How to Troubleshoot Live Smooth Streaming Issues? – Part 2 (Publishing Point UI and States)

Wednesday, March 30, 2011 2:57 PM by samzhang

I've answered your question here: forums.iis.net/.../1976882.aspx

# re: How to Troubleshoot Live Smooth Streaming Issues? – Part 2 (Publishing Point UI and States)

Wednesday, May 4, 2011 1:10 PM by samzhang

You can start from the event log (see part 3). Please use the media forum for support questions.

# re: How to Troubleshoot Live Smooth Streaming Issues? – Part 2 (Publishing Point UI and States)

Monday, March 18, 2013 11:10 PM by soak12

I hope you see this message. I have been trying desperately to shutdown (set to idle) a publishing point when live broadcast is over. I realize I will lose DVR functionality, but I have reason for that. I don't want the live content to be accessible after the live event, but I still do want the archived content. No matter what I try, the state is always STOPPED unless I shut it down on the server. Any other ideas?

Leave a Comment

(required) 
(required) 
(optional)
(required) 
Powered by Community Server (Commercial Edition), by Telligent Systems