How to do Live Streaming with the Smooth Streaming Format SDK

Posted: Feb 08, 2011  5 comments  

Average Rating

Tags
Media

Share this Post

In a previous blog post I wrote about the sample application that comes with the IIS Smooth Streaming Format SDK. One important feature missing in that sample is code showing how to do Live Smooth Streaming. The next release of the SDK will correct that and include a new sample application that supports live streaming, yet that release is not planned to hit MSDN in the next couple of months, and in the meanwhile users are facing difficulties trying to add the functionality to their encoders. On top of that the Application Workflow documentation for the Smooth Streaming Format SDK is a little vague on live streaming. In lieu of that I decided to write this blog post to elaborate on some of the details. This article assumes familiarity with the SDK’s workflow and HTTP.

Setting up the muxer and streams

 

For live streaming the application must put the SDK’s muxer in live mode by setting the SSFMUX_OPTION_LIVE_MODE to TRUE. This must be done before adding streams to the muxer. Basically:

     BOOL fLiveMode = TRUE;
     
hr = SSFMuxSetOption(
            hMux,
            SSF_MUX_OPTION_LIVE_MODE,
            &fLiveMode,
            sizeof(fLiveMode)
            );
    if( FAILED(hr) )
    {
        wprintf( L"Error: 0x%X trying to set Live Mode\r\n", hr );
        goto done;
    } 

Sending the Streams to IIS

 

The Smooth Streaming Live Streaming Protocol requires that each stream sends its header, fragments and index (i.e., the data returned by SSFMuxGetHeader, SSFMuxProcessOutput and SSFMuxGetIndex) to the IIS Server using a HTTP POST request. The URL for the request is specific to each stream and must follow this pattern:

                http://<server>/<pubpoint>/Streams(<identifier>)

The <identifier> part is an arbitrary name chosen by the application that uniquely identifies that stream within the publishing point. It must not contain invalid URL characters, URL separators or parentheses. It can be as simple as just a stream index. The same <identifier> string must be passed as the
pszSourceFileName field of the SSF_STREAM_INFO struct when adding the stream to the muxer with
SSFMuxAddStream.

The IIS Server is very flexible in terms of how it receives live streams. Fragments can be sent using multiple HTTP POST requests (one per fragment) or a single long running HTTP POST (one per stream). The server automatically filters out-of-order or duplicate fragments to produce the best possible continuous stream. For more details about the server operation, check this article.

Long running HTTP POST requests should be chunk transfer encoded. This type of HTTP encoding allows applications to start a HTTP request without knowing in advance the entire size of the payload (i.e., the “Content-Length”).

If you’re going to use WinHTTP to handle a long running HTTP POST, keep in mind that although WinHTTP can receive data in chunk transfer encoding transparently, when it comes to sending chunked data you have to handle the “chunking” on your own. The first thing to do in this case is to add the “Transfer-Encoding: Chunked” header to the HTTP request (note this precludes the need for a “Content-Length” header).

Here’s a quick code snippet showing how one would go about starting a chunked HTTP POST for a live stream with WinHTTP.


 
    WCHAR szAdditionalHeaders[] = L"Transfer-Encoding: Chunked";

     ...

     fResult = WinHttpSendRequest(
                    hHttpRequest,
                    szAdditionalHeaders,
                    ARRAYSIZE(szAdditionalHeaders)-1,
                    WINHTTP_NO_REQUEST_DATA,
                    0,
                    0,
                    NULL
                    );
    if( !fResult )
    {
        hr = HRESULT_FROM_WIN32( GetLastError() );
        goto done;
    }

 

And here’s an example of a function for posting HTTP chunks (assuming WinHTTP is operating in “synchronous” mode):


 
    HRESULT WriteToSmoothStreamOutput(
        __in HINTERNET hHttpRequest,
        __in_ecount(cbData) LPCVOID pbData,
        __in ULONG cbData )
    {
        HRESULT hr = S_OK;
        BOOL fResult = FALSE;
        DWORD cbWritten;

 
       char szHttpChunkHeaderA[32];
        char szHttpChunkFooterA[] = "\r\n";

 
       //
        // Send the HTTP Chunk Transfer Encoding chunk-start mark
        // Observe the use of UTF-8 strings.
        //

        hr = StringCchPrintfA(
                szHttpChunkHeaderA,
                ARRAYSIZE(szHttpChunkHeaderA),
                "%X\r\n",
                cbData );
        if( FAILED(hr) )
        {
            goto done;
        }

        fResult = WinHttpWriteData(
                        hHttpRequest,
                        szHttpChunkHeaderA,
                        (DWORD)( strlen(szHttpChunkHeaderA) * sizeof(char) ),
                        &cbWritten
                        );
       
if( !fResult )
        {
            hr = HRESULT_FROM_WIN32( GetLastError() );
            goto done;
        }

        //
        // Send the actual chunk data
        //

        if( cbData > 0 )
        {
            fResult = WinHttpWriteData(
                            hHttpRequest,
                            pbData,
                            cbData,
                            &cbWritten
                            );
            if( FAILED(hr) )
            {
                hr = HRESULT_FROM_WIN32( GetLastError() );
                goto done;
            }
        }

        //
        // Send the HTTP Chunk Transfer Encoding chunk-end mark
        //

        fResult = WinHttpWriteData(
                        hHttpRequest,
                        szHttpChunkFooterA,
                        (DWORD)( strlen(szHttpChunkFooterA) * sizeof(char) ),
                        &cbWritten
                        );
        if( FAILED(hr) )
        {
            hr = HRESULT_FROM_WIN32( GetLastError() );
            goto done;
        }

    done:
         return( hr );
    }  

Encoder Workflow

 

The encoding workflow for on-demand and live Smooth Streaming is nearly identical. The main difference is that instead of writing the data returned by SSFMuxGetHeader, SSFMuxProcessOutput and SSFMuxGetIndex to the stream’s destination file, the application posts the data to the stream URL using an HTTP POST (for example using the WriteToSmoothStreamOutput function above).

Streams can arrive at the IIS server in any order. Stream headers embed a mini-manifest that lists all the streams participating in the publishing point. The server uses these mini-manifests to wait for all streams to start sending data before it [the server] starts the publishing point. The insertion of this manifest in the stream header can be turned off by setting the muxer option SSF_MUX_OPTION_ENABLE_STREAM_MANIFEST_BOX_FOR_LIVE_MODE to FALSE (the default is TRUE). In this case the server will start the publishing point as soon as the first stream starts, and clients connecting at that time may see only a subset of all streams available. Because of that, this option should only be used when the presentation is generated by multiple encoders running independently.

Despite the server flexibility, applications should try to keep the streams in sync with regard to presentation time. Usually this is not a problem for live streams coming from a real-time encoding pipeline (e.g., a webcam). However, applications streaming on-demand content as live must be careful with runaway streams, in particular if these applications receive the media samples as fast as possible from the input pipeline. In this case a low bitrate stream could “play through” much faster than the rest and take over the network bandwidth to send its fragments while other streams fall behind in presentation time. In cases like this the application should try to pace or throttle the streams so they stay relatively in sync (in presentation time).

Ending a Stream

 

When a stream ends, the encoder application calls SSFMuxGetIndex to obtain the index data for that stream and send it to the server. In “live mode” the “index” is actually an EOS (End Of Stream) packet that tells the server the stream just ended. At this time applications using long running HTTP POST requests should finalize the transmission. To finalize a chunk transfer encoded request one simply sends a 0 length chunk. For the WriteToSmoothStreamOutput function shown above, that means calling it with cbData = 0. Finally, the application can receive the server’s response for the HTTP POST and close the socket. Here’s sample code using WinHTTP (in synchronous mode) to do that.


       
//
        // Receive the response from the IIS server for this
        // stream
        //

        fResult = WinHttpReceiveResponse(
                        hHttpRequest,
                        NULL
                        );
        if( !fResult )
        {
            hr = HRESULT_FROM_WIN32( GetLastError() );
            goto done;
        }

        //
        // Get the status code
        //

        cbStatusCode = sizeof(dwStatusCode);

        fResult = WinHttpQueryHeaders(
                        hHttpRequest,
                        WINHTTP_QUERY_STATUS_CODE | WINHTTP_QUERY_FLAG_NUMBER,
                        WINHTTP_HEADER_NAME_BY_INDEX,
                        &dwStatusCode,
                        &cbStatusCode,
                        WINHTTP_NO_HEADER_INDEX
                        );
        if( !fResult )
        {
            hr = HRESULT_FROM_WIN32( GetLastError() );
            goto done;
        }

        //
        // Handle status code and such
        //

        ...

       
//
        // Close the request
        //

        WinHttpCloseHandle( hHttpRequest );  

Manifests

 

In Live Smooth Streaming the manifests are generated on-the-fly by the IIS Server when a client connects to the publishing point. This is due to the dynamic and ongoing nature of a live presentation. In that case the encoder does not get to produce any of the manifests. In fact calling SSFMuxGetClientManifest or SSFMuxGetServerManifest when the muxer is in “live mode” is an invalid operation and will return an error code.

Conclusion

 

Live Smooth Streaming is a flexible yet simple protocol. However, the lack of detailed documentation makes its implementation a taxing task. I hope this blog post will help clarify some of the details.

 

Comments

I am little bit confused. Here what I do:

Open HTTP connection (I am going to have looong chunked request)

POST buffer from SSFMuxGetHeader

Then in loop POST buffer from SSFMuxProcessOutput

On IIS side I can see that pubpoint changed status to  "Started", but Incoming Bit Rate is 0, So I get nothing in my Silverlight player.

Am I doing something wrong ?

Tnx

Aleksey

Sep 07 2011 by hippocat

Hello Thales,

   I got Error on IIS Live Streaming Server when I push content to. The error message is "Element not fount", and Error Code is 0X80070490.

  My flow are : (for each stream)

  1. Post ftyp + XML + moov. (the XML is combind with ISM and ISMC, to point out meta-data)

  2. Post (moof+mdat) together, one by one.

  3. Post mfra

  4. Post EOS

  The Error should be happen on Step 2: Post (moof+mdat). Can you point me what should be possible reason ?

Kevin Kao

Oct 23 2012 by Kevin Kao

Hi,

Is there a release version of this SDK? The blog post date is 2011. I would have expected the SDK to be available by now. Is it scrapped?

Apr 19 2013 by sudeepforu

Some important piece of information is still missing here.

I can create a functional stream containing only audio, or only video, but

not both.

If I call SSFMuxGetHeader() individually for video and audio, and if I send this

data to the IIS, then I'm not able to send any compressed data to the IIS,

because the IIS terminates the connection immediately.

On the other hand, if I call SSFMuxGetHeader() only once to get both

(audio and video), I get the header containing both, send it to the IIS and I

can push the stream, but it seems like IIS is not accepting pushed data.

And of course, clients cannot play anything.

Aug 18 2013 by nkalisni

Got it! Turned off SSF_MUX_OPTION_ENABLE_STREAM_MANIFEST_BOX_FOR_LIVE_MODE and it works...

Aug 18 2013 by nkalisni

Submit a Comment

  • Plain text is accepted.
  • URLs starting with http:// are converted to links.