Thales' Blog

How to do Live Streaming with the Smooth Streaming Format SDK

2011-02-08T19:17:00Z

In a previous blog post I wrote about the sample application that comes with the IIS Smooth Streaming Format SDK. One important feature missing in that sample is code showing how to do Live Smooth Streaming. The next release of the SDK will correct that and include a new sample application that supports live streaming, yet that release is not planned to hit MSDN in the next couple of months, and in the meanwhile users are facing difficulties trying to add the functionality to their encoders. On top of that the Application Workflow documentation for the Smooth Streaming Format SDK is a little vague on live streaming. In lieu of that I decided to write this blog post to elaborate on some of the details. This article assumes familiarity with the SDK’s workflow and HTTP.

Setting up the muxer and streams

For live streaming the application must put the SDK’s muxer in live mode by setting the SSFMUX_OPTION_LIVE_MODE to TRUE. This must be done before adding streams to the muxer. Basically:

     BOOL fLiveMode = TRUE;
     hr = SSFMuxSetOption(
            hMux,
            SSF_MUX_OPTION_LIVE_MODE,
            &fLiveMode,
           sizeof(fLiveMode)
            );
    if( FAILED(hr) )
    {
        wprintf( L"Error: 0x%X trying to set Live Mode\r\n", hr );
        goto done;
    }

Sending the Streams to IIS

The Smooth Streaming Live Streaming Protocol requires that each stream sends its header, fragments and index (i.e., the data returned by SSFMuxGetHeader, SSFMuxProcessOutput and SSFMuxGetIndex) to the IIS Server using a HTTP POST request. The URL for the request is specific to each stream and must follow this pattern:

http://<server>/<pubpoint>/Streams(<identifier>)

The <identifier> part is an arbitrary name chosen by the application that uniquely identifies that stream within the publishing point. It must not contain invalid URL characters, URL separators or parentheses. It can be as simple as just a stream index. The same <identifier> string must be passed as the pszSourceFileName field of the SSF_STREAM_INFO struct when adding the stream to the muxer with SSFMuxAddStream.

The IIS Server is very flexible in terms of how it receives live streams. Fragments can be sent using multiple HTTP POST requests (one per fragment) or a single long running HTTP POST (one per stream). The server automatically filters out-of-order or duplicate fragments to produce the best possible continuous stream. For more details about the server operation, check this article.

Long running HTTP POST requests should be chunk transfer encoded. This type of HTTP encoding allows applications to start a HTTP request without knowing in advance the entire size of the payload (i.e., the “Content-Length”).

If you’re going to use WinHTTP to handle a long running HTTP POST, keep in mind that although WinHTTP can receive data in chunk transfer encoding transparently, when it comes to sending chunked data you have to handle the “chunking” on your own. The first thing to do in this case is to add the “Transfer-Encoding: Chunked” header to the HTTP request (note this precludes the need for a “Content-Length” header).

Here’s a quick code snippet showing how one would go about starting a chunked HTTP POST for a live stream with WinHTTP.

     WCHAR szAdditionalHeaders[] = L"Transfer-Encoding: Chunked";

     ...

     fResult = WinHttpSendRequest(
                    hHttpRequest,
                    szAdditionalHeaders,
                    ARRAYSIZE(szAdditionalHeaders)-1,
                    WINHTTP_NO_REQUEST_DATA,
                    0,
                    0,
                    NULL
                    );
    if( !fResult )
    {
        hr = HRESULT_FROM_WIN32( GetLastError() );
        goto done;
    }

And here’s an example of a function for posting HTTP chunks (assuming WinHTTP is operating in “synchronous” mode):

     HRESULT WriteToSmoothStreamOutput(
        __in HINTERNET hHttpRequest,
        __in_ecount(cbData) LPCVOID pbData,
        __in ULONG cbData )
    {
        HRESULT hr = S_OK;
        BOOL fResult = FALSE;
        DWORD cbWritten;

        char szHttpChunkHeaderA[32];
        char szHttpChunkFooterA[] = "\r\n";

        //
        // Send the HTTP Chunk Transfer Encoding chunk-start mark
        // Observe the use of UTF-8 strings.
        //

        hr = StringCchPrintfA(
                szHttpChunkHeaderA,
                ARRAYSIZE(szHttpChunkHeaderA),
                "%X\r\n",
                cbData );
        if( FAILED(hr) )
        {
            goto done;
        }

        fResult = WinHttpWriteData(
                        hHttpRequest,
                        szHttpChunkHeaderA,
                        (DWORD)( strlen(szHttpChunkHeaderA) * sizeof(char) ),
                        &cbWritten
                        );
      if( !fResult )
        {
            hr = HRESULT_FROM_WIN32( GetLastError() );
            goto done;
        }

        //
        // Send the actual chunk data
        //

        if( cbData > 0 )
        {
            fResult = WinHttpWriteData(
                            hHttpRequest,
                            pbData,
                            cbData,
                            &cbWritten
                            );
            if( FAILED(hr) )
            {
                hr = HRESULT_FROM_WIN32( GetLastError() );
                goto done;
            }
        }

        //
        // Send the HTTP Chunk Transfer Encoding chunk-end mark
        //

        fResult = WinHttpWriteData(
                        hHttpRequest,
                        szHttpChunkFooterA,
                        (DWORD)( strlen(szHttpChunkFooterA) * sizeof(char) ),
                        &cbWritten
                        );
        if( FAILED(hr) )
        {
            hr = HRESULT_FROM_WIN32( GetLastError() );
            goto done;
        }

    done:
         return( hr );
    }

Encoder Workflow

The encoding workflow for on-demand and live Smooth Streaming is nearly identical. The main difference is that instead of writing the data returned by SSFMuxGetHeader, SSFMuxProcessOutput and SSFMuxGetIndex to the stream’s destination file, the application posts the data to the stream URL using an HTTP POST (for example using the WriteToSmoothStreamOutput function above).

Streams can arrive at the IIS server in any order. Stream headers embed a mini-manifest that lists all the streams participating in the publishing point. The server uses these mini-manifests to wait for all streams to start sending data before it [the server] starts the publishing point. The insertion of this manifest in the stream header can be turned off by setting the muxer option SSF_MUX_OPTION_ENABLE_STREAM_MANIFEST_BOX_FOR_LIVE_MODE to FALSE (the default is TRUE). In this case the server will start the publishing point as soon as the first stream starts, and clients connecting at that time may see only a subset of all streams available. Because of that, this option should only be used when the presentation is generated by multiple encoders running independently.

Despite the server flexibility, applications should try to keep the streams in sync with regard to presentation time. Usually this is not a problem for live streams coming from a real-time encoding pipeline (e.g., a webcam). However, applications streaming on-demand content as live must be careful with runaway streams, in particular if these applications receive the media samples as fast as possible from the input pipeline. In this case a low bitrate stream could “play through” much faster than the rest and take over the network bandwidth to send its fragments while other streams fall behind in presentation time. In cases like this the application should try to pace or throttle the streams so they stay relatively in sync (in presentation time).

Ending a Stream

When a stream ends, the encoder application calls SSFMuxGetIndex to obtain the index data for that stream and send it to the server. In “live mode” the “index” is actually an EOS (End Of Stream) packet that tells the server the stream just ended. At this time applications using long running HTTP POST requests should finalize the transmission. To finalize a chunk transfer encoded request one simply sends a 0 length chunk. For the WriteToSmoothStreamOutput function shown above, that means calling it with cbData = 0. Finally, the application can receive the server’s response for the HTTP POST and close the socket. Here’s sample code using WinHTTP (in synchronous mode) to do that.

        //
        // Receive the response from the IIS server for this
        // stream
        //

        fResult = WinHttpReceiveResponse(
                        hHttpRequest,
                        NULL
                        );
        if( !fResult )
        {
            hr = HRESULT_FROM_WIN32( GetLastError() );
            goto done;
        }

        //
        // Get the status code
        //

        cbStatusCode = sizeof(dwStatusCode);

        fResult = WinHttpQueryHeaders(
                        hHttpRequest,
                        WINHTTP_QUERY_STATUS_CODE | WINHTTP_QUERY_FLAG_NUMBER,
                        WINHTTP_HEADER_NAME_BY_INDEX,
                        &dwStatusCode,
                        &cbStatusCode,
                        WINHTTP_NO_HEADER_INDEX
                        );
        if( !fResult )
        {
            hr = HRESULT_FROM_WIN32( GetLastError() );
            goto done;
        }

        //
        // Handle status code and such
        //

        ...

        //
        // Close the request
        //

        WinHttpCloseHandle( hHttpRequest );

Manifests

In Live Smooth Streaming the manifests are generated on-the-fly by the IIS Server when a client connects to the publishing point. This is due to the dynamic and ongoing nature of a live presentation. In that case the encoder does not get to produce any of the manifests. In fact calling SSFMuxGetClientManifest or SSFMuxGetServerManifest when the muxer is in “live mode” is an invalid operation and will return an error code.

Conclusion

Live Smooth Streaming is a flexible yet simple protocol. However, the lack of detailed documentation makes its implementation a taxing task. I hope this blog post will help clarify some of the details.

IIS Smooth Streaming Format SDK - Sample Application

2010-03-15T22:02:00Z

The IIS Smooth Streaming Format SDK is a library of code for encoder applications to pack media samples in the IIS Smooth Streaming container format (fragmented MP4 format) and generate the corresponding server and client manifests. The output of the SDK can be either written to files for media-on-demand scenarios or pushed to a live publishing point on the IIS server for immediate distribution.

The SDK’s installation package comes with an example of a basic muxing application. This application uses the SDK to generate the Smooth Streaming format files and manifests from a set of media files encoded in WMA, WMV/VC-1, H.264 or AAC-LC formats. The source code for this application serves as a functional example of how to use the SDK to mux Smooth Streaming content.

Here’s a very high-level view of the main blocks in the sample application:

The application uses the Microsoft DirectShow (DShow) application programming interface to grab the media types and compressed media samples from the input files. It then feeds those types and samples to the muxer object from the IIS Smooth Streaming Format SDK. Finally, the application takes the resulting fragmented MP4 chunks and manifest data generated by the muxer and writes them to the corresponding files (.ismv, .isma, .ismc or .ism).

Note: DirectShow does not ship with H.264 or AAC-LC filters. To enable these formats in the sample application you will need to install DShow filters for them.

1. Parse the input parameters

The application takes a text file as input. This text file contains a list of the media files and corresponding bitrates to mux, one media file per line. The format of each line is:

<bitrate> , <filename>

For example, the text file could contain a list like this:

904000, Movie_1.wmv
760000, Movie_2.wmv
580000, Movie_3.wmv
77000, Sound_1.wma

I will not go into the details of how the input file is parsed, but I should explain why the sample application requires users to input the bitrates explicitly. In theory the application could get the bitrate property directly from DShow, but the fact is that exposing this property is optional for DShow filters, and not every one of them does it, so to be more generic while maintaining the code simple the application requires the bitrate information as an explicit parameter.

2. Setup the media pipeline

The next step is to set up the input pipelines in a DShow graph, one branch for each input file. The pipeline will output the compressed elementary stream media samples to the application which will in turn feed them to the muxer. For each branch of the pipeline the application uses a custom “sample grabber” DShow filter with a simplified callback interface to receive the compressed samples. The default sample grabber filter that ships with the DirectShow API does not handle all the media types the application targets, hence the need for a custom filter. This custom filter comes in the dshowhlp.lib static library that accompanies the sample code. This library is intended for use in this example only and should not be used or re-distributed in any other applications.

The grabber filter has a single input pin that accepts any media type. Samples received in this pin are relayed to the application’s callback.

The interesting code to look at here is the StreamContext_AddToDShowGraph function, which sets up the DShow branch of the pipeline for each input file. Let’s take a look.

//------------------------------------------------------------------------------
HRESULT StreamContext_AddToDShowGraph(
    __inout StreamContext* pCtx,
    __in IGraphBuilder *pIGraph )
{
    HRESULT hr = S_OK;
    IBaseFilter* pBaseFilter = NULL;
    IPGI* pGrabber = NULL;

First we create a grabber filter for the file and stick it in the DShow graph. PGI_CreatedInstance is defined in dshowhlp.lib.

    hr = PGI_CreateInstance( IID_IPGI, (void**)&pGrabber );
    if( FAILED(hr) )
    {
        printf( "Error 0x%08x: failed to create sample grabber filter", hr );
        goto done;
    }

    hr = pGrabber->QueryInterface( IID_IBaseFilter, (PVOID*)&pBaseFilter);
    if( FAILED(hr) )
    {
        goto done;
    }

    hr = pIGraph->AddFilter( pBaseFilter, NULL );
    if( FAILED(hr) )
    {
        printf( "Error 0x%08x: failed to add sample grabber to graph", hr );
        goto done;
    }

Next we ask the DShow graph to render the input file. This means adding that file to the pipeline. DShow will instantiate the proper source filter for the file’s type and try to pull the necessary filters to complete the pipeline for that source. Since the grabber is preloaded in the graph, DShow will try it first, and because the grabber accepts all media types, DShow will go ahead and connect the source’s first output pin to the grabber’s only input pin.

Note this implies the source can have only one output pin. A limitation of this code example is it requires the input files to be single-stream. If there more than one media stream per file, let’s say an audio and video stream, DShow will connect the first to the grabber filter and then try to load default filters to connect the remaining streams. This can result in weird behavior like, for instance, the tool playing a stray video stream on the screen.

To support multiple streams per file would require constructing the pipeline programmatically. In other words, we would have to instantiate the source filter, enumerate its output pins, and connect each to a grabber filter. That would complicate the code more than its purpose.

    hr = pIGraph->RenderFile( pCtx->szInputFileName, NULL );
    if( FAILED(hr) )
    {
        printf( "Error 0x%08x: failed trying to open file in DShow graph", hr );
        goto done;
    }

    if (FALSE == IsConnected(pBaseFilter))
    {
        printf( "Error 0x%08x: failed to connect sample grabber", hr );
        goto done;
    }

Once the new branch of the pipeline is connected, we register the application’s callback in the grabber filter. That’s the callback the grabber will call to deliver the compressed samples.

    hr = pGrabber->set_Callback2( HandleSampleHook, pCtx );
    if( FAILED(hr) )
    {
        printf( "Error 0x%08x: failed to set the IPGI callback", hr );
        goto done;
    }

Finally we extract the media type of the new branch, which we’ll need later to setup the SDK’s muxer. I won’t go into detail on how to get that media type. Hopefully the code is self-explanatory.

    hr = StreamContext_SetTypeInfoFromRenderer(
                pCtx,
                pBaseFilter );
    if( FAILED(hr) )
    {
        goto done;
    }

done:

    SAFE_RELEASE( pBaseFilter );
    SAFE_RELEASE( pGrabber );

    return ( hr );
}
//------------------------------------------------------------------------------

3. Create the SSF muxer and set it up

Once the input pipeline is set up, the next step is to instantiate the muxer object from the IIS Smooth Streaming Format SDK. Let’s look at the code.

First we use SSFMuxCreate to create the muxer.

    hr = SSFMuxCreate( &hSSFMux );
    if( FAILED(hr) )
    {
        printf( "Error 0x%08x: Failed to create SSF muxer\n", hr );
        goto done;
    }

Then we configure the basic options.

    UINT64 timeScale = 10000000; // time scale in HNS

    hr = SSFMuxSetOption( hSSFMux, SSF_MUX_OPTION_TIME_SCALE, &timeScale, sizeof(timeScale) );
    if( FAILED(hr) )
    {
        printf( "Error 0x%08x: Failed to set SSF_MUX_OPTION_TIME_SCALE\n", hr );
        goto done;
    }

    BOOL fVariableBitRate = TRUE;

    hr = SSFMuxSetOption( hSSFMux, SSF_MUX_OPTION_VARIABLE_RATE, &fVariableBitRate, sizeof(fVariableBitRate) );
    if( FAILED(hr) )
    {
        printf( "Error 0x%08x: Failed to set SSF_MUX_OPTION_VARIABLE_RATE\n", hr );
        goto done;
    }

Finally we add one stream in the muxer for each input file. We need to add all streams we intended to mux upfront.

    //
    // Add streams to the muxer
    //

    for( ULONG i = 0; i < cStreams; ++i )
    {
        hr = AddStreamToMux(
                    &rgStreams[i],
                    hSSFMux,
                    ( AUDIO == rgStreams[i].eStreamType ) ? cAudioStreams++ : cVideoStreams++ );
        if( FAILED(hr) )
        {
            printf( "Error 0x%08x: Failed to set add stream to mux\n", hr );
            goto done;
        }

        rgStreamIndices[i] = rgStreams[i].dwStreamIndex;
    }

Let’s take a look in AddStreamToMux to see what it does:

//------------------------------------------------------------------------------
HRESULT AddStreamToMux(
    __inout StreamContext* pCtx,
    __in SSFMUXHANDLE hSSFMux,
    __in ULONG iStream )
{
    HRESULT hr = S_OK;
    SSF_STREAM_INFO streamInfo;

    memset( &streamInfo, 0, sizeof(streamInfo) );

    pCtx->hSSFMux = hSSFMux;

Here we format the archive name for the stream. We use “Audio<n>.isma” for audio streams and “Video<n>.ismv” for video streams. Basically that’s the name of the ISM file to contain the media stream. The current version of the SDK allows one stream per ISM file only. Although the SDK does not write the fragmented MP4 data to the files directly, the data may contain offsets that impose assumptions on how the data is persisted to files, hence the limitation. The next version of the SDK (Beta 2) shall have API to synchronize offsets and give more freedom to the application to merge multiple streams in one ISM.

    if( 0 > swprintf_s(
                pCtx->szArchiveName,
                MAX_PATH,
                ( pCtx->eStreamType == AUDIO ) ? L"Audio%d.isma" : L"Video%d.ismv",
                iStream ) )
    {
        hr = E_UNEXPECTED;
        goto done;
    }

Next we choose how long each fragment will be. For simplicity we’ll fix the value at 5 seconds for audio streams and 2 seconds for video streams. We select this value here for convenience, but note that it’s not used directly by the SDK’s muxer. Instead, we’ll use it later in the callback function to decide where to split the fragments.

    if( pCtx->eStreamType == AUDIO )
    {
        pCtx->rtChunkDuration = 50000000; // 5 seconds in hns units
    }
    else
    {
        pCtx->rtChunkDuration = 20000000; // 2 seconds in hns units
    }

    streamInfo.streamType = pCtx->eStreamType;
    streamInfo.dwBitrate = pCtx->bitrate;
    streamInfo.pszStreamName = pCtx->szArchiveName;
    streamInfo.pTypeSpecificInfo = pCtx->pbTypeInfo;
    streamInfo.cbTypeSpecificInfo = pCtx->cbTypeInfo;
    streamInfo.wLanguage = MAKELANGUAGE('e','n','n');

Finally we add the stream to the muxer with SSFMuxAddStream. If successful, the API returns a stream index that we use to refer to this particular stream for now on.

    hr = SSFMuxAddStream( hSSFMux, &streamInfo, &pCtx->dwStreamIndex );
    if( FAILED(hr) )
    {
        goto done;
    }

done:

    return( hr );
}
//------------------------------------------------------------------------------

4. Get the Smooth Streaming headers

The next step is to get the headers for each stream and write them to the corresponding .ismv or .isma file. Note that SSFMuxGetHeader takes a list of stream indices. This is a provision to allow applications to merge multiple streams into a single .ismv or .isma file. However, due to the limitations mentioned before, that’s not fully supported in the Beta 1 version of the SDK.

Note that the sample application will re-write the headers later when it finalizes the files. This needs to be done because the contents of the headers change as the muxer processes media samples. In fact, the only reason why the sample application requests the headers at this early point is to reserve room for the final headers in the files. The contents of the headers change, but their total lengths do not. Later the application can just re-write the final headers without the need to shift the contents of the files.

    for( ULONG i = 0; i < cStreams; ++i )
    {
        hr = SSFMuxGetHeader( hSSFMux, &rgStreams[i].dwStreamIndex, 1, &outputBuffer );
        if( FAILED(hr) )
        {
            printf( "Error 0x%08x: Failed to get FMP4 header\n", hr );
            goto done;
        }

        hr = StreamContext_WriteDataToFile(
                &rgStreams[i],
                L"wb",
                outputBuffer.pbBuffer,
                outputBuffer.cbBuffer );
        if( FAILED(hr) )
        {
            printf( "Error 0x%08x: Failed to write header to FMP4 file\n", hr );
            goto done;
        }
    }

5. Mux the Media Streams

It’s time to run the DShow graph and feed the incoming compressed samples into the muxer. That’s controlled by the Play function, a regular DShow run loop that starts the graph to play back the input streams through the grabber filters and then dispatches events from the graph until the playback ends.

The important code to look at here is the StreamContext_HandleSample function. This is the function that receives the samples and feeds them into the muxer. First let’s take a look at HandleSampleHook, the callback that the application hooks up to each grabber filter. This function maps the callback application data back to the StreamContext structure that binds an input stream in the graph to an output stream in the muxer.

//------------------------------------------------------------------------------
HRESULT WINAPI HandleSampleHook( __in void* pCtx, __in_opt IMediaSample* pSample )
{
StreamContext* pStreamCtx = (StreamContext*)pCtx;

return( StreamContext_HandleSample( pStreamCtx, pSample ) );
}
//------------------------------------------------------------------------------

Back to StreamContext_HandleSample, it feeds samples into the muxer by calling ProcessInput. If a fragment is complete, the function calls ProcessOutput to close that fragment and extract its data. Note that it is up to the application to decide when to close a fragment. This does not mean the application can close a fragment at any sample. The fragmented MP4 format requires fragments to contain full GOPs (Group of Pictures), yet the SDK does not attempt to detect these boundaries in order to enforce them – that is a responsibility of the application.

StreamContext_HandleSample uses the IsSyncPoint method of the incoming sample’s IMediaSample interface to establish GOP boundaries. Note, however, that all IsSyncPoint really does is to tell if the sample is an I-frame or not, yet not every I-frame needs to be a GOP boundary (unless the content was encoded that way). This implies the sample application only works with input files that have been encoded such that every I-frame is in its own closed GOP with no addition I-frames. Some H.264 content may contain an IDR frame at the beginning of the CVS (coded-video sequence), but it can also contain I-frames in the CVS prior to the next IDR frame. The sample application may generate invalid .ismv files for input files that are encoded with multiple I-frames in a GOP. This is a limitation of this example in particular. The SDK can handle the scenario correctly provided the application makes sure no GOP gets split in multiple fragments.

//------------------------------------------------------------------------------
HRESULT StreamContext_HandleSample(
    __in StreamContext* pCtx,
    __in_opt IMediaSample* pSample )
{
    HRESULT hr = S_OK;

    BOOL fProcessOutput = FALSE;

    REFERENCE_TIME rtStartTime = 0;
    REFERENCE_TIME rtStopTime = 0;

    //
    // Verify the conditions to pull a fragment for this stream
    //

    if( NULL == pSample )
    {
        //
        // A null sample means the stream has ended.
        // Pull the last fragment then.
        //

        fProcessOutput = TRUE;
    }
    else
    {
        //
        // Pull a fragment if the sample is a sync point
        // and the fragment duration has been reached
        //

        hr = pSample->GetTime( &rtStartTime, &rtStopTime );
        if( FAILED(hr) )
        {
            goto done;
        }

        pCtx->rtChunkCurrentTime = rtStartTime;

Apart from looking for sync-point samples, the function also uses the fragment’s accumulated duration to decide when to close the fragment. A fragment gets closed right before the first sync-point sample that exceeds the expected duration (which in this example is 5 seconds for audio streams and 2 seconds for video streams). The sample that triggers this will be carried over to the next fragment.

IMPORTANT: Before closing the current fragment we must call SSFMuxAdjustDuration with the current sample’s start time to update the duration of the fragment that is about to get closed. This step is necessary because certain media types do not provide accurate duration for the media samples, and to find a sample’s correct duration one needs to look at the start time of the sample that comes next. In cases like this the muxer does not know how to compute the fragment’s total duration solely from the media samples in that fragment; the muxer also needs the start time of the very next sample right after the last one that went in the fragment.

The SSFMuxAdjustDuration API needs to be called only on fragment boundaries.

Of course the other condition that causes a segment to get closed is when the end-of-stream arrives. This is indicated by a null pSample pointer.

        if( ( S_OK == pSample->IsSyncPoint() )
            && ( pCtx->rtChunkCurrentTime - pCtx->rtChunkStartTime >= pCtx->rtChunkDuration ) )
        {
            fProcessOutput = TRUE;

            hr = SSFMuxAdjustDuration( pCtx->hSSFMux, pCtx->dwStreamIndex, rtStartTime );
            if( FAILED(hr) )
            {
                goto done;
            }
        }
    }

Calling SSFMuxProcessOutput closes the current fragment and returns a buffer with the data to be appended to the corresponding .ismv or .isma file. The content of the buffer is valid only until the next call to SSFMuxProcessOutput for the same muxer. Applications should not hold this buffer more than necessary. If needed, applications must copy the data to a separate buffer.

    //
    // Pull a fragment if the conditions require
    //

    if( fProcessOutput )
    {
        SSF_BUFFER outputBuffer;

        hr = SSFMuxProcessOutput( pCtx->hSSFMux, pCtx->dwStreamIndex, &outputBuffer );
        if( FAILED(hr) )
        {
            goto done;
        }

        printf( "Stream #%d TS=%I64d Duration=%I64d\n",
                pCtx->dwStreamIndex,
                outputBuffer.qwTime,
                outputBuffer.qwDuration );

        ++pCtx->dwChunkIndex;

        hr = StreamContext_WriteDataToFile( pCtx, L"ab", outputBuffer.pbBuffer, outputBuffer.cbBuffer );
        if( FAILED(hr) )
        {
            goto done;
        }

        pCtx->fChunkInProgress = FALSE;
    }

The next step is to feed the sample into the muxer. That’s done through the SSFMuxProcessInput API. If a fragment has just been closed in the previous portion of the code, or if this is the very first sample for this stream, the API will start a new fragment for that stream.

    //
    // Process the input sample if any
    //

    if( NULL != pSample )
    {
        SSF_SAMPLE inputSample = { 0 };

        BYTE* pbSampleData = NULL;

        hr = pSample->GetPointer( &pbSampleData );
        if( FAILED(hr) )
        {
            goto done;
        }

        inputSample.qwSampleStartTime = (UINT64)rtStartTime;
        inputSample.pSampleData = pbSampleData;
        inputSample.cbSampleData = pSample->GetActualDataLength();
        inputSample.flags = SSF_SAMPLE_FLAG_START_TIME;

        if( pCtx->fSampleDurationIsReliable )
        {
            inputSample.qwSampleDuration = rtStopTime - rtStartTime;
            inputSample.flags |= SSF_SAMPLE_FLAG_DURATION;
        }

        if( !pCtx->fChunkInProgress )
        {
            pCtx->rtChunkStartTime = rtStartTime;
            pCtx->fChunkInProgress = TRUE;
        }

        hr = SSFMuxProcessInput( pCtx->hSSFMux, pCtx->dwStreamIndex, &inputSample );
        if( FAILED(hr) )
        {
            goto done;
        }
    }

done:

   return ( hr );
}
//------------------------------------------------------------------------------

It is not necessary to send the end-of-stream signal back to the main loop in the Play function. The DShow graph does that automatically once all input streams reach the end.

6. Finalize the .ismv and .isma files

To finalize a .ismv or .isma file the application needs to update the header and append the index that corresponds to that file.

Early on the application had already written a placeholder header to each file to reserve room for the final header. Since only the contents of the header changed during muxing, but not its total length, all the application has to do now is to call SSFMuxGetHeader once again to get the header for each individual stream and rewrite the data to the beginning of the file corresponding to that stream. The same goes for the index, the only difference is that the application uses SSFMuxGetIndex to get it and appends the data to the end of the file. The index contains indexing information for all the fragments in that file plus the closing MP4 atoms that mark the end of the file.

Note that SSFMuxGetIndex is similar to SSFMuxGetHeader in the sense that both can be used to obtain an index (or header) that combines multiple streams in a single file. As mentioned before, however, the Beta1 version of the SDK does not fully support the scenario due to other limitation.

7. Generate the Manifests

The last step in the sample application is to emit the server and client manifests. To get those pieces of data the application calls SSFMuxGetServerManifest and SSFMuxGetClientManifest, and then it writes the data to the corresponding manifest files.

Note that although the SDK has a Unicode API set, SSFMuxGetServerManifest and SSFMuxGetClientManifest in particular return the manifest text in ASCII format. This is to allow the application to write the data verbatim to the manifest files without the need for UTF8 or UTF16 encoding.

Another important thing to notice is that SSFMuxGetServerManifest takes a list of streams in the same way that SSFMuxGetHeader and SSFMuxGetIndex do. However, in this case the application does call the API passing in the entire set of streams. This is because the server manifest is supposed to contain the entire set.

8. Conclusion

Despite being a simple tool, the basic muxer sample application is a good example of the workflow that encoder applications need to follow when using the IIS Smooth Streaming Format SDK to generate on-demand Smooth Streaming files.