Vishal Sood : Media

Advanced Logging – How does Real-time logging work?

vsood — Fri, 20 Mar 2009 17:41:00 GMT

We announced and release a Beta for a sparkling new Logging module for IIS at MIX 09. This is a super rich logging module with a lot of functionality that our team will continue to blog about. Today, I wanted to share more technical details on how my favorite feature, real-time logging work in Advanced Logging.

Before we go into details, you may want to take a short pause and read some overview material in my blog post, extension page for Advanced Logging or go over the custom logging walkthrough. Okay, I will now assume you are somewhat familiar with this module.

Next, let us take a scenario - You have a SL or any HTTP client that collects analytic information that you want to be able to send to IIS web server and get that captured in a log file. Let’s add a little twist, while you want this you also want to be able to write custom code that can consume this log information in real time and enable you to do some real time reporting. The reason you want to do that is just to provide a report on real-time interactions by clients or simply monitor your network in real-time.

Here is how IIS Advanced Logging helps

Let us go over this architecture, step-by-step:

The clients on the left hand side (could be Silverlight application, native Silverlight runtime, could be ASP.net or anything really), sends a HTTP POST message with the log in the body. It is a simple XML format and you can find details in the walkthrough Real-time Logging.
This message lands on an IIS server that has Advanced Logging (AL) installed. The client logging functionality in AL parses this message and raises an internal event.
The core server-side logging functionality gets this event and then looks up the log configuration to compose a log entry and writes it to the file. (Note: you would need to have relevant configuration so that AL knows what fields to expect. You can find an example in the Client logging walkthrough.
In addition, just before an entry is written to the log file, server-side logging raises another event using public APIs (IIS HTTP_TRACE_EVENT infrastructure) that can be consumed by a simple module written by YOU. Believe me, it is a simple module. You can find details and sample code in the walkthrough Real-time Logging.
Now, your module can take this information push it to a service in the cloud, a DB and produce reports in real-time.

Let me know if you need more details. The best to learn more is to try it out :).

IIS Advanced Logging Beta is now Live

vsood — Wed, 18 Mar 2009 19:29:00 GMT

We proudly announce the Beta of IIS Advanced Logging. With Advanced Logging you will be able to capture rich details in your logs and process them in real-time. These capabilities make this feature really compelling by helping you monetize your content with better analytics and improve service with rich QOS information.

IIS Advanced Logging includes the following features:

Client-side logging. Advanced Logging builds capabilities into IIS to accept client-side consumption metrics. This provides relevant and important user engagement metrics that your customers will love.
Centralized logging. Advanced Logging provides capabilities to set up a central log farm that collects and logs client-side metrics. This reduces the effort required to collect distributed logs, and consolidated log collection results in more efficient post-processing. I wanted to clarify that this is not for all logging. Note: This is for client-side logging.
Multiple Logs per request. Advanced Logging provides capabilities to filter information relevant to specific purposes by creating multiple logs per request, with each log containing data relevant to the log's purpose. For example, Quality of Service (QoS) data in one log and Analytics data in another. All the post-processing required to distill the information using today's solutions is no longer needed.
Real-time logging. Advanced Logging can extend the platform to support real-time analytics, helping you to provide real-time reports to customers or work with partners to do the same.
Rich/Flexible logging. Advanced Logging does not assume what is important to you. Instead, it provides enough control to specify what fields are important to you, add additional fields with ease, and provide policies around log rollover and request filtering. These capabilities do not exist today. Now you can log any of the HTTP request/response headers, IIS server variables, and client-side fields with simple configuration through the management console.
Application Logging. Making use of the IIS configuration system, Advanced Logging provides capabilities to generate separate logs for sites, applications and so on.
Module Specific Log fields. If you are developing modules for IIS, you can specify custom fields that can be collected in logs.

You can learn more about the feature by reading the walkthroughs:

Getting started with Advanced Logging
Real-Time Logging
Client Logging
Learn more about the feature on - http://www.iis.net/extensions/AdvancedLogging

We eagerly await your feedback and comments on the Advanced Logging Discussion Forum.

Web Playlists SMIL conversion tool

vsood — Fri, 12 Sep 2008 16:38:00 GMT

With the RC version of Web Playlists, we moved from a custom XML format to a SMIL-based format for storing playlists. As a result of this change the old isx format has been retired. However, Brian Blum (a dev) in our team crafted a tool to help convert the old files. The tool can be downloaded here. Thank you Brian!

As both old and the new SMIL-based playlist formats are XML by nature, this tool uses a XSL to transform the old isx to the new SMIL-based isx format. Another reason for doing this is Brian wanted you to be able to do any minor changes if you so desire without needing to look into the code.

The tool usage is simple and as follows:

isxTranslate.exe /? Coverts prerelease input files for IIS 7.0 Web Playlists to the release format.

Usage: isxTranslate /i PLAYLISTDIR /x XSLT

where PLAYLISTDIR is the path to your playlist files and

XSLT is the xslt to use to convert the playlists.

Example: isxtranslate /i c:\inetweb\wwwroot\playlists /x c:\tools\isx.xsl

Note: This tool is just to help migrate old files and is not released as a part of Web Playlists RC download. As a result this tool is not officially supported. We will be happy to answer any questions and would encourage you to use media forum for your questions.

DOWNLOAD THE TOOL HERE

Web Playlists Release Candidate (RC) Released

vsood — Fri, 12 Sep 2008 16:08:00 GMT

I am pleased to announce that the Web Playlists Release Candidate (RC) release is ready for you to download and deploy.

This release supports upgrade from the previous Go Live release. (Note: It is recommended that if you have a release earlier than Go Live, you should uninstall the old build and then use the downloads below to do a fresh installation.)

You can get the RC downloads here:

32 bit - Download
64 bit - Download
SDK - Download

Here is a list of walkthroughs published that will help you get going with this release:

Please refer to the online readme for more details.

For general information on how Web Playlists work, please see this post.

For us, this release was primarily about two things:

A step towards embracing open standards - The playlist format now is SMIL-based and additionally you can output the playlist in any XML/text format. The walkthrough talks more about this feature and illustrates and example where you can use ATOM as an output format.
Even more thorough testing - This is incremental release over our earlier Go Live release and our test team has tested it even more than previous release

The three key new features in Web Playlists RC release are:

Ability to customize output format - This release enables you to change the response to a Web Playlists request to a format understood by your app. This feature works by taking and XSL style-sheet from you and transforming the ASX response according to it. The walkthrough talks about this in more details.
Allow Client-side Caching - Earlier releases of Web Playlists disabled the client-side caching using http headers. We got customer requests suggesting that although this is a great security feature, they would like to control this so in this release we allow you just that. You can control, whether clients can cache content or not.
SMIL-based playlist format - Earlier releases used a custom XML. With this release, we are moving a step closer to open standards. We are now using a SMIL-based format. While, Web Playlists does not support the entire SMIL syntax, the sub-set supported is fully SMIL compliant. For those upgrading from previous builds, there is a tool available here to convert old isx files.

We would like to hear what you feel about this release and would love to hear how you are using it.

Got more questions or need support?: Please use media forum.

Web Playlists - using SMIL as input format

vsood — Thu, 07 Aug 2008 16:40:00 GMT

In our strive to make it utmost convenient for our customers to use Web Playlists, we are considering another feature. In my last post I discussed the option of being able to configure the output format for Web Playlists and use open standards like ATOM as possible output in addition to ASX today. We are actually thinking of allowing you to add a XSLT and then be able to configure the output format of your choice that way.

Continuing on the same lines I wanted to explore the option of a standard input format this time. Today we use POX. While this works, it has the problems of not being understood by other media servers and honestly you have to learn a new format. The idea we have now is to use SMIL as an input format. Windows Media Services already understands the SMIL input for Server-side Playlists and so do many other media server platforms. For this release, we are exploring supporting only a sub-set of SMIL modules- SMIL Structure, Media Object (audio,video and ref) and Metainformation modules.

Here is an example of the SMIL playlist would look like. We haven't closed on this yet and would love to hear any inputs you might have here (the example uses SMIL 2.1 but it would really work with SMIL 2.0 or SMIL 3.0 too)

<!DOCTYPE smil PUBLIC "-//W3C//DTD SMIL 2.1//EN"
                      "http://www.w3.org/2005/SMIL21/SMIL21.dtd">  
<smil xmlns="http://www.w3.org/2005/SMIL21/Language">
  <head>
    <meta name="title" content="Web Playlists Sample"/>    
    <meta name="author" content="IIS.net"/>    
    <meta name="copyright" content="(c)2008 Microsoft Corporation"/>
    <meta name="timeToLive" content="1440"/>    
    <meta name="inactivityTimeout" content="120"/>    
  </head>
  <body>
    <ref src="a.wmv" >
      <param name="canSkipForward" value="true" />
      <param name="canSkipBack" value="true" />
      <param name="canSeek" value="true" />
    </ref>
  </body>
</smil>

How do you feel about this change? We would love to hear from you.

Web Playlists output format

vsood — Sat, 26 Jul 2008 21:36:00 GMT

We recently release the Go Live of Web Playlists module and I am sure a lot of you are already beginning to realize the potential of this feature.

After the release our very smart set of engineers are figuring what can we do with this feature next? There is always a balance we need to draw in the features we can support and how soon can we get the product out, so that our customers (you) can benefit from it.

One thing we wanted to explore and hear your feedback on is the output format for the Web Playlists. While we allow you to change the input format, using custom providers, we do not have a way for you to change the output format. Currently, Web Playlists outputs ASX (client-side playlist) as the output. If you don' know much about ASX, it is basically a client-side playlist format (XML based) supported by Windows Media Player and Silverlight. You can also make it work with the Flash player and we have more details here. You can read more about the ASX format on MSDN here.

The question I have for all you is how much do you care about the output format? Would it be beneficial if we enabled you to output the client side playlist in format of your choice (other than ASX) too, or supported something like ATOM syndication format (RFC 4287)out of the box? In our team we deeply care about your feedback so any thoughts are really helpful.

If you have more questions, I will love to answer them.

Making Web Playlists URLs better with URL Rewrite module

vsood — Tue, 22 Jul 2008 03:21:09 GMT

I have been recently reading about the entire RESTful web services and constantly looking at how we can make our modules more REST friendly in terms of URL look-n-feel and also the output they return (if any). I blogged about RESTful web services in case anyone is interested in knowing more about REST.

At the same time, if you were following Bill's blog, IIS team released URL re-write module. IIS pipeline architecture is such that it almost always requires an extension, URL re-write module helps overcome this limitation. In this post I plan to cover how we can make Web Playlists URLs look prettier (and aligned to REST principles).

Today, Web Playlists URLs are look something like this http://www.example.com/a.isx. For someone looking at this URL and trying to understand, it does mean much. As you have been following my blog all this while ;), you know what isx is (Web Playlists file) but not everyone does. One of the principles of REST as I see applied is that URLs should tell you a lot about what to expect in the response. Let us see how we can live this principle for Web Playlists.

Consider the URLs below

These URLs clearly tell me what to expect as a response. It tells me that response will be a playlist of some kind.

Creating a Rule

Let us take the URL http://example.com/Playlist('a'). In this case the URL tells you that there is a playlist 'a' that this URL refers to. This syntax is very much similar to the Astoria syntax. "Astoria" is the code name for ADO.net Data Services. They are heavily investing in defining URL syntax. Let us see how we can get our existing URLs to look like more like Astoria URLs.

We need to first install URL re-writer module. This is an Out of Band module available for free download from http://www.iis.net site: 32-bit and 64-bit. Once you install you will see the Icon for URL re-writer appear in IIS Manager (highlighted below)

(click to see a larger image)

Double-click on the icon and then click "Add Rule...". Then add rule as per image below:

(click to see a larger image)

For the people familiar with IIS configuration files, this creates the following entry in the IIS config file

(click to see a larger image)

How does this rule work?

If we dissect this rule, "^(.*)/*Playlist$(.*)$$" pattern says that in URL match everything before "/Playlist" and then match everything in between (). As per the URL re-writer module semantics these will be stored in {R:1} and {R:2} variables which are then used while re-writing the rule. (E.g., in http://example.com/Playlist('a') {R:1} becomes blank and {R:2}becomes a. This is because only URL relative to site root are matched. Thus as per the rule the initial URL will be re-written as http://example.com/a.isx which is understood by the IIS7 pipeline.

Re-cap

In this post I ran through an example of how you could use URL-writer to create friendlier URLs for Web Playlists. I would really like to encourage you to play with URL re-write module to create the other URL form (http://example.com/Playlist/a) and share it as comments to this blog.

I would suggest starting with the walkthroughs here. Happy re-writing and looking forward to your comments.

Web Playlists Go Live release is now ready for downloads

vsood — Mon, 21 Jul 2008 21:33:00 GMT

I am really excited to announce that Web Playlists Go Live was just released to public. Here are the download links for the same:

Here is a list of walkthroughs published on the all new learn.iis.net:

Here is more about this release: http://www.iis.net/Downloads/files/playlist/WebPlaylists_Readme_GoLive.htm.

For general information on how Web Playlists work, please see this post.

Web Playlists Go Live release includes these key features:

Support for all digital media files types. Any digital media file type that can be downloaded from a Web server can be added to a Web playlist by using this feature.
Nested Web playlist media entries. Web playlist files (.isx) can be added as media entries to the Web playlists that you create, up to five levels deep. For more information about how nested playlists work in this feature, see Creating a Simple Playlist.
Remote HTTPd media entries. Content locations that begin with the httpd:// URL prefix can be added as media entries to the Web playlists that you create, allowing you to reference a remote Web server page that returns a custom URL or path for the media item to be played.
Session settings. Allows you to specify the maximum number of clients that can connect to the playlist and the amount of content that must be watched before skip functionality is enabled on the client.
Session provider settings. Allows you to add ASP.NET session persistence and provides an additional level of control for the content on a Web server by configuring multiple options that specify how long the content referenced by a playlist is available to users.
Impersonation settings. Allows for media items that are protected by user credentials to be added to playlists.
ASX import. Allows for quick conversion of client-side playlists (files with .asx file name extensions) for files stored on the Web server to Web playlists.
Custom providers. A COM interface is included that you can use to write native and .NET-based custom providers that extend the default behavior of Web Playlists. For more information, see the Extending Web Playlists through custom providers.

Got more questions or need support?: Please use media forum.

Web Playlists - How does that thing work?

vsood — Thu, 01 May 2008 04:31:00 GMT

Web Playlists! CTP2 been out there for a little more than a week now and some of you simply love it. Others, I am sure love it but I just don't know about that. Some others wonder how it works and the documentation available is a drop in the ocean. Firstly...(read more)

Short-cut to integrate Web Playlists with an existing web application

vsood — Tue, 29 Apr 2008 05:57:32 GMT

If you read my post / walkthroughs you probably know about the extensibility interface shipped with CTP2. You can use that to integrate into existing business logic applications or web applications. However, in this post I am going to talk about a shortcut...(read more)

Bit Rate Throttling is the pick of the week on Channel 9

vsood — Tue, 22 Apr 2008 20:30:00 GMT

Bit Rate Throttling was Brian's pick of the week on channel 9. We love you Brian :) Here is the link to video and the post on the site - Click (To see the relevant section in the video seek to around 16:28 into the video.) Here is the excerpt from the...(read more)

Exploring Custom providers - Wrapper playlists

vsood — Tue, 22 Apr 2008 18:40:00 GMT

In my blog post yesterday I mentioned that we will be releasing sample code written by our teams as reference. Here is the first one. Prakash blogged about a custom provider that provides a wrapper playlist like functionality through a simple playlist...(read more)

IIS 7 Media Pack Web Playlists Customer Technology Preview 2 (CTP2) is Live

vsood — Tue, 22 Apr 2008 00:28:00 GMT

I am really happy to announce that Web Playlists CTP2 is now released. Web Playlists CTP2 builds on top of CTP1 release that we had in February. It has the following additional features Extensibility through custom providers – We are releasing interfaces...(read more)

Bit Rate Throttling is 1 week old - some interesting thoughts and applications

vsood — Fri, 21 Mar 2008 20:55:00 GMT

Last week, we announced the release of Bit Rate Throttling (BRT) module and there has been a lot of positive feedback already. I love the way the world of media latches on to new things so fast. Scott Guthrie posted about the release in his blog here...(read more)

Dynamic Throttling with Bit Rate Throttling

vsood — Mon, 17 Mar 2008 20:56:00 GMT

In the latest RTW release of Bit Rate Throttling, we have an amazing feature - dynamic throttling. It can also be looked at as bandwidth splitting. The primary intentions of this post is to serve as an introduction to this feature. In our earlier releases...(read more)