Advanced Logging – How does Real-time logging work?

vsood — Fri, 20 Mar 2009 17:41:00 GMT

We announced and release a Beta for a sparkling new Logging module for IIS at MIX 09. This is a super rich logging module with a lot of functionality that our team will continue to blog about. Today, I wanted to share more technical details on how my favorite feature, real-time logging work in Advanced Logging.

Before we go into details, you may want to take a short pause and read some overview material in my blog post, extension page for Advanced Logging or go over the custom logging walkthrough. Okay, I will now assume you are somewhat familiar with this module.

Next, let us take a scenario - You have a SL or any HTTP client that collects analytic information that you want to be able to send to IIS web server and get that captured in a log file. Let’s add a little twist, while you want this you also want to be able to write custom code that can consume this log information in real time and enable you to do some real time reporting. The reason you want to do that is just to provide a report on real-time interactions by clients or simply monitor your network in real-time.

Here is how IIS Advanced Logging helps

Let us go over this architecture, step-by-step:

The clients on the left hand side (could be Silverlight application, native Silverlight runtime, could be ASP.net or anything really), sends a HTTP POST message with the log in the body. It is a simple XML format and you can find details in the walkthrough Real-time Logging.
This message lands on an IIS server that has Advanced Logging (AL) installed. The client logging functionality in AL parses this message and raises an internal event.
The core server-side logging functionality gets this event and then looks up the log configuration to compose a log entry and writes it to the file. (Note: you would need to have relevant configuration so that AL knows what fields to expect. You can find an example in the Client logging walkthrough.
In addition, just before an entry is written to the log file, server-side logging raises another event using public APIs (IIS HTTP_TRACE_EVENT infrastructure) that can be consumed by a simple module written by YOU. Believe me, it is a simple module. You can find details and sample code in the walkthrough Real-time Logging.
Now, your module can take this information push it to a service in the cloud, a DB and produce reports in real-time.

Let me know if you need more details. The best to learn more is to try it out :).

IIS Advanced Logging Beta is now Live

vsood — Wed, 18 Mar 2009 19:29:00 GMT

We proudly announce the Beta of IIS Advanced Logging. With Advanced Logging you will be able to capture rich details in your logs and process them in real-time. These capabilities make this feature really compelling by helping you monetize your content with better analytics and improve service with rich QOS information.

IIS Advanced Logging includes the following features:

Client-side logging. Advanced Logging builds capabilities into IIS to accept client-side consumption metrics. This provides relevant and important user engagement metrics that your customers will love.
Centralized logging. Advanced Logging provides capabilities to set up a central log farm that collects and logs client-side metrics. This reduces the effort required to collect distributed logs, and consolidated log collection results in more efficient post-processing. I wanted to clarify that this is not for all logging. Note: This is for client-side logging.
Multiple Logs per request. Advanced Logging provides capabilities to filter information relevant to specific purposes by creating multiple logs per request, with each log containing data relevant to the log's purpose. For example, Quality of Service (QoS) data in one log and Analytics data in another. All the post-processing required to distill the information using today's solutions is no longer needed.
Real-time logging. Advanced Logging can extend the platform to support real-time analytics, helping you to provide real-time reports to customers or work with partners to do the same.
Rich/Flexible logging. Advanced Logging does not assume what is important to you. Instead, it provides enough control to specify what fields are important to you, add additional fields with ease, and provide policies around log rollover and request filtering. These capabilities do not exist today. Now you can log any of the HTTP request/response headers, IIS server variables, and client-side fields with simple configuration through the management console.
Application Logging. Making use of the IIS configuration system, Advanced Logging provides capabilities to generate separate logs for sites, applications and so on.
Module Specific Log fields. If you are developing modules for IIS, you can specify custom fields that can be collected in logs.

You can learn more about the feature by reading the walkthroughs:

Getting started with Advanced Logging
Real-Time Logging
Client Logging
Learn more about the feature on - http://www.iis.net/extensions/AdvancedLogging

We eagerly await your feedback and comments on the Advanced Logging Discussion Forum.

Vishal Sood : Advanced Logging

Advanced Logging – How does Real-time logging work?

IIS Advanced Logging Beta is now Live