WCAT Fiddler Extension for Web Server Performance Tests

For web server performance testing the Microsoft Web Capacity Analysis Tool (WCAT) is the tool of choice of the IIS team as well as the Windows Performance Team. Simple tests that hammer IIS with thousands of requests to the same HTTP page to very sophisticated web hosting and benchmark scenarios - there is nothing that WCAT can't do. Even on ApacheCon I heard some hard-core Apache guys talking about using WCAT for their performance tests.

WCAT is extremely versatile and can simulate thousands of concurrent users making requests to a single web site or multiple web sites.  The WCAT engine uses a simple script to define the set of HTTP requests to be played back to the web server. Extensibility is provided through plug-in DLLs and a simple API. WCAT is the best tool to hammer your web server with requests and make it really sweat.

Writing a WCAT script is not rocket science. However, writing a WCAT script to replay client requests is a lot of work because all hyperlinks and other resources have to be added manually. Lot's of typing… So I thought I write a Fiddler extension that does the work (download here). To use it is simple. Selecting a set of requests in the Fiddler UI and clicking the "Run WCAT script for selected sessions" in the context menu will do the trick. The WCAT Fiddler extension will then generate a WCAT script from the selected sessions and run the selected requests as a WCAT transaction against the configured server. Once the test is over it will show the results using the built-in reporting page.

WCAT Fiddler Extension Details

After you installed the WCAT Fiddler Extension you get two additional menu items in Fiddler. The first one is the "Configure WCAT Run" menu item in the "Tools" menu.

In the configuration dialog below you can configure the following:

1. Server Name
   This might look a little weird given that the server name could be inferred from the host name used in the selected sessions. WCAT is a server performance tool though and for this reason it will run the selected sessions only against the server that is specified in this dialog. The WCAT Fiddler extension uses localhost as the default. So even if sessions get selected that Internet Explorer requested from a remote web server, e.g. www.microsoft.com, WCAT will establish a connection with the WCAT-specified server and run the www.microsoft.com requests against it (i.e. the host header will say www.microsoft.com but the server that receives the request is the local machine. WCAT will also not use the Internet Explorer HTTP interfaces to make its requests. It's using its own infrastructure. WCAT requests won't show up in Fiddler nor can a proxy server be used with WCAT.
2. Virtual Clients
   WCAT has a controller/client architecture that allows multiple physical machines (clients) to team up and send requests against a single web server. For the WCAT Fiddler Extension this feature is not used. The WCAT Configuration dialog allows the configuration of virtual clients however. Virtual Clients are the number of concurrent threads that the WCAT Client will use to send request against the web server. One thing to remember: the default configuration uses the local machine as the WCAT client as well as the WCAT server. Both need resources (CPU/memory) and performance tests will definitely be skewed by running client and server on the same machine.
3. Duration
   The duration of the test. Default is 30 seconds.
4. Warm-up time
   The time WCAT gives the web server to warm-up its caches. After the warm-up time will release the throttle and send everything it has against the web server.

Run WCAT script for selected sessions

The second menu item is available in Fiddler's context menu.

The following happens if the menu "Run WCAT script for selected sessions" is clicked:

1. The code checks if WCAT is installed. It show message if not and returns.
2. It iterates through all selected sessions and writes a WCAT script file to the temp directory (scripts can be found via "dir %temp%\*.wcat).
3. It starts a WCAT controller and a WCAT client which executes the WCAT script using the parameters configured in the WCAT Configuration dialog.
    
4. After the test completes the results are displayed in Internet Explorer (results file can be found via "dir %temp%\*.wcat.xml").

Results

There is a lot of interesting data in the WCAT results page. The two most important are shown in the summary. Requests/sec is pretty obvious. "Transactions per Second" is not hard to understand either. The generated WCAT script puts the selected Fiddler sessions in a so-called transaction. WCAT executes all requests of a given transaction before it starts a new one. So "Transactions per Second" is the number of these selected Fiddler sessions that were handled per second. These two results give a rough ballpark figure how fast your web server runs.

The results have to be taken with a grain of salt. There are many things to look at before these numbers will reflect the maximum of what your web server can handle. For example:

• the WCAT client will consume CPU cycles if you run the WCAT client on the same machine as your web server
• the network might be maxed out and not be able to send more data
• the virtual clients you configured might not be able to generate enough traffic for the web server

I will follow up with another post discussing how to properly test web server performance.

Known Issues:

• To show WCAT Reports in Internet Explorer WCAT's reports.XSL File is needed. Unfortunately it needs to run scripts and shows the following dialog:
   
  The dialog sometimes ends up in the background and IE appears to hang.
  SOLUTION: Just TAB through your windows and click "OK" once you find the dialog.
• Currently only GET requests are supported by the WCAT Fiddler Extension.
• I'm pretty new to writing installers. The MSI installer puts the WCAT Fiddler Extension in the c:\Users\<username>\My Documents\fiddler2\scripts directory. No messages, no dialogs. It just does it.