My Web Site is so slow… And I don't know what to do about it!
TechEd Northamerica started today. It's huge - more than 11,000 attendees from what I hear. One of my talks, titled "My Web Site is so slow…and I don't know what to do about it", happened this afternoon (here the PPTX).
I had so much information plus eight demos that I ended exactly 6 seconds before they cut me off. I promised the audience to post all the resources and tools I used to come up with this talk on my blog. So I'm sitting here in the Sheraton Canal Street lobby, enjoy free WIFI, a pint of beer assembling all the information I collected on web performance over the last year.
Before I start listing Best Practices for specific topics I need to call out the three books I found the most useful:
Books
1) The two Steve Souders books "High Performance Web Sites" and "Even Faster Web Sites" are probably a must-have for everybody concerned about web performance.
Unfortunately Souders is thin when it comes to IIS; ASP.NET isn't mentioned once. Reason enough to give this TechEd talk.
2) Only recently I discovered "Ultra-Fast ASP.NET" by Richard Kiessig. The title wasn't too appealing but I browsed through it anyway. And "Eureka" - this is a great book. As a Program Manager in IIS for almost 10 years I'm used to authors who know how to write but have no idea what they are writing about. Not so with Richard Kiessig's book. It's very thorough, everything is well-reasoned and explained. I learned quite a few things about IIS and ASP.NET that I wasn't aware of.
General Web Performance Tools
Well, the obvious ones:
Fiddler - Eric Lawrence's Web Debugging Proxy is tremendously to understand what's going on between a web browser and the web server
YSlow - for a quick look on how a particular web-site is doing with regards to performance.
PageSpeed - Google's answer to YSlow.
WCAT - WCAT is the tool you want if you want your web server to break a sweat. And if you are like I am and too lazy to write WCAT scripts you probably want the WCAT Extension for Fiddler. With it you can select a few Fiddler sessions and the WCAT Extension for Fiddler will create and run the WCAT script for you - all automatically.
Reference web sites
Yahoo's exceptional performance team
Steve Souders site and Blog Feed.
There are probably more, but I'm at Marguerites now, its getting late and there is still a lot to write.
Improving Web Server Performance
I structured the talk in five parts. First I explained why performance is important, then I went through some terminology and discussed the areas where web performance can be improved. Then I went through every performance area and discussed Best Practices. Here are the three areas:
- Speeding up Server Execution
- Optimizing Network Transfer
- Frontend Optimizations
Best Practices
Here are the Best Practices discussed. As already said it in the talk: web performance is a huge area and one could probably talk for days about Performance Best Practices. I picked the ones I thought have the most impact for my TechEd attendees.
Server Execution Best Practices
The Server Execution section was about doing more, faster with fewer resources. This is the area where IIS really shines. No other general purpose web server is capable of sustaining the throughput IIS/ASP.Net are able to handle. I think however that by focusing too much on Server Execution we neglected the other areas, i.e. network transfer and frontend optimizations. But be it as it is - here are the Server Execution Best Practices I discussed:
| Best Practice | Links/Resources/Comments |
| Run Windows Server 2008 (R2) on new hardware | I didn't find an article that explicitly discusses this. We ran into this issue however when Windows Server 2008 was in Beta. The Windows Server 2008 bits are optimized to take advantage of the features of new CPU architectures. You probably will see that Windows Server 2008 doesn't perform well on old hardware compared to Windows Server 2003. |
| IIS: Put existing default document on top of the Default Document list | The list of IIS default documents increased over the years (iisstart.htm, default.htm, default.asp, index.htm, index.php etc.). Problem is that IIS has to go through the list of default documents and look in the file system until it finds one that actually exists. You can increase the execution speed by stripping the list of default documents or putting the one you are using on top of the list. |
| IIS: Use IIS7 or ASP.NET output caching | There is a good tutorial on the ASP.NET web-site on how to take advantage of Output Caching. Ultra-Fast ASP.NET also has a chapter on it. IIS7 Output Caching is discussed in this Learn.IIS.Net article. |
| IIS: Use authentication and SSL only if necessary | Couldn't find something linkable but the problem is obvious: authentication can introduce a round-trip to the backend, e.g. Active Directory and encryption is CPU extensive. |
| IIS: Turn off unnecessary features like Request Tracing | Request and ETW Tracing is pretty lean and invaluable for troubleshooting problems. But if you are under a lot of load you should probably turn these features off. |
| PHP: Use FastCGI and not CGI | The IIS team is committed to make PHP run well on Windows. Providing a first class FastCGI implementation was the first step which made this a reality. The Web Platform Installer makes installing PHP on IIS a one-click experience. Everything else on http://php.iis.net |
| PHP: Enable Wincache | if you run PHP without an output cache you will quickly run into performance problems. Wincache is an Open Source output cache for PHP developed by the IIS team. The simplest way to install it: Web Platform Installer |
| PHP: Set FastCGI MaxInstances to 0 | FastCGI spawns multiple PHP processes for each web application. If you are in a hosted environment this might soon become an issue. Here is a great blog post on how MaxInstances can mitigate this problem. |
| ASP.NET: turn of debug="true" in web.config | Increased memory footprint and unoptimized code are only two of the problems you'll see if the debug flag is set to true. ScottGu blogged about this a while ago. |
| ASP.NET: Read through the ASP.NET Performance Coding Architecture Guides | Great book and also free. |
| ASP.NET: Read about the Large Object Heap | When I talked with the ASP.NET developers about this TechEd presentation they urged me to talk about the Large Object Heap (LOH). Every object larger than 85k goes into the LOH. The problem with the LOH is that the .NET Framework has a hard time to Garbage Collect LOH objects frequently enough in certain web scenarios, e.g. if you allocate a large object for every incoming HTTP request. Read more about the LOH here |
| ASP.Net: Use ASPNET_COMPILER to optimize for the first impression | ASP.NET applications can take a long time to start up. Due to human nature the first impression counts however. That's why the Web Platform Installer runs ASPNET_COMPILER at the end of an install of an ASP.NET application. ASPNET_COMPILER does a batch compile of all ASP.NET files and resources. This speeds up startup, especially on slow disks! More about ASPNET_COMPILER here. |
| ASP.NET: Review IIS idle timeout and proactive recycling settings | Given the considerable startup time of ASP.NET applications you might want to look at the IIS recycling and idle timeout settings. IIS recycles Application Pools proactively every 29 hours. It also times out Application Pools when the site is inactive. A proactive recycle might be initiated right when you have the most traffic on your site and an idle timeout might happen just because you removed your server from the web farm for a few minutes. If you can afford it you might want to turn off idle timeout and find some better recycling settings for your ASP.NET application. |
| ASP.NET: review Thread Pool settings | there is a setting in ASP.NET that might not work well for high throughput sites. It's the maxConcurrentRequestsPerCPU setting. It's set to 12 by default, i.e. not more than 12 concurrent requests per CPU will be executed. Usually this setting works well. We've seen instances however where a higher setting is necessary. Thomas has a great blog post on the thread settings in IIS7. |
That's it from the Server Execution side. Now on to Network throughput.
Optimizing Network Transfer Best Practices
| Best Practice | Links/Resources/Comments |
| Use Content Distribution Networks | Steve Souders book has a whole chapter about it. The only thing to add is that Microsoft hosts JQuery on a CDN for you - free! |
| Use compression | There are several great articles on compression on several blogs: Scott Forsyth's: IIS7 Compression. Good? Bad? How Much? Kanwal's "Changes to compression in IIS7" and the IIS compression configuration reference: httpCompression urlCompression |
| Minify Javascript and CSS | Enough said in Steve Souders book. Minification tools which work well on Windows: Microsoft Ajax Minifier 4.0 JSMin by Douglas Crockford |
| Use Doloto for splitting the initial Javascript payload | Very well explained on the Doloto web site. No reason for me to rehash. |
| Do not scale images in HTML | Steve Souders first book |
| Optimize Images | Steve Souders: Even Faster Web Sites |
| ASP.NET: Disable ViewState | Before you remove ViewState you should understand it. Here a good start. Once you are sure you don't need it - EnableViewState="false" is your friend. |
| Use the Cache-Control header to help the browser cache | IIS allows you to set the Cache-Control header via the UI. I wrote a blog entry on how to do it a couple of weeks ago. Here is also a command-line script if using the UI is too cumbersome: appcmd set config "Default Web Site/scripts" |
| Avoid Redirects. If you can't use permanent redirects (301's) instead of temporary redirects (302's). | A redirect introduces another network round-trip. Avoid it. Best described here or in Steve Souders first book.
|
| Combine Javascript and CSS | Same as row above. |
| Strip unnecessary response headers | In IIS it's easy to remove the X-Powered-By header via the UI. The ETag header is not as easy unfortunately. "Ultra-Fast ASP.NET" has an example how to remove the ETag in managed code. I have code how to remove the ETag in native code for all responses. I will post it later however. It's getting really late :) |
| Reduce DNS Lookups | DNS lookups introduce network roundtrips because the name has to be resolved and for that the browser needs to talk to a DNS server. Sometimes it's a good idea to use another one or two DNS names however. For static content for example. Static content doesn't need all the other stuff you might need for dynamic content, e.g. cookies. Steve Souders has a whole chapter about this topic. |
| Use authentication/SSL only when necessary | Do you really need authentication or SSL for images or scripts? Are they really worth protecting? Authentication can introduce additional round-trips because sometimes needs a challenge response authentication dance is needed to authenticate successfully. |
| Use the IIS7 bandwidth throttling module if your bandwidth is misused by somebody | Great example in "Ultra-Fast ASP.NET". You can throttle the bandwidth of a response based on the user, system load or any other criteria. Probably worth considering if you don't have unlimited bandwidth, especially if you have media content. Remember: On average only 20% of a video is usually watched by a user before he moves on to something else. If you already downloaded the full movie in this time you wasted a whole lot of bandwidth. |
| Review Cookies and their sizes | Cookies can get pretty big. Consider using the Cookie path feature if you can't reduce the size |
| ASP.NET: Keep Master Pages lean | Master Pages are part of every response. Don't bloat them. |
| Careful with ETags on IIS 6.0 | KB articles 922733 and 922703 explain why. |
| Use lowercase to reference your resources | Windows is not case-sensitive and it doesn't matter if you use lowercase, uppercase or a mix of both. The problem is the browser however. Because other OSs are case-sensitive the browser has to be case-sensitive as well. This means that it will cache multiple versions of a resource if you use different cases.
Another issue: resources compress better if lowercase is used. Why? Because most letters are lowercase. Using a smaller set of characters will produce more dictionary matches and compression is more effective. I read this is in "Ultra-Fast ASP.NET". Need to try this though! |
Frontend Optimizations
Ok, people are leaving the bar. Need to wrap this up. Let's go:
| Best Practice | Links/Resources/Comments |
| Scripts should go to the bottom of the page | Downloads of additional resources are blocked if a Javascript download is discovered. If Javascript is at the top of the page rendering of the page is probably not as fast as it could be. Responsiveness of the web application suffers. Here is the Steve Souders example page |
| Stylesheets on top | Rendering can only be done once the styles are downloaded. Steve Souders example page |
| Avoid CSS Expressions | Same here: Steve Souders example page |
And now good night from New Orleans!