Compression module provides IIS the capability to serve compressed responses to compression enabled clients. Clients which can accept compressed responses send Accept-Encoding header indicating compression schemes they can handle. If IIS can compress the response using a compression scheme which client can understand, IIS will send a compressed response with Content-Encoding response header indicating the scheme which was used to compress the response.
So request response look something like
REQUEST: **************
GET /static/index.htm HTTP/1.1\r\n
Accept-Encoding: gzip, deflate\r\n
Host: localhost\r\n
Accept: */*\r\n
\r\n
RESPONSE: **************
HTTP/1.1 200 OK\r\n
Content-Type: text/html\r\n
Content-Encoding: gzip\r\n
Last-Modified: Tue, 28 Mar 2006 03:44:06 GMT\r\n
Accept-Ranges: bytes\r\n
ETag: "087e0dc1952c61:0"\r\n
Vary: Accept-Encoding\r\n
Server: Microsoft-IIS/7.0\r\n
X-Powered-By: ASP.NET\r\n
Date: Tue, 13 Jun 2006 02:23:41 GMT\r\n
Content-Length: 438\r\n
This is useful to save network bandwidth and offer clients faster download times. This was first added in IIS6 and could be enabled for static/dynamic content individually. Following set of changes are made to compression modules in IIS7:
- Static compression is ON by default in IIS7. Dynamic compression is still OFF by default which can be turned ON for all content by running
appcmd set config -section:urlCompression /doDynamicCompression:true
- In IIS6, static compression was happening on a separate thread. So on receiving a request, first response was uncompressed and IIS used to start a separate thread to compress the file and keep it in compressed files cache. Requests for compressed content reaching IIS after the compression is complete used to receive compressed response. In IIS7, compression happens on the main thread. But to not incur cost of compression for all requests compression happens only for frequently requested content. Definition of frequently requested content is controlled by properties frequentHitThreshold and frequentHitTimePeriod under section system.webServer/serverRuntime. If IIS receives more than threshhold number of requests in frequentlyHitTimePeriod for the same url, IIS will go ahead and compress the file to serve compressed response for the same request which made IIS reach threshhold. This compressed response is saved in the compressed files cache as in IIS6. If compressed response was already present in compression cache, frequentHitThreshhold logic is not applied as compresed content will be picked from cache and there will be no additional cost for compressing the content. Hit count is maintained per URL. So sending first request with Accept-Encoding: gzip and second with deflate will still qualify as frequentlyHit content and IIS will go ahead and compress the response. This will require cachuri.dll to present in the globalModules section as that is the module which keeps URL hit count (More about globalModules someday later).
- Temporary compressed files folder has a nested directory structure in IIS7 compared to a flat one in IIS6. We create folder for each apppool in temporary compressed files and then create separate folders for different schemes under each apppool. Under these scheme folders, we create folder structure similar to folder from where the content was picked. So if iisstart.htm from D:\inetpub\wwwroot got compressed using gzip, cache entry will get created in
D:\inetpub\temp\IIS Temporary Compressed Files\DefaultAppPool\$^_gzip_D^\INETPUB\WWWROOT folder. We ACL the apppool folder with worker process identity to protect the content from worker processes serving some other apppools. Directory is still configurable from config but the default is moved from %windir%\iis temporary compressed files to %SystemDrive%\inetpub\temp\iis temporary compressed files. Also with this change, maxDiskSpaceUsage limit is applied per apppool. So if you had a value of 100MB for HcMaxDiskSpaceUsage in IIS6, that limit was applied to all the compressed content in the compressed files cache. Now this limit applies to compressed files per apppool. If you have 10 apppools and have maxDiskSpaceUsage set to 100MB, total space allocated to compressed files cache is actually 1GB.
- As static compression is enabled by default and compression is happening on the main thread, we added on the fly compression shutoff/resume depending on CPU load. Four properties are added to system.webServer/httpCompression section to control this behavior. These are
- staticCompressionDisableCpuUsage - compression is disabled when average CPU usage over a time period is above this number.
- staticCompressionEnableCpuUsage - compression is enabled back if average CPU usage over a time period falls below this number.
- dynamicCompressionDisableCpuUsage and dynamicCompressionEnableCpuUsage enables or disables dynamic compression depending on load of CPU.
We calculate average CPU utilization every 30 sec.
- In IIS7, you can enable/disable compression depending on the content-type of response. In IIS6 this was possible on extension basis which is no longer possible. Now you can have just one entry in the config to enable static or dynamic compression for text/html responses. You no longer need to pick up all extensions which return text/html responses. While configuring these mimeTypes under httpCompression section, you can use * as wildcard. If response type is text/html, we look for an entry for text/html and if found use the corresponding enabled value. If text/html is not found, we look for text/* or */html. If both are present we pick the one which comes first and use that enabled property value. If these are also not found, we look for */* and use the corresponding enabled value. For enabling compression for all content types, add an entry under httpCompression section in applicationHost.config which says
<staticTypes>
<add mimeType="*/*" enabled="true" />
</staticTypes>
- Bunch of IIS6 compression properties are removed in IIS7. These are
- HcMaxQueueLength. As compression is happening on main thread, queue length property is not used anymore and is removed.
- HcCreateFlags property which was used to pass a value to gzip.dll (indicating gzip or deflate compression) is removed as it was not useful for the customers.
- HcFileExtensions and HcScriptFileExtensions are removed as we no longer enable/disable compression based on extension.
- HcIoBufferSize and HcCompressionBufferSize properties are removed as we believe those are not useful for customers.
- HcDoOnDemandCompression is removed. For enabling static compression, only doStaticCompression property is required to be set.
- HcFilesDeletedPerDiskFree is no longer there in IIS7. We just delete as many files as required to get to 90% of the maxDiskSpaceUsage. We pick least recently used files to delete.
- HcPriority is removed as scheme to use when multiple ones appear in Accept-Encoding header of request is picked depending on scheme which is appear first in Accept-Encoding header (assuming no q factor). This is as per HTTP specs.
I think ability to add new schemes in config is not very useful. I have not seen any compression scheme implementation outside Microsoft which can be used with IIS. If you want to implement your own scheme, you need to write a dll with following exports
- Compress
- CreateCompression
- DeInitCompression
- DestroyCompression
- InitCompression
- ResetCompression
If you want signatures of these functions, write to me and i can take your case to the management. Most probably you will get the signatures very easily.
- maxDiskSpaceUsage entry is changed to be configured in MB rather than bytes. We realized that people don't really want to configure the limit to the byte level but the main reason was we have the limit as UINT and we didn't want customers to set it to a value which cannot be stored in UINT. With large disks today, having a large value won't be uncommon.
- With static compression enabled by default, now we only cache compressed responses in the kernel (HTTP.Sys). So if compression is enabled for a file but the current request didn't contain Accept-Encoding header (or compression didn't happen because it was the first request), we don't tell http.sys to cache it. Only compressed response is cached in the kernel for which compression is enabled. Dynamically compressed responses are not cached in any of the caches (even in compressed files as was in IIS6).
- Deflate is removed in default configuration but the functionality is still present in gzip.dll. To add deflate scheme, add the following in httpCompression section:
<scheme name="deflate" dll="%Windir%\system32\inetsrv\gzip.dll" />
You can use following appcmd command.
appcmd set config /section:httpCompression /+[name='deflate',dll='%Windir%\system32\inetsrv\gzip.dll']
- As static compression is enabled by default, default value of staticCompressionLevel is changed from 10 to 7.
Most of the compression properties (present under system.webServer/httpCompression) can be set in applicationHost.config for the server and cannot be configured on URL basis. We might have per apppool properties someday and if we end up doing per apppool properties, properties under httpCompression will be configurable per apppool. doStaticCompression and doDynamicCompression boolean properties under system.webServer/urlCompression section can be set at URL level which can be used to enable/disable compression per URL. This is similar to what was in IIS6.
I think i have mentioned all the changes that are coming in IIS7 as far as compression is concerned. If i realize that i missed something, i will do another post.
-Kanwal
Comments