Jimin Gao's Blog

Concepts of IIS Bit Rate Throttling – Part 2

2008-05-02T01:22:00Z

Managing Service Quality

There are many situations under which we would like to impose limits to file serving speed and the number of clients served. For example, a large rogue media file with a huge (faked) bit rate may induce unfair server bandwidth contention with legitimate media files of small bit rates and cause their serving quality to deteriorate; similarly serving too many requests simultaneously for certain popular media files can also cause server saturation that defeats the purpose of progressive download—nobody can have a smooth viewing experience. The Bit Rate Throttling settings maximum throttle rate and connection limit are designed for better management of such undesirable scenarios. In addition, a default throttle rate can be applied to all files at a certain location, being it a site, virtual directory, or physical folder, so that these files, which may be of too many different types to have a throttling setting for each, can all be served at a reduced constant speed when no explicit throttling settings apply.

Maximum Throttle Rate

The maximum throttle rate is the largest permitted speed to serve a throttled file. For example, if the maximum throttle rate is set to 500kbps the previous FLV file configured to be throttled at 363kbps would still be served at 363kbps, while with a maximum throttle rate of 300kpbs (smaller than the calculated throttle rate) the file will be served at 300kbps. The same logic also applies to data file throttling. Note however the maximum throttle rate only affects files to be throttled (by file-type throttle settings or default throttle rate settings) and those without applicable throttle settings will still be sent at full speed regardless.

The maximum throttle rate can be used by server administrator to impose restrictions on different sites to ensure fairness and serving quality when used in combination with connection limit. Typically it should be set and locked in applicationHost.config. The maximum throttle rate can be configured through Bit Rate Throttling GUI (in Advanced Settings dialogue), as demonstrated in this walkthrough, or by editing IIS configuration files.

Connection Limit

The maximum number of simultaneous HTTP connections, or the connection limit, can be set for a collection of file types in an IIS site using the Bit Rate Throttling Module. For example, in a site we may define all FLV, AVI, ASF, and WMV files as a connection limit group called Video, and MP3 and WMA files as another called Audio. For each defined connection limit group we can set a connection limit, for example, 100 for Video and 200 for Audio. Then the maximum number of simultaneous downloads for all files belonging to Video will be 100, and the 101th request will be rejected with HTTP status code 503 (Service Unavailable). Similarly the 201th request for Audio files will be rejected. Notably different from other Bit Rate Throttling configuration settings, the connection limit settings and restrictions are site based, and therefore

A file may belong to different connection limit groups for different sites.
Connection limits can be different for different sites.
Any of these settings at levels lower than site are ignored.
The current number of connections for a connection group is aggregated for each site and the connection limit restrictions are applied to each site independently.

Multiplying maximum throttle rate with the total connection limit of a site can give us the peak bandwidth usage for all the regulated files in this site. So used together these settings limit the bandwidth usage for different sites, especially of large media and data files to balance the service quality of different types of contents. For example, we may set the maximum throttle rate and connection limits for video files for every site, ensuring that their total peak usage will not exceed our server bandwidth so that every (served) user's experience can be guaranteed. We may also set these numbers to ensure the serving of HTML web pages is not disrupted by the bandwidth usage of media contents. The connection limit groups and connection limits can only be set by editing IIS configuration files and are not accessible for GUI. See Jack's blog for how this can be done. It is recommended they are configured and locked in applicationHost.config only.

(Note: there is an issue in the current release that connection limits are at times not correctly enforced. This is to be addressed by a forthcoming patch.)

Default Throttle Rate

The default throttle rate acts as the fallback throttle rate when no explicit throttle settings can be applied to calculate the throttle rate of a file. When set to 0, default throttle rate is disabled and files without applicable throttle settings will be sent at full speed at HTTP responses. The default throttle rate is a convenience feature to manage the send rate of all file types in a particular location, which may be a site, a virtual directory, or a physical folder (through web.config), so that we do not have to add a throttle settings for each type. By purposed arrangement of files into different locations, the default throttle rate to an extent can be used for managing total bandwidth usage and service quality as well.

Dynamic Throttling

The maximum throttle rate and connection limits are typically used for setting a bandwidth allowance for a site, but this peak estimate is rarely used in full; files usually are throttled at a rate much lower than the maximum throttle rate, and connection limits are not reached most of the time. Dynamic Throttling is a Bit Rate Throttling feature designed to make better usage of the reserved bandwidth for a site or connection limit group. The per-site setting target throughput rate explicitly states the total bandwidth allowance for regulated files in a site, and when under light load the Bit Rate Throttling Module will scale up the throttle rates to make maximal usage of this allowance. The throttle rates are adjusted up and down on-the-fly depending on the throttle rate presets and the number of connections for the files but never lower than the presets. For example, there is a file A with the throttle rate (calculated from configuration) of 400kbps and a file B with the throttle rate of 100kbps belonging to the same site with its target throughput rate set to 5000kbps. Their actual throttle rates will be scaled up to 4000kbps and 1000kbps so the total bandwidth is equal to the target throughput rate 5000kbps when A and B are simultaneously served. Now suppose another request for A comes in. Serving this additional request, the Bit Rate Throttling Module will adjust down the throttle rates proportionately. File A will then be sent at 500 * 5000 / (500 * 2 + 100) = 2273kbps and B at 100 * 5000 / (500 * 2 + 100) = 455kbps; the total bandwidth usage for all 3 connections is still 5000kbps. When one of the responses is finished, the rest will be again adjusted up. Note that the throttle rate will never be adjusted to lower than the configured rate, 500kbps for A and 100kbps for B in this case, even when the target throughput rate is exceeded due to a large number of connections. Un-throttled files will not be affected by dynamic throttling.

In addition to sites, the target throughput rate can be configured for connection limit groups as well with similar logic (note connection limit group itself is a site-level only setting). The configuration can be done from the GUI (sites only) or by editing IIS configuration files. See Vishal's blog for a more thorough description of this feature.

(To be continued …)

Concepts of IIS Bit Rate Throttling – Part 1

2008-04-23T06:29:00Z

Bit Rate Throttling is the first IIS extension module released in IIS Media Pack. It adds to IIS the capability of per-response bandwidth control for the delivery of media and data contents. The idea for this module initially grew out of the need to serve progressive media downloads while saving bandwidth costs. Its core functionality was to automatically detect the encoded bit-rate of files of a wide range of video and audio formats and serve the requests at a ratio to the encoded bit-rate as specified by the administrator. However, during its development we saw the needs of more general bandwidth control capabilities that can be logically combined with its media functionalities and over time Bit Rate Throttling Module has evolved into a full-rounded bandwidth control package for IIS.

There are several blog articles describing in detail aspects of Bit Rate Throttling. Here however I would like to give an overview of the capabilities of this module, not too drawn into the technical details but with enough information for a clear understanding of its capabilities so that users looking for solutions to their problems can make informed decisions. This article also tries to serve as the string for the beads and will provide links to previously available articles within this context, so that those interested in more hands-on details can easily locate the needed resources.

The basic promise of Bit Rate Throttling is the measured slowing down of sending out a HTTP response. The actual send speed can be controlled in various ways for various purposes and the most notable one is the automatic throttling of media files.

Intelligent Media File Throttling

Bit Rate Throttling for IIS can automatically detect the encoded bit rate of a media file and send the file at an administrator-configured speed ratio to this bit rate. For example, if an ASF video file has a video data rate of 300kbps (kilobits per second) and audio data rate of 30kbps, and ASF files are configure to be sent at the speed ratio (throttle rate percentage) of 110%, this file will be sent at the speed of (300kbps + 30kbps) * 110% = 363kbps. This speed is sufficient for uninterrupted viewing of the video on the client side assuming ample network bandwidth between the server and the client. However, to factor in possible network fluctuations most media players buffer some of the content before starting to play. Correspondingly Bit Rate Throttling has a configurable fast start time for each media type such that initial media content within this (play) time period will be sent without throttling so that the player buffer can be quickly filled before playing. For example, if ASF files are configured to have a fast start time of 10 seconds, then the initial (300kbps + 30kbps) * 10s = 3300kb of the file above will be sent at full speed and the rest will then be sent at 363kbps.

Supported Media Formats

The types of media files supported by Bit Rate Throttling upon installation includes ASF, WMV, WMA, FLV, MP3, MP4, M4V, MOV, AVI, RM, and RMVB. The throttle rate percentage and fast start time can be configured per file type through Bit Rate Throttling GUI or by editing IIS configuration files. See this walkthrough for how this can be done.

Extended Media Format Support

It is also possible for Bit Rate Throttling to support additional media formats, assuming the format specifications are known and have the following characteristics so that their encoded bit rates can be obtained using a simple search:

1. Bit rate information exists in the media file at a fixed offset from a certain keyword (search pattern), OR

2. Content duration information exists in the media file at a fixed offset from a certain keyword (search pattern).

Most known media file formats satisfy these properties and therefore can be supported by Bit Rate Throttling. In essence, the Bit Rate Throttling Module will first find the search pattern (keyword) in the file and then read the bit rate or duration information at the offset. The bit rate or duration information may be encoded in various formats, such as big or little-endian and float or integer. If there are multiple bit rates available, such as in the previous example where the ASF file has a video bit rate of 300kbps and an audio bit rate of 30kbps, they are added together to obtain the total bit rate 330kbps. If there is only duration information available, it is used to divide the file size to obtain the estimated bit rate.

The media format information can be added to by editing the IIS configuration file and this functionality is not accessible from GUI. See this walkthrough for how this can be done. It is worth noticing that most out-of-the-box media type supports are attained though the same mechanism except for MP3 and AVI files which have dedicated file parsers due to their complexity. Once a file type is added to media formats in IIS configuration files, the media format becomes known to Bit Rate Throttling and their throttle rate percentage and fast start time can be configured just like natively supported media files.

Data File Throttling

Files not treated (known to Bit Rate Throttling Module) as video/audio media can be sent at a fixed rate as HTTP responses using Bit Rate Throttling. They are called data files and similarly to media files their throttle rate can be configured per file type (extension). However the rate is not specified as a percentage ratio but absolute send speed in kbps. There is an also a configurable initial send size (in kilobytes) for each data file type corresponding to the fast start time of media files.

The throttle rate and initial send size for data files can be configured through Bit Rate Throttling GUI or by editing IIS configuration files. The configuration entries for media and data files are distinguished by their throttle types, which may be media or data. This explicit distinction is necessary because a media type known to Bit Rate Throttling can nonetheless be treated as either media or data files for throttling purpose. See this walkthrough for detailed steps to configure data file throttling.

The throttling of data files can be useful for the distribution of large documents such as PDF and PPT files which similarly to media files are inherently serial but do not have the notion of unit-time bit rates.

(Continue to Part 2)