RFC 7151 - File Transfer Protocol HOST Command for Virtual Hosts

I received an email yesterday from the RFC Editor that a new Request for Comments (RFC) document has just been published, RFC 7151, which adds support for a new "HOST" command to FTP. This new command allows hosting multiple FTP sites on a single IP address, much like what Host Headers provide for HTTP.

Here's the URL to the new RFC on the RFC Editor website:

File Transfer Protocol HOST Command for Virtual Hosts
http://www.ietf.org/rfc/rfc7151.txt

Or you can see the HTML-based version at the following URL:

http://tools.ietf.org/html/rfc7151

One minor point which I would like to clarify is that this adds a new command for FTP to specify which virtual host to connect to. I periodically hear people referring to this as "FTP Host Headers", but that is technically incorrect since FTP does not have request headers like HTTP. Here's a simple example of what the communications flow looks like when the HOST command is used:

I need to make sure that I thank my co-author for this RFC, Paul Hethmon, who has authored other FTP-related RFCs over the years. For example, Paul wrote RFC 3659, and he co-wrote RFC 2389 with Robert Elz. As a result, the Internet community has Paul and Robert to thank for several great FTP command extensions in the past. (e.g. FEAT, OPTS, MDTM, SIZE, REST, MLST, MLSD, etc.) Paul and I co-wrote RFC 7151 over the past several years, and it was great working with him.

Support for the HOST command has been built-in to Microsoft's FTP service since IIS 7.0, but now that the RFC has been officially published I hope that this feature will be adopted by other FTP servers and clients. That being said, IIS is not the only implementation of the FTP HOST command; at the time of this blog post, these are the server and client implementations that I am aware of which already provide support for this new command. (Note: there may be more than I have listed here; these are just the implementations that I currently know about.)

• FTP Servers:
  • FTP Daemon (see http://www.pro-bono-publico.de/projects/ftpd.html)
  • Ipswitch (in the WS_FTP service; see http://www.ipswitchft.com/)
  • Microsoft (in the IIS FTP service; see http://www.iis.net/)
  • RhinoSoft (in the Serv-U File server; see http://www.serv-u.com/)
• FTP Clients:
  • Microsoft (in the Visual Studio 2010/2012/2013 and Expression Web 4 products)
  • Rebex (in the Rebex FTP/SSL for .NET library; see http://www.rebex.net/)
  • RhinoSoft (in the FTP Voyager client; see http://www.rhinosoft.com/)
  • Scooter Software (in the Beyond Compare client; see http://scootersoftware.com/)
  • SmartFTP (in the SmartFTP client; see http://smartftp.com/)

In addition to the clients listed above, if you have been reading my series on FTP clients over the past few years, I have posted details on how to use the FTP HOST command with some other FTP clients which do not provide built-in support. For example, the Core FTP Client allows you to specific pre-login commands as part of an FTP site's connection properties, so you can manually type in the HOST command and save it along the site's settings.

A Little Bit of History

When I joined the feature team which was creating the FTP service for Windows Server 2008, one of the things that bothered us was that there was no way at the protocol level to host multiple FTP sites on the same IP address. There were several ways that FTP server implementations were approximating that sort of functionality, for example the User Isolation features that we ship with FTP for IIS, but each FTP server seemed to be implementing its own workaround and there was no standardization.

Because of this limitation, our team received a lot of requests to add "FTP Host Headers," although as I explained earlier FTP has no concept of request headers. To help address some of the questions which I was often seeing, I explained the lack of hostname support for FTP in detail in the comments section of my FTP User Isolation with Multiple User Accounts blog that I posted back in 2006, which was shortly before we began work on implementing the HOST command. I will paraphrase some of my comments here:

While I realize that the ability host multiple FTP sites on the same IP address and port like HTTP is a desired configuration, the simple answer is that FTP does not currently support this at the protocol level. To put things in perspective, RFC 959 is the governing document for FTP, and that was published in October of 1985. FTP was simply not designed for the Internet as we use and understand it today, even though it is a generally reliable protocol that many people will continue to use for some time. HTTP/1.1 was designed much later and resolved this problem, but only for HTTP requests.

There are three ways that you can create unique bindings for a web or HTTP site: IP address, port, or host header. FTP can create unique bindings by IP address or port, but the FTP protocol does not currently provide support for hostnames.

Here's why: HTTP packets consist of a set of request headers and possibly a block of data. Here's an example of a simple GET request:

GET /default.aspx HTTP/1.0 [CrLf]
Accept: */* [CrLf]
[CrLf]

When HTTP 1.1 was published in RFC 2068 and RFC 2616, it defined a header for specifying a "host" name in a separate name/value pair:

GET /default.aspx HTTP/1.1 [CrLf]
Host: example.com [CrLf]
Accept: */* [CrLf]
[CrLf]

The "Host" header allows multiple HTTP virtual servers ("hosts") on the same IP address and port that are differentiated by host name. While this works great for the HTTP protocol, FTP currently has no comparable functionality. As such, the FTP protocol would have to be updated to allow multiple hosts on the same IP address and port, then FTP servers and clients would need to be updated to accommodate the changes to FTP.

While my explanation may have clarified root cause of the FTP limitation for anyone who was asking about it, I personally thought the situation was unacceptable. This inspired me to research the addition of a new command for FTP which would allow FTP clients to specify hostnames. As I was researching how to propose a new RFC document to the IETF, I discovered that Paul Hethmon had been researching the same problem a few years earlier. I contacted Paul and offered to combine our work, and he agreed. After several years of work and a great deal of supportive assistance from dozens of great people whom I met through the IETF, RFC 7151 has finally been published.

There are a lot of people besides Paul whom I should thank, and we mention them in the acknowledgments section of our RFC, which you can read at the following URL:

http://tools.ietf.org/html/rfc7151#appendix-B

One final note - two of my coworkers, Jaroslav Dunajsky and Wade Hilmo, are mentioned in the acknowledgments section of the RFC. Jaroslav is the developer who implemented the FTP HOST command for IIS, and Wade is a senior developer on the IIS team who graciously allowed me to bounce ideas off him while I was doing my research over the past few years. (I probably I owe him a lunch or two.)