Microsoft Web Deployment Team Blog

How to write skip and replace rules for MSDeploy

2008-05-23T19:43:00Z

MSDeploy has two built-in general purpose rule handlers (for more information about rules, take a look at the core components of MS Deploy blog post) which can be used to create rules to skip sync/migrate of some object or to replace the source information with something else on the fly. You can create and add new rules using these general purpose rule handlers in the configuration file (Microsoft.Web.Deployment.Config) or you can use –skip or –replace switches on the command line.

If you add a rule in the configuration file, you can either set isDefault attribute to true in which case the rule is enabled by default. You will need –disableRule switch on the command line to selectively disable it. Alternatively, you can keep isDefault=false and use –enableRule switch to selectively enable it.

When creating rules using –skip, -replace switches on the command line, you can specify more than one -skip/-replace switch to add multiple skip/replace rules. Before moving on to learning how to write these rules, let’s understand how each object that will be added/updated/deleted on the destination is represented internally and understand the terminology that will be useful later. As an example, let’s use an XML view of few source entries that are generated by the command below.

msdeploy.exe -verb:dump "-source:appHostConfig=DefaultWeb Site/abc" -xml

<appHostConfig path="Default Web Site/abc">
    <application path="/abc"
                 applicationPool="DefaultAppPool"
                 enabledProtocols="http">
        <virtualDirectoryDefaults path=""
                                  physicalPath=""
                                  userName=""
                                  password=""
                                  logonMethod="ClearText"
                                  allowSubDirConfig="true" />
        <virtualDirectory path="/"
                          physicalPath="d:\Compiler"
                          userName=""
                          password=""
                          logonMethod="ClearText"
                          allowSubDirConfig="true">
            <dirPath path="d:\Compiler" />
        </virtualDirectory>
    </application>
</appHostConfig>

Text in brown is considered the objectName, which is usually the provider name but can be something else when object are Xml/Config files. Red text is attribute names, which will be required when defining scopeAttributeName, targetAttributeName in replace rules. Blue entries are the attribute value.

Most providers entries have a “path” or “name” associated with them to uniquely identify the object from a collection of objects of same type which is what is referred to as “absolutePath” in skip rules below. You can use the xml dump to find out the values of objectNames, absolutePaths, attribute names/values, etc. Below is how you use this information to write skip/replace rules.

1. Skip Rule
You can create a skip rule to skip an entry based on objectName, absolutePath, skipAction or xPath. Each entry in the source XML view is matched with available information in the skip rule and is not considered for sync/migrate if a match is found. Below is how available information is interpreted in the SkipRuleHandler.

è objectName – optional case-insensitive regular expression. If specified, entries with matching objectName are considered for skip. If not specified, all elements are considered for skip.
è absolutePath – optional case-insensitive regular expressions and are matched against absolute path of each element. If present, entries with absolutePath matching this value are considered. If not specified, all are considered.
è skipAction – optional value to specify what operations to be considered for skip. This can be source or destination or update or delete or addchild. Default is set to source or destination which means that skip all operations on source and destination which match objectName, absolutePath, xPath criteria.
è xPath – optional xPath expression for entry which should be skipped. You can use a non-xml dump to see xPaths of various entries.

Here are some examples of skip rules which can be added to Microsoft.Web.Deployment.Config.
<rule name="SkipAllOperationsOnDefaultAppPool"
    type="Microsoft.Web.Deployment.SkipRuleHandler" objectName="add"
    absolutePath="DefaultAppPool" isDefault="true" />
<rule name="DoNotDeleteFilesOnDestination"
    type="Microsoft.Web.Deployment.SkipRuleHandler" objectName="filePath"
    skipAction="Delete" isDefault="true" />
<rule name="SkipAllRegistryOperations"
    type="Microsoft.Web.Deployment.SkipRuleHandler" objectName="^reg"
    isDefault="true" />
<rule name="SkipRegistryKeysUnderInetInfo"
    type="Microsoft.Web.Deployment.SkipRuleHandler" objectName="regValue"
    absolutePath="inetinfo" isDefault="true" />

DoNotDeleteFilesOnDestination will look like below if specified on the command line.
-skip:objectName=filepath,skipAction=delete

2. Replace rule
Replace rules require following information based on which entry is considered for replacement.

è objectName – optional case insensitive regular expression. If specified, only these entries will be considered for replace. If not specified, all entries are considered.
è scopeAttributeName – optional case insensitive regular expression. If present, entries which have an attribute matching this regular expression are picked to be matched with scopeAttributeValue. If not present, all attributes are considered for matching with scopeAttributeValue.
è scopeAttributeValue – optional case insensitive regular expression. If present, scopeAttributeName attribute value is matched with this. If an attribute with expected value is found, this entry is considered for replace. If not present, no filtering is done and all entries are considered for replace.
è targetAttributeName – optional case insensitive regular expression. If present, value of this attribute is matched with “match” value. If not present, all attribute values are matched with “match”. These are the attributes which are replaced.
è match – optional case insensitive regular expression. If present, entries with attributes whose values match this regular expression are considered for replace. If not present, all attribute values are replaced.
è replace – Required and is not interpreted as a regular expression. This is the new value of optional target attributes which match optional match. If both targetAttributeName and match are not defined, all attributes of entries which are picked for replace will be replaced by this value. Replace is done only on part which is matched in the value by regular expression.

Few examples of replace rules which can be added to config are below.
<rule name="ReplaceIPAddressInBinding"
    type="Microsoft.Web.Deployment.ReplaceRuleHandler" isDefault="true"
    objectName=”binding” targetAttributeName=”bindingInformation”
    match=”x\.x\.x\.x” replace=”y.y.y.y” />
<rule name="ReplaceJpegWithJpgInFiles"
    type="Microsoft.Web.Deployment.ReplaceRuleHandler" isDefault="true"
    objectName=”filePath” targetAttributeName=”path”
    match=”\.jpeg$” replace=”.jpg” />
<rule name="SetAllowSubDirConfigToFalseForImageFoldersInIIS7"
    type="Microsoft.Web.Deployment.ReplaceRuleHandler" isDefault="true"
    objectName=”virtualdirectory” scopeAttributeName=”physicalPath” scopeAttributeValue=”Image”
    targetAttributeName=”allowSubDirConfig” match=”true” replace=”false” />

On command line, ReplaceIPAddressInBinding rule will look like following.
-replace:objectName=binding,targetAttributeName=bindingInformation,
match=x\.x\.x\.x,replace=y.y.y.y

Read this for help with regular expressions.

Thanks,

Kanwal, Faith, Yamini

Web Deployment Tool Beta 1 (Go Live) just released

2008-05-13T21:58:00Z

Today the Web Deployment Team released the Beta 1 version of the deployment tool! If you're not familiar with the tool already, it is here to help you keep sites or servers in sync with IIS6 or IIS7, as well as migrate from IIS6 to IIS7. One of the key things we've worked on is the flexibility of the underlying framework, which you can see in the number of rules, the ability to configure dependencies and especially, the different providers that we support.

You can sync or migrate not only the configuration of a web site, but also it's physical content and any SSL certificates. We also allow you to create your own manifest and define an application the way that you choose. Does your application depend on an assembly in the GAC? Or a registry key? These can be added to the manifest.

Okay, enough about the tool, for everyone who has been waiting patiently for this latest release and given us excellent feedback in the forums, THANK YOU! Please give the tool a try and let us know what you think.

Listed below are the links for the download pages for each of the individual installation packages:

32-bit Installation Package:
- Microsoft Web Deployment Tool (x86)
64-bit Installation Package:
- Microsoft Web Deployment Tool (x64)

We've loaded this version with many great new features such as:

PowerShell Support - We have PowerShell cmdlets so that you can integrate MS Deploy commands with PowerShell directly.
Enhanced Dependency Checking - We have IIS7 dependency information listed, plus the ability to see where a dependency is being triggered from. For example, if you have a dependency on Windows Authentication, you can now determine where this is set in the configuration.
Detailed Help File - We have a Help chm file included in the tool so that you can browse through all the functionality and flexibility offered by the tool, instead of looking through online walkthroughs.

And much more. :)

Please don't forget to check out the latest release notes available online at http://www.iis.net/Downloads/files/MSDeploy/beta1/msdeploy_readme.html for important known issues!

Thanks again, and many happy deployments,
-Faith Allington, Program Manager, and the entire Web Deployment team

Microsoft.com uses the Web Deployment Tool

2008-04-14T23:20:00Z

The folks at Microsoft.com used the Web Deployment Tool to perform a migration of an IIS 6.0 server to IIS 7.0. They published a whitepaper describing their experiences with Tech Preview 1, available at their new operations site.

http://technet.microsoft.com/en-us/mscomops/cc424869.aspx

BTW, the new operations site also happens to have some interesting articles about other IIS 7.0 functionality, such as how they are using delegated configuration and streamlining their installations. I think we can expect to see more whitepapers in the future.

Enjoy!

Thanks,
-faith

What is the long-term support for the MS Deploy tool?

2008-02-27T04:26:00Z

In the past week or so, we've gotten quite a few questions about our long-term support of the tool. Yes, the tool will be fully supported by Microsoft Product Support at RTW (release to web). Currently it is in Tech Preview so the only support comes from the forums, but the team is pretty active on the forums as we're getting questions.

Also, we've gotten some great feedback from folks testing the tool and really appreciate it. Some of the requests have included the ability to ignore files whose only changes are attributes (like write time, etc.). Some of the good bugs found have been with script maps and dependency checks. We're spending a lot of time to improve these areas, this has come directly from customers trying it out.

Keep that feedback coming. This is the best time to request changes and influence our first version. We really want to know if the providers that we have are a good set, that we haven't missed any critical ones and we'd love to get some customers trying it on more complex applications.

Thanks again to everyone who is posting questions and helping us test!

Core components of Microsoft Web Deployment tool

2008-02-07T06:27:00Z

There are four concepts that are important for you to understand to be able to effectively use the new Microsoft Web Deployment Tool.

1. Providers

Providers are pieces of code that provide data to the tool for various operations like sync, migrate. The tool always starts with one provider, with an optional path for the provider. The provider builds the view of the part of the system it can handle, and uses other providers to represent system parts that they handle. These views are internally represented in xml. For example, if you begin with the dirPath provider, it will build an xml view of the folder (specified by the path), which will include files and subfolders present in it. It will delegate the task of handling files to the filePath provider and handling subdirectories to another dirPath provider object.

Parameters you pass in the –source and –dest switches at the command line specify the providers which builds xml views of source and destination systems. Then, the tool only needs to compare these xml views and call for Add, Update, or Delete operations on the destination to make its xml view same as source. Each provider is responsible for handling each of these operations.

Some examples of providers that are provided out of the box are dirPath, appHostConfig, and metaKey. You can see the full list of supported providers in Microsoft.Web.Deployment.config or by running the tool with no parameters to see the Help output. The tool also supports custom providers, which are built by combining one or more existing providers. Two examples of built-in custom providers are the webServer and webServer60 providers, whose definitions can be found in Microsoft.Web.Deployment.config. You can also create these custom manifests in a separate xml file and then use –source:manifest=<xmlfile> to use it as a source. When doing a sync or migrate, the same provider must be used for the source and destination. So if you are using a manifest as the source of your operation, you must also use the manifest provider as the destination. The only exception is the archiveDir provider, which can be used with other providers.

2. Links

Link extensions (or links) are executed for each entry generated by the providers. Links basically connect otherwise unrelated data based on logic coded in the extensions. An example of a link extension is the ContentExtension, which pulls in content whenever it sees a virtualDirectory configuration element added by appHostConfig provider or sees a path metaProperty. It does this by adding a dirPath entry under the virtual directory configuration, and the dirPath provider then pulls in all the content. Another example is the CertificateExtension, which is responsible for pulling in certificate whenever SSLCertHash is seen in the configuration.

Link extensions are executed for all verbs, so a dump will also cause link extensions pull in additional data. If you want to disable a link extension while executing a command, you can use –disableLink switch provided. For example, –disableLink:ContentExtension will cause content to not be included in a view of virtual directory configuration, which would allow for syncing of two servers without moving any content. Link extensions rely on providers to actually pull in the data and are limited by what providers are supported and can do. Together providers and link extensions provide the view of the system that can be dumped, archived, synced, or migrated.

3. Rules

Rules are small pieces of code that have logic to do something special and are evaluated only when a migrate or sync operation is performed (not during a dump). A few examples of things that rules do are normalizing paths using environment variables, skipping UNC paths, skipping configProtectedData configuration, etc. By default, rules only run if they are marked with the isDefault attribute set to true. The -enableRule and -disableRule switches will enable or disable a rule explicitly at the command line. For example, –disableRule:SkipUNC will cause UNC content to be included when doing a sync or migrate, because it is disabling the default rule to skip all UNC content. The list of rules can be seen in Microsoft.Web.Deployment.config.

It is also possible to create custom rules that are variations of existing rules. For example, a rule to skip all content located on drive G: would be of the same type as the SkipUNC rule in the configuration (Microsoft.Web.Deployment.SkipRuleHandler). But the absolutePath attribute would be set to “G:\\.*” instead of “\\\\.*” in the configuration file. –skip and –replace switches provided by msdeploy.exe enables creation of new rules as a command-line parameter at runtime, that are included while doing a sync or migrate operation.

4. Methods and Verbs

Methods and verbs are the operations allowed by the tool, which are called using the –verb switch at the command line. If the verb specified is not dump, sync, or migrate, it is expected to be a method that is specified in the configuration. Currently, the supported methods are GetSystemInfo and GetDependencies.

Hope this helps.

Kanwal, Nina, Faith

Using MS Deploy to compare two web sites

2008-02-02T18:07:00Z

Given that we have a metric ton of options :), I wanted to go through a couple common questions that our product support folks get. This is the first in a series of server administrator tasks I'll cover using MS Deploy. When our support folks learned that the tool will help you compare two web sites, they were thrilled.

We've all seen it happen. Two servers are configured the same way, but suddenly, one of them stops working. The content is same, everything looks right, but one server is exhibiting unusual behavior. You could try to take the config from the working server and overwrite the broken one. But that won't tell you what happened, nor if it might happen again.

There's also the case where you only have one server, which suddenly breaks. Restoring from backup if you have one or manually sifting through config might be your only option. But often the problem isn't easy to spot and backups take time and effort to restore.

Enter MS Deploy. You can compare two live sites, a site with an archive, or compare two archives. In this case, I'll show you comparison of a live web site and an archive. An archive is like a snapshot of a site or server at a given point in time. When you take the archive, you may be doing it for a manual backup, to do an offline sync or to make a copy of a specific version of your app. It's a great way to quickly take a snapshot of your working app (and config) before you make changes.

Anyway, let's take a backup of MyWebSite (creative name, I know :)), using the sync to archive functionality.

msdeploy -verb:sync -source:metakey=lm/w3svc/2 -dest:archivedir=c:\MyWebSiteArchive

This will take all the configuration, content and any certificates, putting them into an archive folder that contains an archive.xml (contains everything except for the content). It will create the folder if it doesn't already exist, and ACL for administrators only. For more info, there's an archive walkthrough available in our documentation.

Now let's say "someone" makes a change. We'll pretend that the poor server admin (that's me) doesn't know. All I know is that Monday morning the server broke. Luckily I took an archive the week before. What's next? Let's do a comparison by performing a sync with the archive as the source and the live site as the destination, using my favorite parameter, the -whatif flag. This tells the framework to show you would happen but not actually do it. Hint: It's really important to use the -whatif or you will overwrite your site from the archive.

msdeploy -verb:sync -source:archivedir=c:\MyWebSiteArchive -dest:lm/w3svc/2 -whatif

On my server, here's the output:

Action: Updating metaProperty (/lm/w3svc/2/root/AuthFlags)
Change count: 1

Now we have narrowed the problem down to AuthFlags. For authentication, it's probably a simple enough matter to check the settings. But let's say it was more complex or you wanted to know the exact change. Now we need to use that very useful flag, verboseLevel. This flag lets you control how much info is shown about an operation.

msdeploy -verb:sync -source:archivedir=c:\MyWebSiteArchive -dest:metakey=lm/w3svc/2 -whatif -verboseLevel:Informational

Here's the output (much longer):

Informational: Dependency Check: 'IisVersion' Level: 'Fatal' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'Iis5IsolationMode' Level: 'Fatal' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'ConfigRedirection' Level: 'Fatal' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'InUseDependency' Level: 'Warning' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'InstalledComponent' Level: 'Warning' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'DriveSpace' Level: 'Fatal' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'IsapiCgiExistence' Level: 'Warning' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'UsingW3SVCMimemap' Level: 'Warning' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'AppPoolNotFound' Level: 'Warning' Passed: 'True' Fail Operation: 'False'
Informational: Dependency Check: 'DisabledDependency' Level: 'Fatal' Passed: 'True' Fail Operation: 'False'
Informational: Destination metaProperty (/lm/w3svc/2/root/AuthFlags) does not match source (/lm/w3svc/2/root/AuthFlags) differing in attributes (value['5','3']). Update pending
Informational: Source metaProperty (/lm/w3svc/2/root/AuthFlags) replaced with changed attributes () due to rule EnvironmentVariableNormalize
Action: Updating metaProperty (/lm/w3svc/2/root/AuthFlags)
Change count: 1

We have narrowed down the problem now. But you'll notice a lot of information on dependency checking, this is an automatic process to ensure that your destination has the right components, drive space and even down to the application pool needed by the site. If you want to supress these during operations like a comparison, you can do that using rules.

msdeploy -verb:sync -source:archivedir=c:\MyWebSiteArchive -dest:metakey=lm/w3svc/2 -whatif -verboseLevel:Informational -disableRule:Dependency*

The -disableRule flag allows me to avoid running a rule during an operation. You shouldn't disable the dependency rules unless you're pretty comfortable with the tool and you're using it in a case when you don't need to know if there are dependencies that aren't present on the destination.

After we run without the dependency checking:

Informational: Destination metaProperty (/lm/w3svc/2/root/AuthFlags) does not match source (/lm/w3svc/2/root/AuthFlags) differing in attributes (value['5','3']). Update pending
Informational: Source metaProperty (/lm/w3svc/2/root/AuthFlags) replaced with changed attributes () due to rule EnvironmentVariableNormalize
Action: Updating metaProperty (/lm/w3svc/2/root/AuthFlags)
Change count: 1

This is what we care about: Destination metaProperty (/lm/w3svc/2/root/AuthFlags) does not match source (/lm/w3svc/2/root/AuthFlags) differing in attributes (value['5','3']).

You can see the auth flags changed from 5 on the source, which is the archive of the working site, to 3, which is the live site. If you lookup the auth flags, you'll see 5 is Windows auth and 3 is Basic. In this way you can use the tool to pinpoint any differences between two objects, including sites, directories and other providers that we support. As the support folks pointed out to me, it's a great way to locate any changed configuration between two sites.

Hope this helps!

Thanks,
Faith Allington, Program Manager

Web Deployment walkthroughs are now in RTF format

2008-01-26T17:05:00Z

Just a quick note to let everyone know that we updated the walkthroughs slightly based on comments we've gotten, and also turned them into .rtf files. The link to download is http://go.microsoft.com/?linkid=8100895 and they've been updated on the download pages.

Thanks!

Welcome to the Web Deployment Team blog

2008-01-22T19:40:00Z

Welcome to the Microsoft Web Deployment Team blog - we hope you will find this a helpful place to get the inside scoop plus plenty of tips and tricks from the team who is building the Web Deployment Tool for IIS.

So what is this new deployment tool? You may have read Scott Guthrie’s post about the future of ASP.NET and IIS. In the post he mentioned the roadmap for a web deployment framework, that’s us. :) In our first version, we’re releasing a command-line tool called msdeploy.exe that provides support for deploying, synchronizing and migrating IIS 6.0 and 7.0.

It supports moving configuration, content, SSL certificates and other types of data associated with a web server. You can choose to sync a single site or the entire web server. Because we know that one tool can never ‘automagically’ guess what your application relies on, we’ve tried to be pretty flexible and powerful – you can customize exactly what you want to sync using a manifest file. You can also skip sites or other objects, or you can perform regular expression replacements during a sync (like changing the home directory on the destination machine).

The goal of the tool is to help you keep servers in sync, to make deployment easier and also to help with migrating to new versions of IIS. You could use a sync on two machines in a web farm, for example. Or maybe you need to move to a new server of the same version, you can use this tool. Of course, we also enable you to do a migration from IIS 6.0 to 7.0.

You can learn more about the tool by reading our walkthroughs. Starting with the Learning Roadmap, you’ll see there is an installation walkthrough and also an introduction to the tool, which goes through operations in detail and shows you not only how the tool works, but why.

We hope you download Technical Preview 1 and provide us with feedback. We’re interested in how well the tool captures your web site/server, if there are any really important data types missing and how we can improve the overall experience.

Learn more about the tool from our walkthroughs.

Download the x86 version or the x64 version of our Technical Preview 1.

Thanks, and happy deployments!
Faith Allington, Program Manager, and the entire Web Deployment team