Installation Authoring and Testing Techniques

By Johnson Luong, Software Engineer

Everyone talks about how great our Synergy/DE products are and how much work we put into making sure they are the best representation of months—and sometimes years—of effort, right? However, no one ever talks about how much work goes into the installation for those products. The development and QA teams here at Synergex work together to not only ensure that Synergy/DE products go out the door as bug free as possible, but also that our installations install those products correctly, across multiple operating systems, upgrading from multiple previous Synergy versions, and with a variety of feature configurations.

Synergex uses WiX, the Windows Installer XML Toolset, and a Visual Studio extension called Votive to author installations. WiX is by far the easiest tool to use to author MSI installers, because all WiX source files are simple XML text files, which makes them easy to track in source control. WiX also includes many utilities that are great for writing and testing installations. Best of all, WiX is free and open source. You can find out more about WiX at http://wixtoolset.org/.

Some of you may be curious how Synergex uses WiX to author installations and how our test engineers test those installations to make sure that everything installs, uninstalls, modifies, and repairs successfully. This article describes several techniques Synergex uses in its installations, which you can use to author and test your own WiX installations. We hope these tips are helpful to you for your own WiX authoring and testing purposes.

The first technique is file harvesting. Usually, when developers create an installation, they have to code in every single file that they want the installation to install and manage. This can mean a lot of menial work, especially for large installations with many files. In addition, there could be issues that arise out of this type of authoring, such as source files that frequently change.

The solution to these issues is file harvesting, which is the process of serializing files. A specialized file harvester for installation authoring can, for example, take in a directory path and output a generated WiX code file that a developer can use right away. Luckily, WiX comes with a file harvester, called Heat, out of the box. Heat supports many harvesting types, including directories, project outputs, and registry files. For example, you can add Heat as a prebuild task to harvest files out of a folder before every build of the installer:

heat dir InstallerFiles -gg -cg files -var var.InstallerFilesDir -out Files.wxs

You can read more about Heat at http://wixtoolset.org/documentation/manual/v3/overview/heat.html.

If you have multiple installations, you’ll probably want to make use of common libraries. Installations can share a lot of common code, such as error dialogs and UI. Rather than constantly reinventing the wheel, you want to make sure that you adhere to the DRY (don’t repeat yourself) principle. Common libraries can help you do this by promoting code reuse and ease of maintenance. And best of all—with common libraries, you only need to fix bugs once.

WiX calls these common libraries “setup libraries.” They are similar to class libraries in other languages. The WiX assembler, Candle, builds the setup libraries so that the other installer projects, called setup projects, can use them. To use your own common libraries, just reference them in your setup projects.

An additional technique that you can use in your WiX installations is a custom MSBuild targets file, which enables you to assign specific variables to Candle by setting your solution configuration or platform. For example, a custom targets file enables you to have a single setup project for both your x86 and x64 installations: MSBuild reads in your custom targets file and sets the location of the source files depending on which platform you are building.

To implement this type of solution, you’ll need to put all your relevant variables in PropertyGroup conditions inside the targets file. For example, we split up our custom targets file so that the platform of the build of the Synergy/DE product determines what WiX variables are set. In the code below, you can see that we use the targets file to assign output naming, versioning, and source information:

<PropertyGroup Condition=" '$(Platform)' == '1030303' ">
  <OutputName>101SDI1033c_$(ShortBuildVersion)</OutputName>
  <VS_ROOT>$(SYNWINROOT)\sde1033\VisualStudio\</VS_ROOT>
  <WixVariables>$(WixVariables);VS_ROOT=$(VS_ROOT)</WixVariables>
  <WixVariables>$(WixVariables);NupkgVersion=10.3.3030</WixVariables>
  <DefineConstants>$(WixVariables);BuildVersion=1030303</DefineConstants>
</PropertyGroup>

By splitting up WixVariables this way, we can build and easily manage a release for any prior, current, or future version of a product without having to reconfigure version-specific variables; all we have to do is add a new property group to the targets file.

You can modify your installation files to import a custom targets file by adding this line to the wixproj (WiX project) file directly under the Project element:

<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Import Project="custom.targets"/>

After you finished authoring your installation, you’ll want to have a test engineer make sure that the installation actually installs things correctly and to the correct destinations. WiX does not have a built-in utility for this type of testing; however, it is simple to do in C# by reading a baseline CSV file that contains a list of every file that should be installed. The reason for choosing CSV files instead of JSON or XML files lies in how easy they are for anyone to use without having to understand the file structure. A non-programmer can easily open, edit, and save a CSV file in a user-friendly program such as Excel. In addition, .NET includes a built-in CSV file parser that does not require a NuGet package or any external imports. To read in the CSV file:

using Microsoft.VisualBasic.FileIO;

MemoryStream ms = new MemoryStream();
StreamWriter sw = new StreamWriter(ms);
sw.Write(baselineTestFile);
sw.Flush();
ms.Position = 0;

TextFieldParser tfp = new TextFieldParser(ms);
tfp.SetDelimiters(",");
string[] fields;

// Comparing data occurs inside the while loop
while (!tfp.EndOfData)
{
    fields = tfp.ReadFields();
    if (!File.Exists(fields[0]))
        // Test fails
}

Another thing the test engineer has to verify is whether the installation build script has successfully signed certain files. Unfortunately, there’s no built-in WiX utility for this type of test either, so at Synergex we made our own. Our signing validator first unpacks the .cab file for an installation to test:

using Microsoft.Deployment.Compression.Cab;

new CabInfo($@"{msiFolder}\data1.cab").Unpack($@"{msiFolder}\unpacked");

Then, it runs the Windows Sysinternals Sigcheck utility on the file. Sigcheck can determine if files are unsigned, missing manifests, and more. We run Sigcheck on the entire unpacked folder, direct the results to a CSV file, and compare them with a pre-existing baseline:

sigcheck -a -c -e -l -q -s > results.csv

You can find out more about the Windows Sysinternals Suite and Sigcheck at https://docs.microsoft.com/en-us/sysinternals/.

We hope these ideas are helpful for your own installation authoring and testing. We use all these techniques in the installers you use to install Synergy/DE products. You can use file harvesting to help automate the process of getting files from your sources, you can use common libraries to reduce code bloat and keep yourself DRY, and you can use a custom MSBuild targets file to help separate out your installations without the need to have multiple setup projects. Combine all these techniques together and you get an installer that is as robust as ours!

We also do a lot of testing on our installers to make sure they are working correctly on all the operating systems we support. We use both techniques described in this article: reading CSV files to determine if the installer installed files correctly and unpacking installation .cab files to determine if the files are correctly signed.