Previous Note Previous Note

DI-145 Data Acquisition Starter Kit Developer's Diary

Next Note Next Note

2/22/2011 Note #15 Testing DI-145 Data Acquisition Starter Kit Software for Compatibility

Microsoft has a lot of operating system versions on the market. Many are obsolete, but still supported by them. For instance, Windows XP and Vista are obsolete, but Microsoft will continue to support them until Apr 8, 2014 and Apr 11, 2017, respectively. It is DATAQ Instruments' policy to support all appropriate Microsoft operating systems that are still supported by Microsoft, which means that we need to test our software on no less than five: Windows XP, Windows Vista (32-bit), Windows Vista (64-bit), Windows 7 (32-bit), and Windows 7 (64-bit). How we do this makes for interesting commentary in the DI-45 Developer's Notes Series.

It All Starts with the Driver

All hardware that you connect to your PC requires a driver. A driver is a piece of software that attaches itself to the operating system to allow application programs to access the hardware it's paired with in a defined and controlled manner. Sometimes the driver is provided as part of the operating system because it's implied that all or most PCs will need it. Devices in this category are hard drives, graphics cards, CD-ROM drives, etc. Other hardware is voluntarily added to your PC to perform a specific task that wasn't envisioned by the operating system, like data acquisition, and it's in these non-mainstream application areas where you're likely to encounter a problem. Have you ever tried to install a driver in Windows XP only to see this:

XP's unsigned driver dialog box

This is Microsoft's not-so-subtle way of telling you that the driver you are attempting to install is not signed, meaning that the driver's author has not jumped through the variety of hoops the company deems appropriate to verify that it operates properly, and most especially has not paid the required fee to Microsoft for such verification. Through Windows XP this dialog box was a mere nuisance. All the intrepid PC owner needed to do was to click "Continue Anyway" and the driver installation would progress without further ado. The less adventurous would invariably call tech support for guidance, even though the installation guide stated to click the Continue button. Either way, this attempt by Microsoft to force software authors to have their drivers verified and ultimately signed by the company represented only a minor bump along the road to driver installation while completely ignoring Microsoft's preference. Alas, nothing good lasts forever.

Beginning with the 64-bit versions of Windows Vista and Windows 7, the operating system refuses to install an unsigned driver. There was only one way out of the box and that was to run the obstacle course known as Logo Testing that's required to obtain official Microsoft sanction in the form of their digital signature on your driver. To do that you set up a Windows 2003 server networked to five other PCs each running a unique flavor of Windows depending upon the OSs that you'd like to qualify for. In our case we tested driver installations on Windows XP (32-bit only) and both 32- and 64-bit versions of Windows Vista and Windows 7. The server runs driver test manager software that allows you to specify which of the client PCs you'd like to test after you install your driver with its associated hardware. You can run the entire battery of required tests, or any subset. A complete session takes about 17 hours to complete and consists of about twelve tests per operating system. The test manager keeps track of which tests pass and which do not and generates a detailed report for each. For those that fail some rudimentary guidance is provided that may help indicate reasons for the failure, but all tests for each operating system must ultimately pass to receive Microsoft's official blessing. That's achieved by packaging all of the test manager's reports that passed (don't bother trying to send one that didn't) along with the tested driver and its associated files and uploading the entire bundle to a Microsoft robot. It takes about 24 hours for the robot to verify the test results and return your driver in the exact manner it left, but with the addition of a Microsoft digital signature (it also bills your Microsoft account for the privilege). It is that signature that the OS looks for when installing a driver. If found the driver installs silently in the background so you don't even know it happened. If it doesn't find it, well you already know the consequences.The driver for the DI-145 was officially signed by Microsoft over the 2010 Memorial Day weekend after several weeks of testing.

Packaging and Testing the Installation Program

Ultimately we like to provide a single executable program that has all the elements required for a successful installation embedded so you don't have to run hither and yon to assemble individual elements yourself. This includes the signed hardware driver and its files, the application and its supporting DLLs, as well as a battery of help files. When it's all said and done a single installation can reach 60 MB or larger. It's also desirable to have just one executable for all of the operating systems that the installation supports to minimize the number of installations for a given product and therefore the complexity of servicing them. This approach carries with it the implied assumption that the installation was indeed tested on all of the OSs for which it's intended to be used. For this testing we turn to the same set of PCs that were used for driver certification.

After engineering makes a release candidate available for internal testing, several people install it on their desktop PCs to ensure that nothing critical is missing. Assuming that all appears well, we burn the installation to a standard installation CD and test it that way. The CD (or multiple versions) are taken to the PC farm, as the group of driver certification PCs is affectionately referred to internally. Each PC in the farm has a hard drive with two partitions. The bootable partition contains the OS (which is unique to each PC), and the second contains a ghosted image of a virgin OS installation including all recent Microsoft updates. We learned many years ago that it's not enough to test installations on our personal PCs, especially those that have been "touched" by other installations. There have been situations where an installation worked perfectly on one of these PCs only to fail in the field because a critical element was omitted. That element happened to be present on a tested PC because of a previous installation, but of course was missing from a PC in the field and the installation failed when it counted most. Testing proceeds on each of the farm PCs, ghosting and re-ghosting as necessary until the testers are satisfied that the installation is correct and easy to follow. We may add graphics along the way if we determine that a particular installation requirement isn't clear or needs to be re-enforced.

The first release candidate of the DI-145 installation executable was released today, Feb 22, 2011. Let the testing begin.

DI-145 Product Page: http://www.dataq.com/products/di-145/index.html.