Overview of Application Verifier

The Application Verifier (AppVerifier) tool is a graphical user interface (GUI) tool that will aid IT managers and developers in testing applications on the Windows XP operating system. AppVerifier focuses on two distinct categories of test: those that are difficult to perform due to system limitations of test computers, and subtle problems that are difficult to diagnose. AppVerifier’s purpose is to assist developers in creating stable applications that will run well on Windows XP.

NOTE: Many of the tests that Application Verifier will perform will result in a crash if a problem is detected. Because of this it would be beneficial to the developer to attach a debugger to the test computer before running any tests with Application Verifier. Using a debugger will enable you to gain more information about why the application failed the test.

Applications that ran properly on earlier versions of the Windows operating system may not perform as expected on Windows XP or .NET Server family operating systems. Among the reasons why this may happen:

  • The application refuses to run when Windows reports new, higher operating system version numbers. Often the application will work well on the new version of Windows if the user can get past this block in the application.
  • The application calls older versions of Win32 API functions that return unexpected values on computers with large amounts of resources such as disk free space.
  • The application expects older formats of Windows data.
  • The application expects user information, such as personal and temporary folders, to be in specific locations or specific formats.

Application Verifier runs on Windows 2000, Windows XP, and the .NET Server family.

See Also:

Features of Application Verifier
Testing applications with Application Verifier
Application Verifier modes
Application Verifier Advanced Options
Application Verifier log files
Application Verifier tests


Application Verifier Features

Application Verifier offers the following features:

  • Integration of PageHeap and GFlags: Application Verifier wraps the functionality of the PageHeap and GFlags utilities in a simplified UI. You can enable page heap checking with a single mouse click. The first four tests listed in the user interface are incorporated from these two utilities.
  • Error Logging: Application Verifier places debug information into a log file that can be viewed and manipulated through the Application Verifier UI. This is helpful for non-technical users who do not know how to use a debugger.
  • Application Compatibility Testing: The tests written by the Windows Application and Customer Experience (WACE) team monitor for the most common application problems, including incorrect version checking, bad registry usage, and hard-coded file paths. These tests are also designed to assist you in testing applications in preparation for the Designed for Windows Logo Program.


Testing Applications

  1. Add an application to the list by clicking Add. When an application is added to the list, Application Verifier is globally enabled for it. This means that whether the application is run from Application Verifier, a shortcut, auto-run, or the command line the selected Application Verifier tests will be applied.
  2. Select the appropriate tests from the Test settings list. Note that some of the included test can not be performed together, such as the LogFileChanges test that must be performed in isolation.
  3. Start your application and exercise it. Application Verifier does not perform the tests of your binary; it only monitors and reports suspicious activity that it detects while you perform tests.

NOTE: You do not need to click Run button to exercise your application. It is only provided as a short cut. You can always run the application as you normally would run it (such as through the Start menu, shortcuts, and by double clicking the executable) and Application Verifier will still apply the tests. The test settings are persistent even through a reboot of the computer, and will remain until you specifically remove them.

  4. Once testing has completed, click View Logs to see the log entries that were generated.
  5. Remove the application from Application Verifier when testing is complete by highlighting it and clicking the Remove button.

Expected Behavior while running Application Verifier:

  • The application may run more slowly than usual. This happens because of test overhead.
  • The application may not launch for any of the following reasons:
    • The application may fail a test during launch. Try clearing a few of the check boxes for the tests and try to launch the application again.
    • The application may not work with a debugger attached.
  • The application may crash and prompt to have a debugger attached. Some tests will crash the application if a fault is detected.


Application Verifier Modes

Application Verifier is a useful tool, but it will not perform automated testing of your applications. This is due to the wide variety of applications that it is designed to work with. When you employ Application Verifier to test your application you must exercise all of the features of the program to ensure full testing.

Application Verifier works in either of two modes:

  • Debugging Mode In debugging mode, Application Verifier is used to detect problems in an application and feeds the resulting information out to a debugger. This is particularly useful when testing applications that exhibit issues that cause access violations or other problems that can be detected with Application Verifier.
  • Logging Mode The logging mode is useful for those who either do no longer need a debugger for their testing, or for those who are unfamiliar with the use of a debugger. In logging mode, all of the output generated by Application Verifier is entered into a text log file for later review.

NOTE: SafeDisc application protection software prevents an application from running while there is a debugger attached to the computer. Because of this, you will not be able to run Application Verifier in debugger mode if your application utilizes SafeDisc.


Application Verifier Advanced Options

The advanced options in Application Verifier can be accessed by clicking the Options button in the main user interface.

The Options dialog has three tabs: General, HighVersionLie, and LogFileChanges.

The General tab provides options that enable you to decide how the tool will process tests. The tab has the following options:

  • Break in the debugger after each logged event (debugger required) You will be required to select the name of the installed debugger you wish to use from the drop down list box.
  • Propagate tests to child processes (Note: the first four tests in the Test settings box cannot be propagated) Selecting this option will instruct Application Verifier to also test any child processes spawned by the selected application.

The HighVersionLie tab provides text boxes for you to enter a fictitious version number for Windows to test the application's ability to run on future versions of the operating system. For instance, Windows XP is version 5.1.2600. That would be major version 5, minor version 1, build number 2600. Anything higher than those numbers would indicate a future (fictitious) version of Windows.

The LogFileChanges tab has three options:

  • Log attributes on file create and modify events This option will cause Application Verifier to record the attributes of any file that is modified or created by the application while it is being tested.
  • Log all changes in the Windows directory and below This will enable Application Verifier to track any changes to the file system beneath the Windows directory to determine if the tested application is storing data incorrectly.
  • Log all changes in the Program Files directory and below Application Verifier will detect and log any changes made to the file system beneath the Program Files directory if this option is selected. Applications may store data that is private to the application under the Program Files folder, but any user-generated data should be stored elsewhere in the file system.

Both the HighVersionLie and LogFileChanges tabs offer a Default button to reset the current values to the default settings.


Application Verifier Log Files

When you have run an application using Application Verifier, you can view the output in the Application Verifier Log window. You can also use this window to view logs that have been exported using Export Log.

The logs are shown in tree form. Each log is made up of expandable items displaying information about the executable's run. This information can be as simple as a notification that the listed executable started and stopped, but it can also include more detailed information about specific Microsoft Windows requirements that are violated by the executable. Expanding an item shows a list of violations that fall under that item's category. Click an item to see more information on resolving the problem in the Possible solutions text box.

The log is not a complete list of issues found. Some tests log data, some throw exceptions (in other words, they crash the application intentionally), and some change the applications environment and rely on the tester to verify that the application continues to function correctly.

Delete Log You can eliminate a single log entry from the window by highlighting its top level node (as shown in the graphic above) and clicking the Delete Log button. This will not clear the entire log file, only the entries for a specified test run of the application.

Delete All This button will enable you to clear all of the log entries for every test run listed in the log file.

Export Log If you want to share an Application Verifier log with another person, click Export Log in the Verifier Log window. This will create a text file that you can send to other individuals. These people can view the log in any program that allows you to view a text file or they can use View External Log to examine the exported log from the main interface of Application Verifier.

View Details This button enables you to open the View Log - Detailed (shown in the following graphic) dialog to see the granular information of the logged events. Information displayed includes the module that caused the event, the memory offset where the issue occurred, the severity of the issue (whether informational or error), and a text description of the problem. To open the detailed view highlight a top-level node entry (as in the previous graphic) and click the View Details button.

The final option on the Logs dialog is a filter option for the events displayed in the list. The default option is to only display error events in the log. A second option enables you to view all events that were captured in the log including informational events such as the start or stop of the application.


AppVerifier Tests

Application Verifier Tests

Application Verifier contains multiple tests that can be used to determine potential issues with an application. Several of these tests can be used as indicators of compliance with the Designed for Windows Application Logo Specification. All of the tests listed below are beneficial to eliminating compatibility issues for Windows-based applications.

The complete list below reflect the tests that are valid for Windows XP and the .NET Server family. On Windows 2000, the Locks, Handles, Stacks, and LogStartAndStop tests are not valid and are not displayed in Application Verifier.

PageHeap - Detect heap corruptions: Checks the memory heap for corruption and adds guard pages to the end of each allocation. This causes access violations when there are buffer overruns. A debugger is required to get full information on any memory problems.

Locks - Check Lock Usage: Checks for errors in file lock usage. This might cause access violations when errors are located. A debugger is required to view results.

Handles - Detect Invalid Handle Usage: Checks for handle errors. This might cause access violations when errors are located. A debugger is required to view results.

Stacks - Check for adequate stack: Checks for adequate stack size by stopping stack growth. This will cause a stack overflow error if the original allocated stack size is too small. You can prevent this from happening by increasing the stack commit size.

LogStartAndStop - Logs Start and Stop: Logs application start and stop. May be very useful if tests are being performed repeatedly on a single application.

FilePaths - Checks System Path Usage: Checks if the application is using system paths that was not obtained using a method approved by the Designed for Windows Logo Program. Problems are logged if they are detected. The tested application will stop responding or close if it does not handle these paths correctly.

HighVersionLie - Checks Version Handling: Notifies the application that it is running on a version of Windows that does not yet exist. The tested application will stop responding or close if it does not handle version information correctly.

RegistryChecks - Checks Registry Usage: Checks if the application is accessing areas of the registry that should be used only by the system. Problems are logged if they are detected.

WindowsFileProtection - Logs Changes to Windows File Protection files: Ensures that an application doesn't modify or delete files protected by Windows. (Logs information only.)

DXFileVersionInfo - Logs DirectX file checks: Logs any attempts that an application makes to perform a version check on DirectX files. (Logs information only.)

LogRegistryChanges - Logs registry changes: Logs all changes that an application makes to the registry. (Logs information only.)

LogFileChanges - Logs file system changes: Logs all changes that an application makes to the file system. (Logs information only.)

ObsoleteAPICalls - Logs calls made to obsolete API’s: Logs calls made to APIs that have been marked as obsolete in the Microsoft Platform SDK. (Logs information only.)

KernelModeDriverInstall - Logs installations for kernel-mode drivers: Logs any attempts made by an application to install a kernel-mode driver. (Logs information only.)

SecurityChecks - Logs potential security issues: Detects and logs potential security issues during calls to Windows' API calls. (Logs information only.)