One of the biggest challenges faced by programmers, architects, testers, and security consultants is to understand the consequences of their applications when deployed into production. Even with access to source code, it is difficult to understand everything that will occur during execution due to a variety of dependencies (for example. Different OS platforms, multiple patch levels, multiple groups contributing to code or leveraging external components).
The Microsoft Application Verifier is a runtime verification tool for unmanaged code that assists in quickly finding subtle programming errors that can be extremely difficult to identify with normal application testing. It is designed specifically to detect and help debug memory corruptions and critical security vulnerabilities. It makes it easier to create reliable applications and on the other hand find potential vulnerabilities by monitoring an application’s interaction with the OS, profiling its use of objects, the registry, the file system, and Win32 APIs (including heaps, handles, locks, and more). It also includes checks to predict how well the application will perform under Least-privileged User Account operation.
Application Verifier Identifies whether the applicaion:
• Is using:
Unsafe TerminateThread APIs.
Correct use of Thread Local Storage (TLS) APIs.
Correct use of virtual space manipulations (for example, VirtualAlloc, MapViewOfFile).
• Is hiding access violations using structured exception handling.
• Is attempting to use invalid handles.
• Contains memory corruptions or issues in the heap.
• Runs out of memory under low resources.
• Usage of critical sections is correctly occurring.
• Will run well in an environment with less privilege.
• Has any potential problems when the application is running as a limited user.
• Has uninitialized variables in future function calls in a thread’s context.
Tests within AppVerifier :
The tests included in the Application Verifier are to help software developers avoid common mistakes by using verification layers to validate the usage of API families:
• Basics Verification – These are the core set of tests that check for issues within heap, handles, locks, and more.
• Low Resource Simulation Verification – This test simulates an environment under low resources for example, out of memory.
• Limited User Account Predictor Verification – This test simulates an environment running as a user with a limited user account in either predictive or diagnostic mode and identifies potential problems accordingly.
• Miscellaneous Verification – This consists of two checks: Dirty Stacks and Dangerous APIs.
Getting and Installing Application Verifier:
The latest version of Microsoft Application Verifier can be downloaded from here . The installation is pretty simple and a straightforward next next.
Using the Application Verifier Tests:
The AppVerifier can be run through either the User Interface and / or automated through the command line. I will describe both these process in detail. The expectation for these scenarios is that the application does not break into debugger and all tests pass with the same pass rate as when run without AppVerifier enabled.
The Basic Tests:
1. Enable verifier for the application(s) you wish to test using:
From the command line: appverif /verify ApplicationToTest.exe or from the user interface:
Add your application by right-clicking within the Applications area and clicking Add Application. Select Basics from the Tests area and click Save.
Note:
• /verify enables the basic tests.
• If you are testing a DLL Application, AppVerifier has to be enabled for the test .exe that is exercising the DLL. You cannot debug a standalone DLL.
• Run the verification layers separately.
2. Run ALL your test exercising the application.
3. Analyze any debugger breakpoints encountered. If a break occurs, you need to understand why and fix it. (The Help contents provide details on the breaks and how to investigate them.)
Finally from the command line: appverif /n ApplicationToTest.exe or from the user interface:
Remove your application by right-clicking within the Applications area and clicking Delete Application and click the Save button.
The Low Resource Simulation and Fault Injection :
The expectation for this scenario is that the application does not break into the debugger. This means that you have no errors that need to be addressed. The pass rate for the tests may decrease significantly since random fault injections are introduced into the normal operation.
1. Enable Application Verifier low resource simulation (fault injection) for the application(s):
From the command line: Appverif /verify ApplicationToTest.exe /faults or from the user interface: Select Low Resource Simulation from the Tests area and click save.
Note: If you are testing a DLL, you can apply low resource simulation (fault injection) on a particular DLL instead of the whole process. The command line format would be:
appverif /verify TARGET [/faults [PROBABILITY [TIMEOUT [DLL …]]]]
Example: appverif /verify ApplicationToTest.exe /faults 5 1000 d3d9.dll
2. Run ALL your tests exercising the application.
3. When finished, delete all settings. Analyze any debugger break(s) encountered. If a break occurs, you will need to understand why and fix it.
4. When finished, delete all settings:
Running with and without fault injection exercises largely different code paths in an application and therefore both scenarios must be run in order to get the full benefit of AppVerifier.
Analyzing AppVerifier Data :
All data created during AppVerifier analysis is stored in the %ALLUSERSPROFILE%\AppVerifierLogs folder in a binary. These logs can then be converted to XML via the user interface or command line for further analysis. To view the XML files, use any: 1. Web Browser 2. Create an XSLT transformation to convert raw XML content 3. Import into Excel or any Database.
Some Key Points:
• AppVerifier is designed to test unmanaged applications (non-.NET Framework applications) on Microsoft (2000 and above) platforms.
• While running a full page heap, it is recommended to be at least 1 GB.
• It does not need access to the source code to execute its tests, although the availability of either the application’s symbols or debug information will make a dramatic difference in the quality and usefulness of the data collected.
How Does AppVerifier Work?
AppVerifier works by modifying the unmanaged DLLs Method Tables so that the required checks are performed before the real function is executed (“Function Hooking”). For example, the address of the Win32 API CreateFileA method is replaced with an internal AppVerifier method that will trigger a series of tests that when positive will be logged. When new processes are started, the use of AppVerifier’s Method Table Hooking techniques is controlled by entries made in specific registry keys. If the registry entry exists, then the AppVerifier DLL will be loaded in a newly created process that will handle the Method Table replacements in the existent and subsequently loaded DLLs. Since these hooks are made when the DLL is loaded, it is not possible to use AppVerifier on a process that is already running.
When a problem is identified a verifier stop will occur. The number provided is used to identify the exact nature and reason for its occurrence.
Detecting Heap Corruptions:
In order to detect heap corruptions (overflows or underflows), AppVerifier will modify the way memory is allocated by padding the requested memory with either full non-writable pages or with special tags before and after the allocated memory.
When padding the requested memory with full non-writable pages, AppVerifier will consume a large amount of virtual memory but has the advantage that heap corruption events are cached in real time when the overflow or underflow occurs. The heap check will place a guard page at the beginning or end of allocation depending on the Backward property. If Backward is set to False, which is the default, it will place a guard page at the end of the allocation to catch buffer overruns. If it is set to True, the guard page is placed at the beginning of the allocation to catch buffer underruns.
When padding the requested memory with special tags (enabled by clearing the “Full” check box item in the heap properties), AppVerifier will check and alert you when this memory is released. The main problem in using this technique is that there are some cases when the memory corruption will only be detected when the memory is released (the minimum amount of memory block is 8 bytes), so when on a 3-byte variable or a 5-byte overflow occurs it will not be immediately detected.
On an underflow event, an attempt will be made to write to a Read-Only page. This will trigger an exception. This exception is only be caught if the target application is being executed under a debugger. Note that the full page heap mode will also detect these errors as it uses padding + guard pages. The reason you would use light page heap is if your machine cannot tolerate the high memory constraints of full page heap.
For memory intensive applications, or when it is required to use AppVerifier during long periods of time (e.g. stress testing), it is better to run normal (light) heap tests instead of full mode due to the performance degradation. However, when you run into an issue turn on full page heap to investigate further.
References and Further Reading :
MSDN, Microsoft Application Verifier Help.
Function Hooking and Patching Articles: Detours
From Code Project: API hooking revealed – Process-wide API spying – an ultimate hack
The Files used for reference can be downloaded from here.
