With everyone eagerly installing Windows 7 on their systems here at OSR, a couple of interesting crash dump mysteries were brought to light. The first mystery turns out to be a remnant of an O/S long forgotten whereas the second mystery represents a new policy regarding Windows' retention of crash dump files.

Did that catch your attention? It certainly caught mine and became a hot topic of conversation at lunch one day. What spurred this comment was the disappearance of the "Complete memory dump" option from the debugging information dialog on a laptop running Windows 7 x86:

The initial collective guess was that the paging file was not large enough on this system to hold a full memory dump, given that the system had 4GB of memory installed. Unfortunately, the paging file on this system was more than large enough so we continued to be stumped.

Looking at another system on the network, we discovered an XP x64 workstation with 8GB of RAM that was also missing the complete memory dump option. Then we discovered a Windows 7 VM with 2GB of RAM that did have the complete memory dump option. Clearly there was something going on here that we didn't quite understand, which meant that this was now officially a challenge.

After spending much time making guesses as to what would cause the O/S to show or not show the complete memory dump option, we finally sat down and started digging into the control panel applet responsible for this dialog, sysdm.cpl. This unveiled an interesting code path, as can be seen in this IDA output:

Notice that this performs a comparison of a variable with 2GB, jumping over the addition of a string from the image's string table to a combo box if the variable value is above 2GB. Loading the string table of the control panel applet, we found that string 0x4E was "Complete memory dump".

Suspicious that the 2GB value was a compare of the amount of RAM installed, we went back to check the amount of memory that was in each system and whether or not the complete memory dump option was available:

1) Windows 7 x86 laptop - 4GB and no option

2) Windows XP x64 workstation - 8GB and no option

3) Windows 7 x64 VM - 2GB and had the option

We then went ahead and added another 512MB of RAM to the Windows 7 VM. As expected, the complete memory dump option disappeared on the next boot!

Knowing for a fact that Windows supported crash dumps larger than 2GB, we took to Google to finally solve this mystery. As it turns out, the I/O Manager originally limited the size of a complete memory dump to 2GB, but that check was removed via a hotfix to Windows 2000. While the hotfix updated the I/O Manager to remove the check, no one has ever bothered to go back and update the control panel applet that displays the crash dump registry settings.

Luckily, the workaround for this is easy enough. You'll need to bypass the dialog and manually modify the registry yourself, setting the \HKLM\System\CCS\Control\ CrashControl\CrashDumpEnabled DWORD value to 1. By doing so you'll configure the O/S to generate a full memory dump on the next system crash and you'll even regain access to the complete memory dump option in the dialog:

Now that that mystery was solved, there was even a greater mystery brewing: Windows 7 systems that were configured to generate crash dumps were crashing but not leaving behind a dump file. After having ruled out the usual reasons such as too small of a paging file or not enough disk space to save the dump, there was definitely something strange going on.

What we discovered is that a new policy has been added to Windows when it comes to crash dump file retention. Apparently, certain customers are complaining about the amount of disk space that crash dump files consume and don't want them saved anymore (we're sure they'll regret this decision in the future if their systems ever crash). In order to address this issue, a new set of criteria has been created that determines whether or not Windows saves a crash dump file.

The new criteria are as follows:

1) If the machine is running a server version of the O/S, the crash dump is always saved.

2) If the machine is joined to a domain, the crash dump is always saved.

3) If the amount of free space on the disk is greater than or equal to 25GB, the crash dump is saved.

Otherwise, the crash dump file will not be saved and you'll be left scratching your head. Luckily, a new registry value has been added that will override this behavior and always cause the crash dump file to be generated. By setting the \HKLM\System\CCS\Control\CrashControl\AlwaysKeepMemoryDump DWORD value to 1 you will guarantee that you will always have a crash dump file after crashing the system.

A new release of the O/S is always time to discover old issues that you never noticed before and experience new issues that never existed before. As adoption for Windows 7 grows I'm sure there will be more articles like this to come in future issues of The NT Insider!