I am currently preparing for a day long tutorial on Windows Volatile Memory Forensics for Incident Response, which Michael Cohen and I are presenting at the AusCERT conference next week. A significant part of the analysis component of the tutorial will focus on the open source volatile memory analysis tool, Volatility.
A current limitation Volatility’s support for Windows is that it doesn’t support analysis of anything other than Windows XP. This comes down to two issues:
- The core kernel structures (symbols) which are related to information about processes, threads, open sockets and the like, tend to change from version to version of Windows; and
- The technique that Volatility uses to initially find those structures does not work on Vista and above.
In this post I will be relating the first steps of how I ported Volatility to accept new symbol definitions in support of Vista in x86 (ie 32 bit) systems. Beta code (complete enough to run pslist amongst others) is now in subversion.
My approach involved integrating automatically generated type definitions for Windows XP SP3, before applying the approach to Vista. It may seem redundant to use automatically generated type definitions for XP SP3, given that volatility works fine with its own hand coded ones for this version, however, doing so provides a useful foundation step, highlighting any assumptions being made higher up in the code.
Generating new structure definitions from PDB files
Thankfully all of the heavy lifting has been done by others. Brendan Dolan-Gavit has produced pdbparse that generates volatility compatible structure definitions from the symbol files that Microsoft produces.
The steps I undertook are:
1. Downloaded Windows XP with Service Pack 3 x86 retail symbols from Microsoft. I installed them to C:\dev\XPSP3_x86\
2. Checked out via subversion the current beta of Volatility.
3. Checked out via subversion pdbparse.
4. I installed the python construct module, on which pdbparse relies.
5. I then ran the tpi_vtypes.py program from pdparse against the symbol file which corresponds to the Windows XP uni-processor kernel, ntoskrnl.pdb, generating a python symbol definition file for volatility in the process.
C:\mysrc\pdbparse>c:\Python25\python.exe tpi_vtypes.py c:\dev\XPSP3_x86\symbols\EXE\ntoskrnl.pdb > xp_sp3_x86_vtypes.py
Adding structure definitions to Volatility
Adding the new structure definitions to volatility is simplified by the new “profile” abstraction recently introduced. Profiles allow for defining sets of structure definitions specific to a particular envionment. In this case, I created a new profile called “WinXPSP3”.
The steps I took are as follow:
1. Moved xp_sp3_x86_vtypes.py to the \plugins\overlays\windows folder within the volatility source tree.
2. Created a new profile implementation called xp_sp3_x86.py. I based this off the existing windows XP SP2 profile, modifying to suit. In essence, the new profile is composed of three things:
- a definition of native types (these appear to generally apply to 32bit windows operating systems (I simply took the exiting windows definitions);
- a definition of the specific structural types which I just generated (contained in vista_sp0_x86_vtypes.py);
- an overlay definition (I simply reused the existing XP SP2 one
class WinXPSP3(xp_sp2.WinXPSP2):
""" A Profile for windows XP SP3 """
native_types = vtypes.x86_native_types_32bit
abstract_types = xp_sp3_x86_vtypes.ntoskrnl_types
overlay = vtypes.xpsp2overlays
3. Running volatility against an appropriate image resulted in the following:
c:\mysrc\Volatility-1.4_beta1>c:\Python26\python.exe volatility.py –profile Win
XPSP3 -f "C:\Users\bradley\Desktop\images\RAM\XPSP3-x86.dmp" pslist
Volatile Systems Volatility Framework 1.4_beta1
invalid syntax (xp_sp3_x86_vtypes.py, line 63)
4. Inspecting the auto-generated structure definitions where the error was yielded the following:
‘Valid’ : [ 0x0, [UNIMPLEMENTED LF_BITFIELD]],
5. Reading the above error indicated that pdbparse does not handle bit-fields in structures. I modified tpi_vtypes.py to simply ignore any structure features that pdbparse tagged as UNIMPLEMENTED.
def print_vtype(lf):
print " ‘%s’ : [ %#x, {" % (lf.name, lf.size)
for s in lf.fieldlist.substructs:
ms = member_str(s.index)
if (ms.find("UNIMPLEMENTED") >= 0):
pass
else:
print " '%s' : [ %#x, %s]," % (s.name, s.offset, ms)
print "} ],"
6. I ran volatility again:
c:\mysrc\Volatility-1.4_beta1>c:\Python26\python.exe volatility.py –profile WinXPSP3 -f "C:\Users\bradley\Desktop\images\RAM\XPSP3-x86.dmp" pslist
Volatile Systems Volatility Framework 1.4_beta1
Error – Flags has no offset in object _MMVAD_SHORT. Check that vtypes has a concerete definition for it.
Error – Flags has no offset in object _CONTROL_AREA. Check that vtypes has a concerete definition for it.
Error – Flags2 has no offset in object _MMVAD_LONG. Check that vtypes has a concerete definition for it.
Error – Flags has no offset in object _MMVAD_LONG. Check that vtypes has a concerete definition for it.
‘NoneType’ object has no attribute ‘Signature’
7. Reading the above error indicated that the new structure definitions were missing some fields. These are most likely related to the removal of the bit-fields. Viewing the structures which correspond in the original vtypes.py file for XP SP2, we have:
‘_MMVAD_SHORT’ : [ 0x18, { \
'StartingVpn' : [ 0x0, ['unsigned long']], \
‘EndingVpn’ : [ 0x4, ['unsigned long']], \
‘Parent’ : [ 0x8, ['pointer', ['_MMVAD']]], \
‘LeftChild’ : [ 0xc, ['pointer', ['_MMVAD']]], \
‘RightChild’ : [ 0x10, ['pointer', ['_MMVAD']]], \
‘Flags’ : [ 0x14, ['unsigned long']], \
} ], \
8. The corresponding auto-generated structure definition is:
‘_MMVAD_SHORT’ : [ 0x18, {
'StartingVpn' : [ 0x0, ['unsigned long']],
‘EndingVpn’ : [ 0x4, ['unsigned long']],
‘Parent’ : [ 0x8, ['pointer', ['_MMVAD']]],
‘LeftChild’ : [ 0xc, ['pointer', ['_MMVAD']]],
‘RightChild’ : [ 0x10, ['pointer', ['_MMVAD']]],
‘u’ : [ 0x14, ['__unnamed_1498']],
} ],
9. In the standard Volatility type definitions, offset 0×14 into the struct is an unsigned long type called Flags, whereas is the auto generated types, the same offset is a field called “u”, which has type ‘__unnamed_1498′. Further investigation reveals that the latter type appears to be a Union structure.
10. In order to resolve this discrepancy, my goal was to take the working field definition for “Flags” from the original “_MMVAD_SHORT” structure, and transplant it in the newly generated structures.
11. The problem with doing that is that editing automatically generated code tends to lead to problems down the line. Fortunately, recent work in the volatility framework has anticipated this kind of need, with the introduction of type “overlays”.
12. The existing XP type definitions already contained an overlay for the flags field, so Michael Cohen gave me the magic incantation necessary to patch the in memory version of the overlay structure. The patched version of the _MMVAD_SHORT overlay contains a field ‘Flags’, which, when the offset of it is read, returns the offset of the ‘u’ field of the auto-generated type definition. Note that I am glossing over significant complexity here.
xpsp3overlays['_MMVAD_SHORT'][1]['Flags'][0] = lambda x: x['u'][0]
My next post will cover adding Vista structure definitions, and a final post will cover the key modifications needed to find the KPCR structure.
Schatz Forensic
