Abstract Link to heading
Being interested in memory forensic for a while now, I have learned a lot about the volatility3 framework. In this article, we will go through how you can use the framework’s libraries to automate your memory analysis tasks and directly exploit the results. I will assume in this article that the reader has a basic understanding of how volatility3 is exploiting memory to extract evidence. If you want to learn more about volatility3, you can check the links in the “References” section.
Volatility3 CLI Link to heading
When using the volatility3 framework on a memory image, the analyst is typically using the command line interface (CLI) component of the framework. It is one of the best way to interact with the framework quickly and get results. In the below example, we are executing the PsList plugin on a memory dump using the CLI.
» vol -r pretty -f Triage-Memory.mem windows.pslist
Volatility 3 Framework 2.0.1
Formatting...0.00 PDB scanning finished
| PID | PPID | ImageFileName | Offset(V) | Threads | Handles | SessionId | Wow64 | CreateTime | ExitTime | File output
* | 4 | 0 | System | 0xfa8003c72b30 | 87 | 547 | N/A | False | 2019-03-22 05:31:55.000000 | N/A | Disabled
* | 252 | 4 | smss.exe | 0xfa8004616040 | 2 | 30 | N/A | False | 2019-03-22 05:31:55.000000 | N/A | Disabled
* | 332 | 324 | csrss.exe | 0xfa80050546b0 | 10 | 516 | 0 | False | 2019-03-22 05:31:58.000000 | N/A | Disabled
* | 372 | 364 | csrss.exe | 0xfa800525a9e0 | 11 | 557 | 1 | False | 2019-03-22 05:31:58.000000 | N/A | Disabled
* | 380 | 324 | wininit.exe | 0xfa8005259060 | 3 | 78 | 0 | False | 2019-03-22 05:31:58.000000 | N/A | Disabled
* | 416 | 364 | winlogon.exe | 0xfa8005268b30 | 3 | 110 | 1 | False | 2019-03-22 05:31:58.000000 | N/A | Disabled
* | 476 | 380 | services.exe | 0xfa8005680910 | 12 | 224 | 0 | False | 2019-03-22 05:31:59.000000 | N/A | Disabled
* | 484 | 380 | lsass.exe | 0xfa80056885e0 | 7 | 650 | 0 | False | 2019-03-22 05:32:00.000000 | N/A | Disabled
* | 492 | 380 | lsm.exe | 0xfa8005696b30 | 10 | 155 | 0 | False | 2019-03-22 05:32:00.000000 | N/A | Disabled
* | 592 | 476 | svchost.exe | 0xfa80056e1060 | 9 | 375 | 0 | False | 2019-03-22 05:32:01.000000 | N/A | Disabled
* | 672 | 476 | svchost.exe | 0xfa800570d060 | 7 | 341 | 0 | False | 2019-03-22 05:32:02.000000 | N/A | Disabled
* | 764 | 476 | svchost.exe | 0xfa800575e5b0 | 20 | 447 | 0 | False | 2019-03-22 05:32:02.000000 | N/A | Disabled
* | 796 | 476 | svchost.exe | 0xfa8005775b30 | 15 | 368 | 0 | False | 2019-03-22 05:32:03.000000 | N/A | Disabled
* | 820 | 476 | svchost.exe | 0xfa800577db30 | 33 | 1073 | 0 | False | 2019-03-22 05:32:03.000000 | N/A | Disabled
[etc...]
Having those results is a great way to help you during an investigation, but what if we want to make the framework perform more specific tasks to include it to another platform ? For example, you may want to integrate the results directly inside a sandbox or a Web UI. Behind the scene of the execution of this command, volatility3 is using multiple libraries from the framework to produce its results. It’s simply parsing the user’s arguments and perform the following steps :
- Create an empty memory context for the memory image;
- Determine if the requested plugin exists in the available list of plugins;
- Determine what configuration options the plugin requires;
- Complete the context configuration using the plugin requirements, automagics and the user’s arguments;
- Run the plugin;
- Render the result into the desired format.
Great, now you have a high level understanding of what’s going on in the background when attempting to run a plugin. Using the Volatility3 documentation, let’s try to write a simple sample of code to run the PsList plugin with volatility3 as a library.
Constructing and running a plugin Link to heading
In this section we will take a look at how we can run the PsList pluging using volatility3 as a library. Let’s first take a look at the following code sample :
import volatility3.framework
from volatility3.framework import contexts
from volatility3 import plugins
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
volatility3.framework.require_interface_version(2, 0, 0)
ctx = contexts.Context() # Construct a blank context
failures = volatility3.framework.import_files(plugins, True) #Load the framework plugins
if failures:
logger.info(f"Some volatility3 plugin couldn't be loaded : {failures}")
else:
logger.info(f"Plugins are loaded without failure")
plugin_list = volatility3.framework.list_plugins()
logger.info(plugin_list)
OUTPUT :
INFO:__main__:Plugins are loaded without failure
INFO:__main__:
{
'windows.statistics.Statistics': ,
'timeliner.Timeliner': ,
'windows.pslist.PsList': ,
...
}
In this first step, we have constructed a blank context and loaded the native framework plugins. Next, we need to choose which plugin we want to run and construct the appropriate context based on its requirements. For this, I propose we create a function which can construct a simple plugin with no specific arguments.
1 def build_context(image_name ,context, base_config_path, plugin):
2 available_automagics = automagic.available(context)
3 plugin_config_path = interfaces.configuration.path_join(base_config_path, plugin.__name__)
4 automagics = automagic.choose_automagic(available_automagics, plugin)
5 context.config['automagic.LayerStacker.stackers'] = automagic.stacker.choose_os_stackers(plugin)
6 context.config['automagic.LayerStacker.single_location'] = "file://" + os.getcwd() + "/" + image_name
7 constructed = construct_plugin(context, automagics, plugin, base_config_path, None, None)
8 return constructed
Let’s decompose the function :
- The image_name : The name or relative path of the memory image you want to analyse.
- The context : The object in which the memory context, plugin configuration and requirements will be stored.
- The base_config_path : The path within the context’s config containing the plugin’s configuration. By default, this value is set to “plugins”
- The plugin : The plugin object choosen from the list of plugins.
Volatility3 is using what is called “automagics” to help with plugin construction. Indeed it is helping a lot to automatically determine specific stack layers. Line 2, 4 and 5 of the code are determining what automagics are available, choose the automagics that apply to the operating system and update the context configuration.
Line 6 is simply giving the memory image location to the context configuration.
Once the context is ready, we can construct the plugin using the framework built-in function “construct_plugin”. This function will check that all the plugin’s requirements are fulfilled and if so, will return the constructed plugin (which is an object). The sample code and output below allows us to see a bit more of what happened behind the scene and construct the ‘PsList’ plugin. Notice that our logger also displays the volatility3 logs !
import volatility3.framework
from volatility3.framework import contexts, interfaces
from volatility3 import plugins
from volatility3.framework import automagic
from volatility3.framework.plugins import construct_plugin
import logging, os
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
volatility3.framework.require_interface_version(2, 0, 0)
def build_context(image_name ,context, base_config_path, plugin):
available_automagics = automagic.available(context)
logger.info(f"Available automagics : {available_automagics}")
plugin_config_path = interfaces.configuration.path_join(base_config_path, plugin.__name__)
automagics = automagic.choose_automagic(available_automagics, plugin)
context.config['automagic.LayerStacker.stackers'] = automagic.stacker.choose_os_stackers(plugin)
context.config['automagic.LayerStacker.single_location'] = "file://" + os.getcwd() + "/" + image_name
constructed = construct_plugin(context, automagics, plugin, base_config_path, None, None)
logger.info(f"Contructed plugin : {constructed}")
return constructed
ctx = contexts.Context() # Construct a blank context
failures = volatility3.framework.import_files(plugins, True) #Load the framework plugins
if failures:
logger.info(f"Some volatility3 plugin couldn't be loaded : {failures}")
else:
logger.info(f"Plugins are loaded without failure")
plugin_list = volatility3.framework.list_plugins()
base_config_path = "plugins"
constructed = build_context("Triage-Memory.mem", ctx, base_config_path, plugin_list['windows.pslist.PsList'])
logger.info(f"Context configuration {ctx.config}")
OUTPUT :
INFO:__main__:Plugins are loaded without failure
INFO:__main__:Available automagics :
[volatility3.framework.automagic.symbol_cache.SymbolBannerCache object at 0x1052d9e20,
volatility3.framework.automagic.mac.MacBannerCache object at 0x1052d9eb0,
volatility3.framework.automagic.linux.LinuxBannerCache object at 0x1052d9f40,
volatility3.framework.automagic.construct_layers.ConstructionMagic object at 0x1063a6340,
volatility3.framework.automagic.stacker.LayerStacker object at 0x1063b4280,
...
]
INFO:volatility3.framework.automagic:Detected a windows category plugin
INFO:volatility3.framework.automagic:Running automagic: ConstructionMagic
INFO:volatility3.framework.automagic:Running automagic: LayerStacker
INFO:volatility3.framework.automagic:Running automagic: WinSwapLayers
INFO:volatility3.framework.automagic:Running automagic: KernelPDBScanner
INFO:volatility3.framework.automagic:Running automagic: KernelModule
INFO:__main__:Contructed plugin : volatility3.plugins.windows.pslist.PsList object at 0x1063c1ee0
INFO:__main__:Context configuration {
"automagic.LayerStacker.single_location": "file:///REDACTED/REDACTED/REDACTED/REDACTED/REDACTED/REDACTED/Triage-Memory.mem",
"automagic.LayerStacker.stackers": [
"AVMLStacker",
"LimeStacker",
"Elf64Stacker",
"QemuStacker",
"WindowsCrashDumpStacker",
"VmwareStacker",
"WindowsIntelStacker"
],
"plugins.PsList.dump": false,
"plugins.PsList.kernel": "kernel",
"plugins.PsList.kernel.class": "volatility3.framework.contexts.Module",
"plugins.PsList.kernel.layer_name": "layer_name",
"plugins.PsList.kernel.layer_name.class": "volatility3.framework.layers.intel.WindowsIntel32e",
"plugins.PsList.kernel.layer_name.kernel_virtual_offset": 272678925664256,
"plugins.PsList.kernel.layer_name.memory_layer": "memory_layer",
"plugins.PsList.kernel.layer_name.memory_layer.class": "volatility3.framework.layers.physical.FileLayer",
"plugins.PsList.kernel.layer_name.memory_layer.location": "file:///REDACTED/REDACTED/REDACTED/REDACTED/REDACTED/REDACTED/Triage-Memory.mem",
"plugins.PsList.kernel.layer_name.page_map_offset": 1601536,
"plugins.PsList.kernel.layer_name.swap_layers": true,
"plugins.PsList.kernel.layer_name.swap_layers.number_of_elements": 0,
"plugins.PsList.kernel.offset": 272678925664256,
"plugins.PsList.kernel.symbol_table_name": "symbol_table_name1",
"plugins.PsList.kernel.symbol_table_name.class": "volatility3.framework.symbols.windows.WindowsKernelIntermedSymbols",
"plugins.PsList.kernel.symbol_table_name.isf_url": "file:///REDACTED/REDACTED/REDACTED/REDACTED/python/site-packages/volatility3/symbols/windows/ntkrnlmp.pdb/2E37F962D699492CAAF3F9F4E9770B1D-2.json.xz",
"plugins.PsList.kernel.symbol_table_name.symbol_mask": 0,
"plugins.PsList.physical": false,
"plugins.PsList.pid": []
}
Once our plugin is constructed, we can run it. However, we need to specify to volatility how we want to render the output. The framework is providing us with multiple renderers. Let’s use the “pretty text renderer” that we have used with the CLI at the begining of the blog and run the plugin.
import volatility3.framework
from volatility3.framework import contexts, interfaces
from volatility3 import plugins
from volatility3.framework import automagic
from volatility3.framework.plugins import construct_plugin
from volatility3.cli import text_renderer
import logging, os
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
volatility3.framework.require_interface_version(2, 0, 0)
def build_context(image_name ,context, base_config_path, plugin):
available_automagics = automagic.available(context)
plugin_config_path = interfaces.configuration.path_join(base_config_path, plugin.__name__)
automagics = automagic.choose_automagic(available_automagics, plugin)
context.config['automagic.LayerStacker.stackers'] = automagic.stacker.choose_os_stackers(plugin)
context.config['automagic.LayerStacker.single_location'] = "file://" + os.getcwd() + "/" + image_name
constructed = construct_plugin(context, automagics, plugin, base_config_path, None, None)
return constructed
ctx = contexts.Context() # Construct a blank context
failures = volatility3.framework.import_files(plugins, True) #Load the framework plugins
if failures:
logger.info(f"Some volatility3 plugin couldn't be loaded : {failures}")
else:
logger.info(f"Plugins are loaded without failure")
plugin_list = volatility3.framework.list_plugins()
base_config_path = "plugins"
constructed = build_context("Triage-Memory.mem", ctx, base_config_path, plugin_list['windows.pslist.PsList'])
if constructed:
result = text_renderer.PrettyTextRenderer().render(constructed.run())
OUTPUT :
INFO:__main__:Plugins are loaded without failure
INFO:volatility3.framework.automagic:Detected a windows category plugin
INFO:volatility3.framework.automagic:Running automagic: ConstructionMagic
INFO:volatility3.framework.automagic:Running automagic: LayerStacker
INFO:volatility3.framework.automagic:Running automagic: WinSwapLayers
INFO:volatility3.framework.automagic:Running automagic: KernelPDBScanner
INFO:volatility3.framework.automagic:Running automagic: KernelModule
Formatting...
| PID | PPID | ImageFileName | Offset(V) | Threads | Handles | SessionId | Wow64 | CreateTime | ExitTime | File output
* | 4 | 0 | System | 0xfa8003c72b30 | 87 | 547 | N/A | False | 2019-03-22 05:31:55.000000 | N/A | Disabled
* | 252 | 4 | smss.exe | 0xfa8004616040 | 2 | 30 | N/A | False | 2019-03-22 05:31:55.000000 | N/A | Disabled
* | 332 | 324 | csrss.exe | 0xfa80050546b0 | 10 | 516 | 0 | False | 2019-03-22 05:31:58.000000 | N/A | Disabled
* | 372 | 364 | csrss.exe | 0xfa800525a9e0 | 11 | 557 | 1 | False | 2019-03-22 05:31:58.000000 | N/A | Disabled
* | 380 | 324 | wininit.exe | 0xfa8005259060 | 3 | 78 | 0 | False | 2019-03-22 05:31:58.000000 | N/A | Disabled
* | 416 | 364 | winlogon.exe | 0xfa8005268b30 | 3 | 110 | 1 | False | 2019-03-22 05:31:58.000000 | N/A | Disabled
* | 476 | 380 | services.exe | 0xfa8005680910 | 12 | 224 | 0 | False | 2019-03-22 05:31:59.000000 | N/A | Disabled
...
Going further
Great, we now know how to run a simple plugin with no specfic configuration and a default text renderer. In this section, we'll try to dump a specific process using the capabilities of the PsList plugin.
If you take a look at the context configuration displayed earlier, you'll notice that some options about the PsList plugin are set by default. What if we want to try to dump a specific process ? With the volatility3 CLI, the command is the following :
» vol -f Triage-Memory.mem windows.pslist --pid 252 --dump k1nd0ne@MacBook-Pro-de-Felix
Volatility 3 Framework 2.0.1
Progress: 100.00 PDB scanning finished
PID PPID ImageFileName Offset(V) Threads Handles SessionId Wow64 CreateTime ExitTime File output
252 4 smss.exe 0xfa8004616040 2 30 N/A False 2019-03-22 05:31:55.000000 N/A pid.252.0x48430000.dmp
The arguments we specified are the PID, which is the process ID we want to filter, and the DUMP option, which means that we want to dump the list of the filtered processes.
If we take a look at the plugin context configuration, the entries reponsible are the following :
"plugins.PsList.pid": []
"plugins.PsList.dump": false In our code, we just have to clarify those two values:
...
ctx.config["plugins.PsList.pid"] = [252]
ctx.config["plugins.PsList.dump"] = True
constructed = build_context("Triage-Memory.mem", ctx, base_config_path, plugin_list['windows.pslist.PsList'])
OUTPUT :
INFO:__main__:Plugins are loaded without failure
INFO:volatility3.framework.automagic:Detected a windows category plugin
INFO:volatility3.framework.automagic:Running automagic: ConstructionMagic
INFO:volatility3.framework.automagic:Running automagic: LayerStacker
INFO:volatility3.framework.automagic:Running automagic: WinSwapLayers
INFO:volatility3.framework.automagic:Running automagic: KernelPDBScanner
INFO:volatility3.framework.automagic:Running automagic: KernelModule
Formatting...
| PID | PPID | ImageFileName | Offset(V) | Threads | Handles | SessionId | Wow64 | CreateTime | ExitTime | File output
* | 252 | 4 | smss.exe | 0xfa8004616040 | 2 | 30 | N/A | False | 2019-03-22 05:31:55.000000 | N/A | pid.252.0x48430000.dmp
This code will be interpreted without error. However you’ll never find your process on your drive ! Why ? Well, in the “build_context” function, I have called the volatility3 framework’s “construct_plugin” function with no File Handler Interface. By default, the volatility3 framework lets you decide how to handle files by rewritting the methods (open, write, …). In this example, we want the file to be directly written in the current directory from where the script is executed. The following code is using the CLIDirectFileHandler from the native framework with a little bit of tweaking. Once integrated into our code, the final result looks like this :
import volatility3.framework
from volatility3.framework import contexts, interfaces
from volatility3 import plugins
from volatility3.framework import automagic
from volatility3.framework.plugins import construct_plugin
from volatility3.cli import text_renderer
import logging, os, io, tempfile
def file_handler(output_dir):
class CLIFileHandler(interfaces.plugins.FileHandlerInterface):
"""The FileHandler from Volatility3 CLI"""
def _get_final_filename(self):
"""Gets the final filename"""
if output_dir is None:
raise TypeError("Output directory is not a string")
os.makedirs(output_dir, exist_ok = True)
pref_name_array = self.preferred_filename.split('.')
filename, extension = os.path.join(output_dir, '.'.join(pref_name_array[:-1])), pref_name_array[-1]
output_filename = f"{filename}.{extension}"
counter = 1
if os.path.exists(output_filename):
os.remove(output_filename)
return output_filename
class CLIDirectFileHandler(CLIFileHandler):
"""We want to save our files directly to disk"""
def __init__(self, filename: str):
fd, self._name = tempfile.mkstemp(suffix = '.vol3', prefix = 'tmp_', dir = output_dir)
self._file = io.open(fd, mode = 'w+b')
CLIFileHandler.__init__(self, filename)
for item in dir(self._file):
if not item.startswith('_') and not item in ['closed', 'close', 'mode', 'name']:
setattr(self, item, getattr(self._file, item))
def __getattr__(self, item):
return getattr(self._file, item)
@property
def closed(self):
return self._file.closed
@property
def mode(self):
return self._file.mode
@property
def name(self):
return self._file.name
def close(self):
"""Closes and commits the file (by moving the temporary file to the correct name"""
# Don't overcommit
if self._file.closed:
return
self._file.close()
output_filename = self._get_final_filename()
os.rename(self._name, output_filename)
return CLIDirectFileHandler
def build_context(image_name ,context, base_config_path, plugin):
available_automagics = automagic.available(context)
plugin_config_path = interfaces.configuration.path_join(base_config_path, plugin.__name__)
automagics = automagic.choose_automagic(available_automagics, plugin)
context.config['automagic.LayerStacker.stackers'] = automagic.stacker.choose_os_stackers(plugin)
context.config['automagic.LayerStacker.single_location'] = "file://" + os.getcwd() + "/" + image_name
constructed = construct_plugin(context, automagics, plugin, base_config_path, None, file_handler(os.getcwd()))
return constructed
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
volatility3.framework.require_interface_version(2, 0, 0)
ctx = contexts.Context() # Construct a blank context
failures = volatility3.framework.import_files(plugins, True) #Load the framework plugins
if failures:
logger.info(f"Some volatility3 plugin couldn't be loaded : {failures}")
else:
logger.info(f"Plugins are loaded without failure")
plugin_list = volatility3.framework.list_plugins()
base_config_path = "plugins"
ctx.config["plugins.PsList.pid"] = [252]
ctx.config["plugins.PsList.dump"] = True
constructed = build_context("Triage-Memory.mem", ctx, base_config_path, plugin_list['windows.pslist.PsList'])
if constructed:
result = text_renderer.PrettyTextRenderer().render(constructed.run())
There we go, when interpreted, this script will dump the process with PID 252 using the PsList plugin. Now that you know more about how to use the volatility3 framework libraries, you can imagine any application you want to a real case scenario! Conclusion
I hope you now have a better understanding of how the volatility3 framework is working behind the scene, how easy it is to exploit and integrate it inside your investigation tools. You can of course write your own renderers to directly inject the results inside a database or any format that suite your needs. VolWeb is using volatility3 as a library and this article is directly linked to the latest release exploiting this capability. You can check VolWeb by clicking the link below.
This article is written from my own understanding of this subject and any critics you may have are welcomed to make this article evolve. You can contact me at felix.guyard@forensicxlab.com.
References Link to heading
- Volatility3 Documentation : https://volatility3.readthedocs.io/en/stable/using-as-a-library.html
- Volatility3 Project : https://github.com/volatilityfoundation/volatility3