Abstract Link to heading

Being interested in memory forensic for a while now I have learned a lot about the Volatility framework. This article will introduce volatility3 core components and focus on kernel symbols. Next, I will explain the steps I took to generate a lot of MacOs SymbolTables. Finally you will be able to retrieve those SymbolsTables directly from github. The final goal is to create a public repository like windows to automatically identify mac os system version and directly download the associated SymbolTables.

Volatility Link to heading

Memory analysis is focusing on the extraction of evidence by exploiting the Random Access Memory (abbreviated to RAM) of a given system. Volaltility is the best state of the art framework to perform digital forensic on RAM after its acquisition. Volatility has become the world’s most widely used memory forensics platform and its python3 version is currently under active development.

Volatility main components Link to heading

When doing memory analysis with Volatility3, the memory is split into three major components :

• Memory Layers : This component allows correct memory mapping and address translation.
• Templates and Objects : A template is the representation of what we know about the structure of the final object we want to extract and explore from the memory layer to perform memory forensic.
• Symbol Tables :
  • When a program is compiled, it is aware of its own structures and how they are stored. This means that when doing offline memory analysis, volatility needs to be aware of the context of the program that were executed.
  • When the compilation of a program is performed, debugging symbols are produce alongside it. A symbol can be viewed as an address associated to a template.
  • We can picture the set of debugging symbols produced at compilation being equal to one SymbolTable which is a JSON structure understandable by volatility. The more SymbolTable volatility possesses the more symbols and templates it can extract from memory.
  • The representation of all of the SymbolTables retrieved by volatility is called the SymbolSpace. The SymbolSpace is stored within the context of the volatility3 framework. When executing volatility3 on a memory dump, it is going to build its context and SymbolSpace from the user’s provided SymbolTables.

As a memory forensics investigator, our goal will be first to extract the debugging symbols generated by the kernel when compiled which will allow us to identify how the system’s essential structures are stored in memory. The majority of basic Volatility3 plugging are using the kernel symbols.

For Windows memory analysis, Volatility is going to build the context using Windows Program Database (PDB) files which are directly fetched from microsoft website and translated into SymbolTables if not already present locally.

For Linux, the task is more complicated. Indeed Linux can be declined into a lot of distributions which are all having a different kernel versions which means different SymbolSpace for each version. If you want to perform memory forensic on a random linux you need to extract the kenel debugging symbols of the target which are not always easy to find according to the distribution.

For Mac the task is similar to Linux, however there is only one single distribution. Mac is tracking the system versions and build numbers. For each system version and build number there is different kernel version therefore potentially different kernel debugging symbols.

The first step to even be able to analyse a memory dump is being able to build the right context. Any incident response team or an investigator would gain a fair amount of time not bothering generating SymbolTables. In the next section we will go deeper on how to generate Intermediate Symbol Format (ISF) file for mac and my journey into the automation of the process for a far amount of mac system versions.

MacOs Kernel Debug Kit Link to heading

If you have an Apple account, you can find what Apple calls “Kernel Debug Kits” (KDK). For one specific mac os version and build number, the package contains development & debug versions of the macOS kernel. These files contain full symbolic information, unlike the equivalent files in a normal macOS installation. This is exactly what we need to create our SymbolTables. Mac as well as Linux are using DWARF file to represent the kernel with debugging symbols. The VolatilityFoundation created a go program call dwarf2json to parse the appropriate DWARF files into a JSON ISF file to build the SymbolTables. The Kernel Debugging Kits are available here.

Therefore here are the goals :

• Download all KDK available from Apple’s website.
• Identify the kernel symbols for each version.
• Generate the ISF files.
• Share and maintain.

Automation Link to heading

The first step was to download all the KDK which are dmg images from Apple’s developper website. We can identify that there is 241 available KDK that are covering mac os version 10.4.11_build_8s216 to 12.3_Build_StarESeed21E5212f.

I have decided to pick one KDK per major system version to identify the differences in the file structures and where the kernel debugging symbols are located. From there we can begin the script.

The major steps are for each KDK :

• Extract the pkg file from the dmg image.
• Unpack the pkg file.
• Find the kernel debugging symbols DWARF files.
• Create the IFS JSON file

I chose to use python3 for the implementation :

import subprocess
import argparse
import logging
import os, os.path
import re

def find_and_generate_kernel_symbols(dir_path, kernel_version):
    """Generate the ISF file for the kernel DWARF files found"""
    cmd = ['./dwarf2json', 'mac']
    for dirpath, dirnames, files in os.walk(dir_path):
        for filename in files:
            if "kernel" == filename or "mach_kernel" == filename:
                todo = os.path.join(dirpath, filename)
                cmd.append('--macho-symbols')
                cmd.append(todo)
    logger.info(f"Generating volatility ISF for {kernel_version} ")
    with open("ISF/"+kernel_version[0]+".json", "a") as outfile:
        try:
            subprocess.run(cmd, stdout=outfile)
        except:
            logger.error(f'Could not generate ISF for {todo}')
    try:
        subprocess.check_output(['tar', '-cJf', "ISF/"+kernel_version[0]+".json.xz", "ISF/"+kernel_version[0]+".json" ])
    except:
        logger.error(f'Could not compress ISF into an xz archive')

def dir_path(string):
    """Check if the directory passed is indeed a directory"""
    if os.path.isdir(string):
        return string
    else:
        raise NotADirectoryError(string)

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description = "Generate all volatility symbole files for the different macOs versions")
    parser.add_argument('--path', type=dir_path)

    """init"""
    args = parser.parse_args()
    logging.basicConfig(level=logging.INFO)
    logger = logging.getLogger(__name__)
    logger.info('Started')
    src_dir = args.path
    reg_exp = r'[k|K]ern[e|a]l_[d|D]ebug_[K|k]it_(.*)\.dmg'

    """routine"""
    for dmg_image in os.listdir(src_dir):
        if "DS_Store" in dmg_image:
            continue
        kernel_version = re.findall(reg_exp, dmg_image)
        logger.info(f'Kernel version : {kernel_version} ')
        logger.info(f'Extracting {dmg_image}')
        try:
            subprocess.check_output(['7z', 'x', src_dir+"/"+dmg_image])
        except subprocess.CalledProcessError as err:
            logger.error(f'Could not extract pkg from : {dmg_image} : {err}')
            continue
        try:
            kernel_debug_kit_path = dir_path("KernelDebugKit")
            find_and_generate_kernel_symbols(kernel_debug_kit_path, kernel_version)
        except NotADirectoryError:
            pass
            try:
                kernel_debug_kit_path = dir_path("Kernel Debug Kit")
                logger.info(f'Extracting pkg...')
                try:
                    subprocess.check_output(['pkgutil', '--expand-full', kernel_debug_kit_path+"/KernelDebugKit.pkg", "KernelDebugKit"])
                    subprocess.check_output(['rm', '-rf', kernel_debug_kit_path])
                    kernel_debug_kit_path = "KernelDebugKit"
                    find_and_generate_kernel_symbols(kernel_debug_kit_path, kernel_version)
                except subprocess.CalledProcessError as err:
                    logger.error(f'Could not extract pkg from : {kernel_debug_kit_path} : {err}')
                    pass

            except NotADirectoryError:
                logger.error(f'Could not find the KernelDebugKit directory for {dmg_image}')
                pass
        os.system("rm -rf KernelDebugKit")

This script is using the compiled version of dwarf2json located alongside the script. Once this code is executed on the directory containing all the dmg images, the ISF files are extracted from the multiple kernel debug symbols into a “.xz” archive.

This archive type is understandable by volatility3 when building the context.

To conclude, I have learned a lot on how volatility3 works while trying to extract kernel symbols from the apple’s KDKs. The next steps is to test those symbols and integrate them to VolWeb. Mac memory forensic is also for me a good way to participate to the creation of modules for volatility3.

This article and the code may evolve with time if new discoveries are made. The community is invited to test those symbols on memory dumps. In a future blog post, I will focus on the existing ways to dump memory on a modern mac system and try to provide an automated tool or a new way to perfom this task.

This blog post is written from my own understanding of memory forensic. Don’t hesitate to reach me at felix.guyard@forensicxlab.com and share your suggestions to make this article more complete.