Friday, May 7, 2021

Highlighting Research from the Next Generation of Memory Forensics Practitioners

Nearly 2 years ago, we published a blog post about our collaboration with Dr. Golden G. Richard III at the Louisiana State University (LSU) Center for Computation and Technology (CCT). We are very happy to report that this collaboration is still going strong, has been a huge success, and has helped the Applied Cybersecurity Lab at LSU flourish. The students from LSU who have finished their studies are now making a real impact in our industry, and those currently pursuing their degrees are continuing to push the state of the art in memory forensics and malware detection. 

Given that another semester has just wrapped up, and that the LSU Office of Research recently published a detailed article about our collaboration, we decided that it was time for us to write our own acknowledgement of these students and their efforts. As you might imagine, many of the students have focused their research on memory forensics and its use in real-world DFIR workflows. These research efforts have been strongly focused on "gaps" in current analysis techniques; significant improvements of existing techniques; and efforts to classify and test the accuracy and reliability of existing tools. For the remainder of this post, we will highlight several of these efforts, as well as introduce the students involved.

Reliability of DFIR Tools

The efforts aimed at the reliability of DFIR tools began with the creation of systems for targeted fuzzing of DFIR frameworks. The first publication from this research was the Gaslight fuzzing architecture aimed at memory forensic frameworks. Gaslight was first presented at DFRWS 2017, and it showcased the ability to efficiently fuzz only the portions of a memory sample analyzed during common investigations. It successfully found numerous error conditions in Volatility 2 that have since been patched, making the framework more reliable in the face of smearing.

A second version of Gaslight was then developed that greatly improved performance and scalability. This was targeted at the Sleuthkit Framework and found several inputs that would cause the library and associated programs to crash or exhaust available resources. This was published in Computers and Security (COSE) in 2020 and served as the basis of Shravya Paruchuri's Master's thesis.

The final iteration of Gaslight involved modifying its architecture to support distributed and high performance processing. This effort was also aimed at Volatility 2 and found many plugin code paths that did not properly account for smear, and that were not successfully triggered during testing with the first architecture version. This project was performed in coordination with the High Performance Computing center at LSU, and their allowance of abundant HPC resources allowed for millions of plugin fuzzing runs. This work was performed by Arian Shahmirza and led to successful completion of her Master's Degree titled "High Performance Fuzz Testing of Memory Forensics Frameworks". 

Beyond fuzzing, there was also a major effort to automate testing the accuracy of memory analysis frameworks. Given that the data structures in memory samples are a constantly moving target due to new operating system and application versions, it is imperative that analysis tools are able to keep up with the rapid changes. This is currently an extremely time-consuming and manual process, so research was performed to automate the analysis and comparison of different module versions to provide for an efficient and tested workflow. Ryan Maggio led this effort, and it contributed to his successful Ph.D. defense. The paper describing this research was accepted at DFRWS 2021 and will be presented this summer. We are also very excited for Ryan's future in the DFIR industry, as he recently accepted an offer to join the research team at MIT's Lincoln Laboratory after graduation.

Bringing Emulation to Memory Forensics

The second portion of Ryan's Ph.D. work involved another major research effort performed by our team and the LSU students: bringing emulation to the forefront of memory analysis research. Existing methods for analyzing malicious code in memory, such as shellcode and API hook stubs, require an expert investigator to manually reverse engineer each code block. Given that an average modern memory sample has thousands of such hooks, most of which are benign, this is no longer a feasible approach. 

To alleviate this issue, research was performed that explored the use of emulation of code inside memory samples to generate automated decisions for commonly seen code blocks, stubs, and patterns. This was a group research effort and led to the creation of HookTracer, a system we built on top of Unicorn that integrates directly with Volatility. While Unicorn provides a bare emulator, HookTracer adds Windows-specific functionality to the emulation environment. This includes support for FS/GS access; per-process address spaces; recording of API calls and parameters for functions of interest; and the ability to record all basic blocks and VADs traversed by particular code paths. All of these features are directly accessible to Volatility plugins through the new HookTracer API.

This research and development of HookTracer led to several peer-reviewed publications, as well as contributed to several successful M.S. and Ph.D. defenses. The first of these was presented at DFRWS 2019 and focused on the automated analysis of API hooks. As shown in this paper, default Windows installations with no 3rd-party products and no malware present still have thousands of active API hooks to support backwards compatibility and other related features. Through the development of a new Volatility plugin that leveraged HookTracer's APIs, all of these benign hooks were successfully filtered, while malicious ones planted by malware were successfully reported.

The second effort focused on message hooks, which are often abused by malware to perform keylogging, copy/paste buffer snooping, and inject code into remote processes. To remove the burden of manual analysis of such hooks, a new HookTracer plugin was developed that could automatically determine if a message hook was legitimate or benign. Furthermore, for all Windows APIs called by such hooks, the plugin is able to report the parameters passed to functions of interest. The following picture shows this plugin in action against a malware sample from the infamous Turla group:


In the output, you can see that HookTracer has successfully emulated the malware's message hook. This includes revealing that the malware recorded the current timestamp, application window title, current key pressed, and current working directory. The malware then writes these to a file on disk (msimm.dat). In many ways, this aspect of HookTracer can be thought of as a sandbox-like execution environment for code in memory. The full details of this plugin and its analysis capabilities can be found in the research team's paper published in COSE in 2020.

There was also a research effort by Austin Sellers to automate the analysis of networking APIs called by memory resident code. Network-based IOCs, such as IP addresses and hostnames, are used throughout all phases of the DFIR workflow. Existing memory analysis techniques enhance this information by automatically determining which process(es) have communicated with the hosts/IPs of interest. These techniques then leave the investigator to manually find the code region(s) that actually perform the communication, which is a crucial task when the ability to decode encrypted packet data is needed or when the C2 protocol needs to be reverse engineered. Austin's work sought to automate the location of these code regions by finding the places where networking APIs are called and extracting the parameters to the APIs. This work was the foundation of his Master's thesis. Since graduation, Austin has worked with several of our team members at Volexity as a software engineer helping to build commercial memory analysis capabilities.

We also have a fourth effort related to HookTracer that is currently being finished and will soon be sent for publication. Given the double-blind nature of most academic security conferences and journals, we cannot say too much yet except that it will provide another great leap forward for the field. We expect a follow-on blog post once it is finished.

Finally, HookTracer was developed for Volatility 2 given that this project was started several years ago. As Volatility 3 stabilizes and approaches feature parity with Volatility 2, we plan to port HookTracer to the new version so that the entire community can build on it and benefit from its use.

Memory Analysis Gaps

Although the memory forensics field has seen substantial research efforts over the last decade, the sheer number of relevant operating systems, applications, and runtime environments means that many research efforts are still needed. The combined research group aimed to address several of the most interesting and relevant of these research "gaps".

This began with Nathan Lewis publishing a paper at DFRWS 2018 on his efforts to build memory analysis capabilities for Linux systems powered by version 1 of the Windows Subsystem for Linux (WSL).  Before this research, all of the WSL-controlled Linux processes, file descriptors, network activity, and other artifacts were not accessible in a structured manner. This led to Volatility plugins being 'blind' to WSL activity. Given the technical effort and usefulness of this research, Nathan's paper was chosen for Best Student Paper award at DFRWS.

There were also efforts to modernize userland analysis of macOS systems. The last major research of the userland runtimes for Swift and Objective-C was published in 2016 and has fallen behind modern versions. To update the support as well as add new analysis features, Ph.D student Modhuparna Manna worked on two major efforts. The first was the analysis of the macOS page queues, which "hide" many present pages in memory by marking them as invalid, even though they are actually in a given memory sample. Without the analysis of the queues, many pages in process memory are unavailable to analysis plugins and results are often very limited. Recovering these pages and making them accessible to Volatility plugins was the subject of a paper that the group published at DFRWS 2020. Modhu also updated the existing Volatility plugins for analysis of the macOS userland runtimes to support modern versions as well as recover several new artifact types. These are covered in a paper that has been accepted by the Digital Investigation journal and is awaiting publication. We will update this blog post with the link to the paper after publication.

Analysis of the Android userland runtimes was the focus of Sneha Sudhakaran's research, in collaboration with her co-advisor Aisha Ali-Gombe, a faculty member at Townson University. Given the popularity of Android devices, along with the amount of malware targeting the platform, this research is highly relevant to modern investigations. The results of this research were novel methods for analysis and recovery of Android application activity. There were several publications as a result of this effort, the first being AmpleDroid for analysis of Large Object Files. The second, DroidScraper, was published at the top-tier academic security conference RAID and focused on structured recovery of application data and artifacts. Sneha is currently working on a third related publication as she wraps up her Ph.D. studies.

Scholarships for Service

Before closing, we wanted to highlight that due to the efforts of Dr. Richard and his colleagues Dr. Ram, Dr. Sun, Dr. Mahmoud and Dr. Peng, LSU is now a part of the National Science Foundation's Scholarship for Service (SFS) program. The full details are on the corresponding LSU webpage, but briefly, this scholarship provides up to 3 years of full support to students. This support includes tuition; a very nice stipend ($25,000/year for undergrads and $34,000/year for graduate students); funding for professional training and certification; and internship opportunities. 

In exchange, students agree to work for a US government institution for the number of the years that the scholarship was used. Past examples of institutions students work for/with include 3-letter agencies, energy-related departments, and other high-profile organizations that operate major components of the country. These are jobs that would be of interest to many Computer Science graduates, and the SFS program provides a direct path to them.

If you know someone who may be interested in the SFS program, please see the link above. We are hoping that it will continue to attract motivated students to the LSU research group, and that we can continue to help mold the future practitioners and researchers of the DFIR industry. 

We also would like to note that no one on the Volatility Team has any involvement with selecting or screening students through this process. If you have any questions, the best place is the previously linked page.

Final Thoughts

In closing, we would again like to acknowledge the great amount of effort put forth by the students in the LSU research group. Please reach out to them if you have questions on their research, or if your organization may be hiring in the future. As a final note, we would also like to congratulate a current Master's student in the group, Raphaela Mettig, for excelling during the intern interview process at Tesla and being awarded a security team internship for this summer.

-- The Volatility Team

Tuesday, January 26, 2021

Malware and Memory Forensics Training Goes Virtual!


We are very excited to announce that our popular Malware and Memory Forensics with Volatility training is now available in a self-paced, online format!

Brought to you by members of the Volatility Team, this course gives you the opportunity to learn directly from the people behind the research and development of Volatility, and it offers you a chance to support our ongoing efforts.

The Course


Our course provides a deep examination of Windows internals, malware operations, attacker toolkits, DFIR workflows, and how memory forensics can be leveraged throughout all of your investigations. The end result of the lectures on these topics is a complete understanding of how memory analysis tools operate, along with the traces left behind by malicious actors and applications. You will gain knowledge that applies to all memory analysis investigations and frameworks, both now and in the future.

The Labs


You will get significant hands-on experience using Volatility against Windows, Linux, and MacOS samples during the training labs. These labs are mirrored directly to match our real-world investigations (1, 2, 3, 4, 5), and they are constantly updated as new threats emerge. After each lab, there is a pre-recorded walkthrough that shows you the precise steps taken to solve the lab as well as explains the rationale behind the workflow used. A written version of the walkthrough is also provided in the Lab Guide.

The Content


Signing up for the course provides you with 4 months of access to all material, including the pre-recorded lectures, demos, and lab walkthroughs. You will also be given a trial version of Surge Collect Pro, which provides reliable and secure memory acquisition across Windows, Linux, and Mac systems. For support, we have a dedicated channel on our Slack server and periodic office hours via Zoom; students can directly ask us questions about any portion of the course, with optional screen sharing. After the course ends, you may retain access to the written materials (Art of Memory Forensics, course slides, lab guide) and the course virtual machine. You are also welcome to stay in the student-only Slack channel and join our alumni-only mailing list.

For those wondering about Volatility 2 vs Volatility 3, our course currently uses Volatility 2 for demos and labs as it is the stable and fully featured version of The Volatility Framework. As mentioned previously, the skills learned in this course are transferable to any memory analysis framework – even those not based on Volatility. We plan to add training modules specific to Volatility 3 slowly over time as it stabilizes and further approaches feature- and plugin-parity with Volatility 2. In general, if you know how to use Volatility 2, then using Volatility 3 will be very simple in the future. As students of this course, you will be the first to gain access to Volatility 3 training material as it is released.

Sign Up!


You can request access to the course here. We provide discounts for military, law enforcement, and groups of students from the same organization. We also provide significant discounts to course alumni who wish to take the course again, as well as to full-time college students studying in a related field. Please inquire about these discounts if you meet the requirements. We can also accommodate private sessions for large group trainings.

We would like to thank the memory forensics community for their years of continued support. We are excited that our course is now available 24/7/365 for students around the world!

- The Volatility Team