Monday, April 25, 2016

Windows Malware and Memory Forensics Training coming to NYC, Amsterdam, and Reston!

We're excited to announce the dates and locations for three new public offerings of Windows Malware and Memory Forensics Training by The Volatility Project.

The following courses are now open for registration:
  • June 27th - July 1st in NYC
  • August 29th - September 2nd in Amsterdam
  • October 3rd - 7th in Reston/Herndon, VA
You can request an invite through our web form or contact us via voltraining @ memoryanalysis [dot] net (pgp). All three of these locations have sold out early in the past, so please contact us ASAP if you wish to attend.

Also, students in our recent Reston course wanted to share some of their feedback with you:
I thought I understood Windows internals until I attended this course. Attend if at all possible! Troy A. (Malware Analyst)

Thanks for a great training. Its one of the best (if not the best) I've been to.  Billy M. 
The course is put together in a way that all skill levels and experience will get valuable information.  Jeremy S. (Senior Research Analyst)

It is one of the greatest forensics classes I ever attended.  Ali K. (IT Security Consultant)
We look forward to meeting all the talented analysis and investigators that attend our classes!

Friday, April 8, 2016

Airbnb Donates $999 to the 2016 Volatility Plugin Contest!

Thank you to Airbnb for donating $999 to the 2016 Volatility Plugin Contest and their continued support for open source memory forensics development.

When we announced the contest a couple days ago, the cash prizes for the top three contest winners were $1500, $500, and $250. In light of Airbnb's donation, we decided to augment the values for the top three and extend the cash prizes for the 4th and 5th place contributors, resulting in not only higher rewards but more cash winners.

1st: $1800 or a free seat at Windows Malware and Memory Forensics
2nd: $800
3rd: $450
4th: $100
5th: $100

Participants who place will still receive the designated shwag (t-shirts, stickers, sync stops, books, etc). Thanks again to Airbnb for their generous donation and support!

Monday, April 4, 2016

The 2016 Volatility Plugin Contest is now live!

This is a quick update to announce that the 2016 Volatility Plugin contest is now live and accepting submissions until October 1st. Winners of this year's contest will be receiving over $2,000 in cash prizes as well as plenty of Volatility swag (t-shirts, stickers, mugs, sync stops, etc).

The purpose of the contest is to encourage open memory forensics research and development. It is a great opportunity for students to become familiar with memory forensics, develop a master's thesis or PhD project, as well as gain experience that will be very desirable by future employers. For those already in the field, submitting to the contest is a great way to gain experience and visibility in the memory forensics community. After the contest is over we promote the work in our conference presentations, blogs, and social media.

If you are looking for inspiration or to see the past winners, please check out the pages from 20132014, and 2015. You will find projects that allow for inspection of virtual machines guests from the view of the host, recovery of in-memory browser artifacts, methods to detect stealthy rootkits, and much more.

If you have any questions please feel free to reach out to us. 

We are looking forward to another year of innovative open source research!

Monday, November 30, 2015

Guest Post: Martin Korman (VolatilityBot - An Automated Malicious Code Dumper)

This is a guest post from Martin Korman, author of VolatilityBot.

Lately, I've found myself manually unpacking different versions of the same malware in order to perform static analysis with IDA and BinDiff. Therefore, I've decided to write a small system that will automate the entire process – the VolatilityBot.

How does VolatilityBot work?
  1. It executes the malware on a VM.
  2. It waits for a pre-defined period of time.
  3. It suspends the VM.
  4. It compares the snapshot to a golden image of the VM, finds new processes, injected code, loaded DLLs or Kernel Modules, dumps them from the memory and fixes the PE file in order to make static analysis easier.

All metadata is saved to a SQLite DB. Dumps are saved to a configured storage path on disk. All PE files pass a short static analysis and reports are stored as well. VolatilityBot can theoretically manage an unlimited quantity of virtual machines, depending on the performance of your host machine. 

Some core capabilities of this automation tool were designed to make it as scalable as possible and easy to work with over time:

  • The Bot-Excavator's Manager can handle an unlimited number of machines at once, all depending on the performance of the researcher's equipment.
  • Automatic "Golden Image" generation is provided in order to ease the process of creating all "Golden Image" data. Just create your configuration file, and execute the script.
  • To simplify scalability, any database supported by SQLalchemy can be set or replaced as needed. The default Bot-Excavator's backend saves data to an SQLite database.
  • Memory dumps and data from all configured post processing modules are saved to a predefined storage directory.
  • Add tags to each execution of the script for quick visual reference to information you may need later.
  • Add dynamic tags to post-processing modules. For example, for malware that loads a kernel mode driver, tag it with "loads_kmd".Volatility Bot-Excavator's modular structure makes it easy to add additional modules and post-processors that may be deemed necessary with time. 

Let's dive deeper into the architecture of VolatilityBot. The Bot-Excavator is made of four (4) major components:


1. The Manager. This is the core of the Bot-Excavator tool. The Manager executes the automatic extraction as well as the post-processing modules. This module also controls the associated machines' activity to streamline the workflow.


2. Machines Module. This is an abstract design of a research machine; it contains five functions: Revert, Start, Suspend, Clean-up and Get memory path. Each machine has a minimal python agent running, that listens and waits for a malware sample. When the sample is sent, it executes it by double clicking the sample, using an AutoIt script. 


The agent is really minimal in order to not affect the behavior of the malware in the machine. The agent does not perform any API hooking and does not control the machine in any form besides executing the malware. The machines are controlled and monitored by the manager component, which knows not to send more analyses to a machine if its busy or if it encountered any errors analyzing the previous sample.


3. Code Extractors. This component consists of a number of modules grouped together for the purpose of extracting all the different malicious code components from memory. These are separate for code injections, new processes, etc. This component is modular and allows researchers to write new code for other extractors they may need.


These are the existing modules for this component:

  • injected_code – uses the Volatility "malfind" plugin in order to find suspect memory areas, after dumping them it tries to determine if the section contains a valid PE or valid shell code. If it found a valid PE, it fixes PE the header. Either way it will extract strings and execute YARA on the section dumped from memory.
  • module_scan - Uses the Volatility "modscan" plugin to load kernel modules. It detects and dumps newly loaded kernel modules.
  • create_process_dump - Uses the Volatility "procdump" and "pslist" plugins to dump new processes created by malware. Executes YARA, and extracts malware strings.
  • create_process_dump_as - Uses the Volatility "procdump" plugin to dump address space. Executes YARA, and extracts malware strings.
  • hooks – Extract API hooks done by malware (User-mode and Kernel-mode)
4. Post-Processing Modules. The post-processing modules kick in after the extraction is complete. They are then tasked with automated actions like fixing the PE, or availing the resulting elements to static analysis, YARA scans, strings and IP address logging etc.


This module type can help with:

  • Executing the configured YARA rules on post-extraction input as defined by the researcher.
  • Extracting strings from the input defined. Other modules can be layered on top in order to extract IP addresses, URLs, etc.
  • Producing a report for static analysis with basic PE analysis of the input file.

Because the malware samples are executed in a virtual machine, in order to avoid VM detection, a few tricks were used:

  • Registry keys cleanup (all VMware stuff I don't think there is a need to describe, as there's a lot of information on the internet regarding this issue).
  • A macro that moves the mouse and executes the malware.
  • Of course, no VMware tools on the machine.

Documentation and installation instructions are available here:
https://bitbucket.org/martink90/volatilitybot_public/downloads/Documentation_Nov_2015.pdf






For any questions, feel free to reach me via twitter (@MartinKorman), or open an issue in the project's Bit Bucket. 

Friday, November 6, 2015

PlugX: Memory Forensics Lifecycle with Volatility

At OSDFCon last week, we discussed a case study showing how we identified manipulated memory artifacts in an infected environment. We were then able to rapidly introduce new capabilities to Volatility that could be used proactively in other environments.

The presentation (hosted on prezi) takes you on a tour of this activity, using a series of videos. Unfortunately, there's no audio (you should have attended the conference!) but it should be fairly self explanatory what's going on. Here's a short description of what the presentation covers:

  • A recent PlugX variant penetrated the target's network. It installed a service named RasTls for persistence and then hid the service from all live tools (WMI, Power Shell, services.msc, sc query, etc). 
  • It injected code into a process and then erased its PE header to evade detection and make dumping the PE file out of memory a bit more complicated. 
  • We identified the injected region with malfind, extracted it with Volatility, then used a hex editor and PE editor to reconstruct the PE file format. 
  • We leveraged the impscan plugin to label API calls inside IDA Pro, then reverse engineered the binary to figure out how it's rootkit component was working. It was clearly targeting the double-linked list of services in usermode space of services.exe. 
  • Next we wrote a new Volatility plugin to automatically detect unlinked services, by comparing the records in memory with the memory-resident registry hives. 

The second part of our talk covered the new features in Volatility 2.5 (including the unified output, community integration, and support for Windows 10, Mac El Capitan, and Linux kernels up to 4.2.3) and all of the submissions to this year's Volatility Plugin Contest.

You can find the slide deck for our talk here (pdf).

Thursday, October 29, 2015

Results from the 2015 Volatility Plugin Contest are in!

The competition this year was fierce! We received 12 plugins to the contest. Similar to last year, ranking the submissions was one of the hardest things we’ve had to do. Each plugin is unique in its own way and introduces a capability to open source memory forensics that didn’t previously exist. Although a few people will receive prizes for their work, the real winners in this contest are the practitioners and investigators in the community that perform memory forensics. 

Needless to say, we're very proud of everyone who submitted to the contest. 

Here are this year’s rankings:
  1. Fred House, Andrew Davis, and Claudiu Teodorescu for the shimcachemem plugin. 
  2. James Habben for the Evolve web interface to Volatility. 
  3. Philip Huppert for the VM migration address space. 
  4. Ying Li for the linux python strings and SSH agent plugins. 
  5. Adam Bridge for the NDIS packet scanning plugin. 
Here is a detailed summary of the submissions. If you have feedback for the authors, we're sure they'd love to hear your thoughts.

1st: Shimcache Memory Scan 


This plugin by FireEye and Mandiant researchers Fred House, Andrew Davis, and Claudiu Teodorescu parses the Windows Application Compatibility Database (aka, ShimCache) from the module or process memory that contain the database. In the authors' own words:

Shim cache is a highly valuable forensic artifact used to identify evidence of file execution. In addition to recording potential file executions, the cache is ordered, meaning that an analyst can identify other files that may have executed before or after a file of interest.

Most forensic tools that parse the shim cache rely on the cache stored in the Windows registry. The cache in the registry is only updated when a system is shutdown so this approach has the disadvantage of only parsing cache entries since the last shutdown. On systems that are not rebooted regularly (e.g., production servers) an analyst must either use out-of-date shim cache data or request a system reboot.

This plugin parses the shim cache directly from the module or process containing the cache, thereby providing analysts access to the most up-to-date cache. The plugin supports Windows XP SP2 through Windows 2012 R2 on both 32 and 64 bit architectures.

Fred's Twitter: @0xF2EDCA5A
Claudiu's Twitter: @cteo13

2nd: James Habben: Evolve Web Interface


This submission provides a web interface to Volatility built with AJAX, jQuery, and JSON. It allows the user to run several plugins at once and leverage the power of multi-processing. It shows the plugins' status and places the completed plugins near the top for easy access. The output is easily searchable given its sqlite3 backend and there is a "morph" option as well for highlighting interesting artifacts. In fact, users can create their own morphs using a plugin-based template system. Existing morphs include associating country codes with IP addresses and marking filenames that do not match the NSRL list. 

3rd: Philip Huppert: VM Live Migration 


This submission includes an address space plugin for accessing memory samples found in network data captured during a VMotion live migration. In particular, it currently supports analysis of samples collected during VMotion migrations between ESXi hosts. This allows an analyst to access the entire runtime state of a virtual machine as it is being transferred between two physical hosts over the network. This has a number of valuable applications toward virtual machine introspection and forensics investigations of cloud environments. 

Philip's plugin represents interesting implications toward VM's that migrate between different cloud providers or countries. Wireshark has the capability to analyze the traffic protocols but not the payload of the traffic, so this is a new and exciting capability for both forensic researchers and offensive security analysts. 

Philip's Twitter: @oilheap
Philip's GitHub: https://github.com/Phaeilo

4th: Ying Li: Python Strings and SSH Keys 


This submission includes a collection of plugins for analyzing memory samples acquired from 64-bit Linux systems. The plugins were initially presented at PyCon 2015. The first plugin in the submission, linux_python_strings, extracts memory resident Python strings from the heap of Python processes. This is accomplished by scanning the heap for Python string objects. The advantage of this approach is that we are able to leverage context that the interpreter has about the particular string and how it is being used. The author also includes a plugin for extracting strings stored in Python dictionaries. The context provided by this plugin could allow an analyst to determine which strings were associated as key-value pairs. The final plugin, linux_ssh_keys, will extract RSA keys from the heap of ssh-agent processes. These keys could be useful when investigating lateral movement or performing an investigation of a suspected person's machine.  

Ying's plugins continue the trend of moving further up the analysis stack into the application to extract more memory resident context. As for RSA keys, extracting crypto artifacts is always interesting. To date, we haven't seen anyone extract the RSA keys from ssh-agents with Volatility. Additionally, Ying created an automated test harness for generating memory samples, which we thought was particularly useful. 

Ying's Twitter: @cyli
Ying's GitHub: https://github.com/cyli
Ying's PyCon 2015 Talk: https://www.youtube.com/watch?v=tMKXcc2-xO8

5th: Adam Bridge: NDIS Packet Scan 


This submission carves packets and ethernet frames from NDIS shared memory sections (regions of RAM shared between the OS and DMA NIC). Although tools exist to scan an arbitrary binary file for packets, the extra context that Adam's plugin provides is extremely valuable. For example, the methodology is far less likely to produce false positives or report fake/decoy packets. In addition to outputting text and pcap formatted results, the plugin also decodes NetBIOS names found in DNS traffic and extracts slack space between packet payloads that other tools may miss. This research represents the beginning of a very exciting realm of future plugins that focus on NDIS private data structures including data waiting in sent/received buffers. 

Adam's Twitter: @bridgeythegeek
Adam's GitHub: https://github.com/bridgeythegeek

The following submissions appear in the order they were received. As previously mentioned, everyone succeeded in solving a specific problem that they (and undoubtedly others) faced. For this, they deserve huge props. We look forward to seeing future work by these authors!

Joe Greenwood: Hacking Team RCS Attribution  


This plugin searches a memory dump for evidence of the Hacking Team Galileo Remote Control System (RCS), and attempts to attribute the infection to particular Hacking Team client. One of our favorite aspects of this plugin is that it detects RCS based on its predictable named shared memory sections, a creative alternative to scanning for the typical byte signatures and mutexes. Its also a really nice touch to provide the extra attribution context via the watermark lookups. 

Alexander Tarasenko: Pykd/Windbg Address Space


This submission lets you integrate Volatility into Windbg through Pykd. You're then able to query for addresses of critical data structures, functions, and other variables using the debugging APIs. Additionally, you can connect Volatility to a running Windows system in debugging mode and run Volatility plugins against the live system, which particularly comes in handy for malware/rootkit analysis (not necessarily IR/forensics). 

The author of this address space also mentioned a new project named Karmadbg (see the link below) which is a GUI intended for development of new debugging and memory analysis scripts. Check it out! Unfortunately, documentation in English is not yet available. 

Pykd's Twitter: @pykd and @pykd_dev
Pykd's Website: https://pykd.codeplex.com
Karmadbg's Website: https://karmadbg.codeplex.com

Loïc Jaquemet: Haystack 


These plugins are an interface between the Volatility framework and the haystack framework. While Volatility establishes a forensic framework to analyse a system's RAM, the haystack framework is intended to analyse a process's RAM, allowing an analyst to search for defined structures in a process's memory. You can build structure definitions from known/public C header files or with Python's ctypes library (based on undocumented or reverse engineered data structures) and then plug them into this framework to scan across process heaps, all process memory allocations, etc. The author provided examples of constraints that can find openssl cipher contexts, session keys, and passphrases, but its surely not limited to those types of data. Give it a shot and see what you can find! 

Loïc's Twitter: @trolldbois
Loïc's GitHub: https://github.com/trolldbois
The python-haystack module: https://github.com/trolldbois/python-haystack

May Medhat (et. al.): GVol Tool


This tool by EG-CERT researchers May Medhat and Mohamad Shawkey provides a (thick) GUI front end for Volatility written in Java. It lets you run preconfigured batch scripts against a memory dump with just a couple mouse clicks or you can customize your own batch scripts. You can choose from categories such as Rootkits, Kernel Artifacts, Networking, etc. and they link up with Volatility plugins in the backend. We imagine this tool will be very handy for analysts who are not comfortable on the command line or with Volatility usage in general. Additionally, for those who are seasoned Volatility users, this tool can reduce the amount of time you spend typing commands before you actually dive into the details of your case. 

Monnappa Ka: Linux Memory Diff 


This plugin uses the Volatility advanced memory forensics framework to run various plugins against a clean and infected Linux memory image and reports the changes. Many times while doing memory analysis (or malware analysis) an analyst is presented with an abundance of data and the analyst has to manually find the malicious artifacts from that data which takes time and effort. This tool helps in solving that problem by comparing the results between the clean and infected memory images. This tool helps speed up analysis, reduce manual effort and allows you to focus on the relevant data.

Monnappa's Twitter: @monnappa22
Monnappa's GitHub: https://github.com/monnappa22
Monnappa's Blog: http://malware-unplugged.blogspot.com

Bart Inglot: Scheduled Task and Job Scanners 


This plugin scans for job files and prints out their information. It's useful from a DFIR perspective, since job files are often used by attackers in order to run programs with SYSTEM privileges, run at scheduled moments, or to move laterally within a network. Aside from the job plugin for Volatility, adapted from Jamie Levy's jobparser.py (http://gleeda.blogspot.com/2012/09/job-file-parser.html) Bart also has written standalone carvers for jobs and scheduled tasks that work against memory samples, disk images, and other binary files. 

Tuesday, August 25, 2015

Volatility Updates Summer 2015

Summer 2015 has been quite a busy time for the memory forensics community. We wanted to write a quick update to talk about some recent events and research as well as upcoming news.

Conferences

Black Hat Vegas 2015

We wanted to again thank everyone who came out and supported us during Black Hat. Between our Arsenal demo, book signing, and party, we met hundreds of Volatility users and fans. Your support and enthusiasm is greatly appreciated. Come back next year for twice the champagne, twice the suite size, and twice the fun!

HTCIA International Conference (Orlando) 

We're putting on a lab session at HTCIA's International Conference in Orlando next week. You can also stop by the Volexity booth for a chance to win a free seat at any upcoming Windows Malware and Memory Forensics Training course.

Open Source Digital Forensics Conference (OSDFC) 2015

The Volatility team will once again be presenting the latest in memory forensic research at OSDFC 2015. This year we will be focusing on the anti-forensic capabilities of PlugX as well as new Volatility capabilities that can auto-detect them. We will also be discussing the results of the 2015 Volatility Plugin contest. 

OSDFC has a great lineup this year so you should try and attend. We hope to see many Volatility users while we are out there.

Research

A New Paper on OS X Memory Forensics

At DFRWS 2015, Dr. Golden Richard and I published a paper on OS X memory forensics entitled: Advancing OS X Rootkit Detection. The purpose of this paper was to document gaps in existing OS X rootkit detection techniques and then develop new methods, in the form of Volatility plugins, to close these gaps. The plugins introduced in the paper will be committed to GitHub in the coming weeks.

DFRWS had a number of good submissions this year, and we recommend browsing the program for other interesting forensics research.

Volatility vs Hacking Team

The malware used by Hacking Team to control victim's computers is known as Galileo RCS, and in a well done blog post, Joe Greenwood shows how to use Volatility to detect RCS in a number of ways.

RCS now joins a long list (Stuxnet, Careto, Flame, and more) of 'advanced', 'stealthy' malware that immediately falls to inspection by memory forensics.

Volatility at PyCon

At PyCon 2015, Ying Li showed how to use newly developed Volatility capabilities in order to find artifacts of Python scripts that were executing on the system. This was very interesting work, and we suggest watching the video of her talk.

Projects Building on Volatility

VolDiff

VolDiff is a project that compares the results of a number of Volatility plugins against two memory samples and automatically the reports the differences. Compared to manually running and comparing the plugins, this can save a substantial amount of time.

The purpose of VolDiff is to compare in-memory artifacts both before (clean state) and after (post-install state) an application, such as a malware sample, has executed. The new artifacts that appear post-installation can be immediately isolated for further analysis and/or for the creation of highly-effective IOCs.

If you wanted to get started with VolDiff then you should read the author's post on analyzing DarkComet with VolDiff.


Evolve

Evolve is an open source web interface to Volatility. It is under very active development and is constantly having new features added. Consult the README on GitHub for the latest features, and be sure to follow the tool's author on Twitter.

To see Evolve in action, check out a video showing the basic features here and advanced features here.

As much as we love the command line, it is sometimes nice to have a GUI visualize and shuffle data for you!


Plugin Contest

The 2015 Volatility Plugin Contest is underway and accepting submissions until October. The contest is a great way to win cash and other prizes, gain recognition in the community and become more familiar with Volatility and Python. We feature the research submitted to the contest on this blog, during conferences, presentations, and all throughout social media and our mailing lists.

New Volatility Capabilities


Windows 10 Support

While not merged in the official Volatility branch (yet), the Windows 10 branch is currently under active development. Nearly all of the plugins are working at the current time, and we would be very appreciative of any bug reports you may have when testing the branch. Please post any bugs to the GitHub issue tracker. Currently supported functionality includes process and kernel module listing, all pool scanning plugins (files, mutexes, processes, drivers, etc.), handles, DLLs, PE file extraction, process memory (VAD) parsing, service enumeration, cached file extraction, and memory signature scanning with Yara.


OS X 10.10.x Support and New Plugins

Volatility now has official support for OS X 10.10.4 and 10.10.5, which are the latest two versions. We have also tested Volatility on a preview release of 10.11, and it appears that all of the plugins work as expected. We will release an official profile for 10.11 once Apple releases debug kits (we are currently using special custom built profiles).

Volatility also has a new plugin named mac_get_profile. This plugin allows Volatility to auto-detect which profile (OS version) matches the given memory sample. To use this plugin you do NOT need any OS X profiles installed. Instead, you can run a fresh checkout of Volatility, determine the profile by using mac_get_profile, and then download the correct profile from our profiles repository.

To use mac_get_profile, simply pass the path to your memory sample as the -f option and then put mac_get_profile after it:
$ python vol.py -f <path to memory sample> mac_get_profile


Linux

Volatility Linux support has now been tested through kernel version 3.19. As many of the data structures do not change between versions, we expect that most or all of the plugins will work with bleeding edge developments kernels as well. Please file an issue if you encounter any bugs.


Memory Forensic Trainings

Our memory forensics training class in Amsterdam is now SOLD OUT.  We appreciate the support and word of mouth praise from past attendees as well as fans of the project.

Our current course is 5 days of memory forensics and malware analysis training against Windows systems. Full information on the course, as well as upcoming dates and locations, can be found here. If our current set of public offerings does not work for your company then please contact us about conducting a private training at one of your facilities.