Advice from Det. Michael Chaves on Memory Forensics, KnTDD, and POS Malware

The following story was shared by Detective Michael Chaves. It describes how he's used Volatility, KnTDD, and memory forensics over the past year to investigate POS breaches at local businesses. Kudos to Michael for applying his skills in an effective and meaningful way, then taking the time to share experiences with others. Without a doubt, detectives in every police department have or will encounter situations like Michael describes.

It's been about year since I've taken the Volatility Windows Malware and Memory Forensics Training in NYC.  I wanted to take this time to share some of my experiences to hopefully help examiners/investigators early on in their exposure to Volatility and to help identify unknown malware.  Over the past 10 months I have responded to about a dozen POS breaches at local businesses; mainly liquor stores and restaurants. These breaches are identified rather quickly from local banks that call me with the details and I usually respond to a location within 2 days. It should come as no surprise that I have yet to respond to a location that was anywhere near being PCI compliant.

The large majority of POS terminals were running Windows XP some with SP2, most with SP3.  Two machines were even running Windows 2K and all had direct connections to the Internet.  Antivirus IF present was either out of date or turned off.  I still have yet to see a firewall present or any security policy in place.  My RAM capture tool of choice is Kntdd and I’ll use FTK Imager Lite to obtain all registry files, App Data directory, $log, $MFT and prefetch directory.  I carry with me several portable drives to make the acquisition from each POS location in the shortest amount of time possible as the store still needs to process customer purchases. 

For most of these breaches I have been able to identify the malware pretty easily.  I usually begin by running, pslist, psscan, psxview and connections (if supported).  In the majority of the breaches, the processes were not hidden and had an active process listed, usually called by ‘explorer’.  If I was not able to easily identify the malware process, I’d run dlllist to locate any programs running from odd locations followed by malfind  and yarascan.  Once I have identified a suspect process, I’d dump that process usually by procdump or dlldump.

During the early part of the investigation, I am not too concerned how the malware works or what the Initial Infection Vector was.   I want to know where the credit cards are going and how are they getting out.  I’d run strings on the exported out suspected malware file and I would generally find, the URL used to send out the cards via POST, an e-mail address associated with the malware and/or IP addresses.  The majority of my cases the POST command was used to send out the cards, in others it was via SMTP.  In 5 of my 12 breaches, the malware family was JACKPOS or Alina variants.  I will search the Internet on file names, URL’s and artifacts that usually result with great write ups that show what I may be investigating, as well researching with Virustotal.   It should also be noted that there have been a few times that I sought assistance from the Volatility community and other students from the NYC class.  They have been extremely receptive to my questions and information provided to was invaluable!

I realize there are many readers out there that have a far greater understanding of malware and memory forensics.  I’m slowly getting there, but I hope this helps out the people just beginning, like I am/was by describing my workflow and perhaps give confidence to some that may otherwise doubt their ability.

Detective Michael Chaves Shares A Memory Forensics Success Story

Detective Michael Chaves from the Monroe CT Police Department shares the following story regarding his experiences with Memory Forensics, Volatility Training, KnTTools, and POS breaches. Michael was also recently quoted in Brian Krebs' article Card Wash: Card Breaches at Car Washes for the key role that he played in that investigation.

Shouts to Michael - keep up the great work!

Before attending this class I had a strong digital forensic background, but lacked an understanding of the "under the hood" workings of RAM and the stuff running in RAM.  I knew I was in for a challenge and boy did I get one.  I always had a desire to learn about memory analysis and I had some knowledge of what it contained, but it was the significant increase in POS breaches that I was investigating that I realized I needed this class sooner than later.

After taking this class in May of 2014 I began investigating a POS breach involving a local business chain.  The chain was a Common Point of Purchase for thousands of credit/debit that were compromised that lead to more than $100,000 in losses from fraudulent use. From the class, I learned of a new memory acquisition tool from GMG Systems, Inc. called " KnTTools".  I tested it out and found it to be an extremely reliable, fast and efficient program that has a very small footprint.  I used KnTTools to acquire numerous RAM dumps from several locations.

Now came the time to figure out what I had.  Going in blind, not knowing where to look or even what I was looking for was a daunting task.  Referring to my notes and the student handbook, I began to use Volatility to try and understand what I had.  I began to use plugins such as pslist, psxview, malfind, apihooks and connections and I started to get some information to look more into.  Understanding the PID/PPID relationship and what process should call another was very helpful.  Working with the business I learned what programs were legit and white listed them.  This was important to me since I did not know what programs and applications were supposed to be running on those computers.  I located three running processes that turned out to be malware.  I used dlllist, dlldump, procdump and dumpfiles to extract out the processes, files and dll's and ran strings on them.  From there I located great information including the POST/GET commands that show where the cards were going to... BINGO!

I located the same malware on ALL other RAM acquisitions.  Although I do not know exactly how the malware got onto the system or fully how it works, I located the necessary information I needed to proceed with my investigation. Without this class or the Volatility tool, I would never have been able to further my investigation.  Volatility is a game changer in memory forensics.  With more and more POS breaches being reported every day both on a local and national scale, responders need the ability to efficiently and effectively analyze the RAM where the malware attempts to run.... and hide.  But you can't hide from Volatility!

Comparing the Dexter and BlackPOS (Target) RAM Scraping Techniques

Up until yesterday when Brian Krebs wrote A First Look at the Target Intrusion, Malware, there weren't many details about the involved code. Now that its out there, I thought it might be interesting to see how the "RAM scraping" feature worked in comparison to the Dexter malware. As it turns out, the two are quite similar, and neither are really exciting. This just goes to show that you don't need advanced (fine tuned, maybe) tools to be successful at cyber crime.

The BlackPOS malware (see Krebs' article) A.K.A Trojan.POSRAM uses EnumProcesses to get the list of active PIDs on the system. It then cycles through the list, skipping its own PID (Dexter also skipped its parent PID). For all other processes, it calls GetModuleFileNameEx to get the full path to the executable, then strips off the file name (i.e. explorer.exe) portion, converts it to lowercase, and compares it against "pos.exe" with strcmp. Had Target known that this specific sample only looks in the memory of processes named pos.exe, it could have renamed its Point-of-Sale application and avoided the news (for a few minutes anyway... other samples are known to exist that looked for other process names).

The "pos.exe" string isn't exactly in plain text, but the malware isn't packed either. It uses a very simple obfuscation technique where the string is split and the characters are shifted around a bit. You can see it in the read-only section of the PE file:

Once it finds a pos.exe process, it opens a handle with OpenProcess and uses VirtualQueryEx to begin iterating through the process's available memory blocks. This is exactly what Dexter did -- the only difference being that Dexter also checked memory protection constants and skipped ranges that were marked PAGE_NOACCESS or PAGE_GUARD. As a result, the BlackPOS malware could easily lead to an access violation (e.g. STATUS_PAGE_GUARD_VIOLATION) capable of crashing the malware. I can only imagine the look on the attackers' faces once they realized that they came up with 0 credit cards because they didn't check page permissions before reading a memory address. Unfortunately for Target, the pos.exe processes must not have had any no-access or guard pages set.

Here's a snippet of code reverse engineered from the malware that shows how it determines which ranges to scan:

VOID ScanMemory(HANDLE hProcess)
{
    int lpAddress = 0;
    int lpMaxAddress = 0x6FFFFFFF;
    int endAddress = 0;
    MEMORY_BASIC_INFORMATION anMBI;
    SIZE_T cbReturned;

    do { 
        cbReturned = VirtualQueryEx(hProcess, 
                        (LPCVOID)lpAddress, 
                        &anMBI, 
                        sizeof(anMBI));

        if (cbReturned && anMBI.RegionSize)
        {
            endAddress = (int)((char *)anMBI.BaseAddress + anMBI.RegionSize);
            ScanRange(hProcess, 
                      anMBI.BaseAddress,
                      endAddress);
        }
        lpAddress = (int)(char *)lpAddress + anMBI.RegionSize;
    } while (lpAddress < lpMaxAddress);
}

Notice the malware stops at 0x6FFFFFFF, to avoid scanning inside system DLLs which normally exist in high regions of process memory space. The ScanRange function (not shown) breaks the ranges identified by VirtualQueryEx into (roughly) 10 MB chunks and uses ReadProcessMemory to read the data into a buffer, which it then checks for patterns and substrings related to the credit card information that it wants to steal. Dexter did something similar, but read data in roughly 400 KB chunks instead.

In conclusion, Dexter and Trojan.POSRAM are really quite similar in terms of how they scan memory for the sensitive data.

-Michael Ligh (@iMHLv2)