Advanced “USN Journal” Forensics
NTFS is the default journaling file system for windows operating systems. Understanding NTFS features and how it works helps Digital Forensics investigators navigate and conduct their analysis for various objectives. The NTFS file system contains several files (called metafiles) to organize and structure the file system, one of those metafiles is the master file table ($MFT) which is used by forensics practitioners to gain insight into all files within an NTFS structured volume. Later, Microsoft added the USN Journal (Update Sequence Number) “$UsnJrnl” metafile which is also called the change journal, to maintain information of all changes occurred to files and folders on an NTFS volume, providing records for what and when changes are made and to which objects.
One USN Journal is stored within each NTFS volume and is stored in the NTFS metafile named “$Extend\$UsnJrnl”. The journal begins with an empty file, and whenever a change is made to the volume, a record is added to the file. Each record will contain a 64-bit Update Sequence Number (USN), the name of the file and a bit flag (e.g. USN_REASON_DATA_OVERWRITE) representing the change that was made.
“$UsnJrnl” has two main data streams which are, “$J”which records file and folder changes that occurred on the volume, and “$MAX”which is a small file that stores metadata about “$UsnJrnl”.
Forensics Value
Since “$J” has records of all changes on files and folders in a volume including deleted files, this opens the doors for digital forensics investigators and threat hunters to empower their analysis with information such as the following:
Detection of malicious tools and bad files that were present at some point in time within the file system, which provides insight into suspicious/malicious user activity.
Detection of “Timestomping” activity, a technique used by attackers which is the alteration of timestamps of files to confuse investigators during their analysis.
Extending “Prefetch” artifacts value, which each contain the dates for the last 8 times an executable was run, which is a limitation of the artifact. This limitation can be overcome to gain the dates for more executions of an executable (Subject to the limitations of USN Journal below).
Limitations
The records indexed in the “$J” data stream has a maximum size and can be checked using the command “fsutil usn queryjournal C:” (C in this example is the target volume) and in busy volumes it can store approximately 20 days of changes on all files and folders.
The “$UsnJrnl” is a metafile, which makes its acquisition a little bit more complex than doing copy and paste.
Acquisition and Parsing of “$J” Data Stream
The “$J” USN Journal data stream is located in “VOLUME:$EXTEND/$UsnJrnl/$J” (note that each volume has its own journal). The acquisition can be conducted by any disk forensics or data preview and imaging software such as Encase, and FTK Imager.
Figure 1. Acquiring the “$J” data stream
The parsing of the “$J” data stream can be conducted using tools such as UsnJrnl2Csv64.exe, or MFTECmd.exe which will be used in this post.
Figure 2. Using “MFTECmd” to parse the “$J” data stream
We used the “--csv” switch to get the parsing results in CSV format so it can be further inspected and analyzed with software made for that purpose. Timeline Explorer.exe is a good choice for our purposes and is highly recommended for forensic investigators.
Figure 3. The parsed “$J” data stream viewed in “Timeline Explorer”
Note that Parent Path has no values, because the “$J” records don’t store such information, such information can be either correlated manually by going through the Master File Table ($MFT) and matching the entry numbers in both records, or “$MFT” can be passed to “MFTECmd” tool as an argument and automatic correlation will be conducted by the tool itself.
Figure 4. Parsing the “$J” data stream and enriching it with parsed “$MFT” information to show the full path
Figure 5. The parsed and merger of “USN Journal” and “Master File Table” records viewed in “Timeline Explorer”
Use Cases for The Utilization of USN Journal for Forensic Analysis
Detection of Deleted Files:
To demonstrate this use-case, we will create a malicious file, use the file, and then remove it permanently from the system, and try to detect its past presence using USN Journal, so we will download psexec.exe (Our model for the malicious tool) on the test machine and then delete it.
Figure 6. “PsExec.exe” is dropped in “C:/Users/User/Desktop/article” folder
Figure 7. Deleting “PsExec.exe” permanently from the machine
Now to hunt for this tool, we will acquire “$UsnJrnl:$J” and “$MFT” parse them and merge their results to enrich the output with the parent path for each record using “MFTECmd” as explained earlier.
Figure 8. Merged $UsnJrnl and $MFT parsed records show the full life-cycle of the file on the volume
As shown in the figure above, using information from the “Update Reasons” and “Update Timestamp” fields, we can draw a timeline of the file activity on the volume from its creation on “2022-10-24 16:12:04” and finally after the attacker used it, its deletion on “2022-10-24 16:15:16”.
2. Detection of “TimeStomping” Activity:
Timestomping is a technique used by attackers which is the change of file attributes that contain dates (MACB) such as the file creation and modification dates, to confuse investigators by diverging certain files from the timeline analysis of a certain incident or activity. Attackers mostly use this technique when planting a persistent malicious backdoor, so incident responders cannot detect it when looking at files planted by the attacker when searching within the incident time range.
We will do a small experiment to demonstrate how this can be done, and how USN Journal analysis can help uncover such activities.
Figure 9. Shows the current modification date for “calcx.exe” as it appears in the system
“nTimeStomp.exe“ is a tool that allows changing timestamps of a file, and in this experiment we used it to alter the date and time information back to “1996-01-07 12:34:56.7890123” for all MACB attributes.
Figure 10. Using “nTimestomp.exe” to alter date and time attributes for “calcx.exe”
Now if we check the file metadata we can see the change reflected on the target file and a new fake date/time appears on its MACB attributes.
Figure 11. Timestomped “calcx.exe”
Now we will acquire the actual MACB dates for the file utilizing the Change Journal (USN Journal) and enriching its output by parsing and mergin the “$J” and “$MFT” metafiles. Looking for entries related to “calcx.exe” we can see the actual dates for the file activities on the volume.
Figure 12. Parsed “$J” output showing the real time and date of the file
As shown the file creation date is a fresh date and not actually back in “1996”, and in “Update Reasons” show “BasicInfoChange” which indicate a metadata change was occurred on the file, hence the the identification of the timestomping technique. This can be further enhanced to hunt for files that are timestomped without having a specific file in question, by collecting MACB attributes from “$MFT” or directly from recursively going over the volume, than joining both results by “File Name” and running an equation to calculate the difference in times for both entries, if they do not match, alert on that for “Timestomping Activity Detected”.
3. Prefetch Output Enrichment:
Any executable that is run on a Windows System, uses a set of imported functions from a set of .dll’s (Dynamic-Link Library). Have you noticed that running an application for the first time takes more time than running it afterwards? That’s what Prefetch is used for. It monitors application execution usage pattern and caching the dlls and other data files and makes it available in memory in advance so they can be accessed quickly when needed hence speeding up application execution. Prefetch serves an additional value, for Digital Forensics investigators which it can be used to identify which applications were running on a Windows system (evidence of execution). It also includes other information such as the last 8 times an executable was run. This advantage has also an obvious limitation that is it gives only the last 8 times an application was run. However, utilizing the “USN Journal”, this limit can be overcome as the “$J” data stream stores changes for any file (including updates on “.pf” files) so it will store all changes occurred on prefetch records (.pf). By looking for “DataExtend|DataTruncation|Close” flags which are assigned in USN records for each time the prefetch file is updated which happens at each application execution, we can identify the dates/times for more number of executions for an application, ergo exceeding the 8 times limit imposed on the Prefetch.
To demonstrate this, we will try to parse the prefetch record for “conhost.exe” in a test machine using “PECmd.exe“ then parse the USN Journal and compare outputs.
Figure 13. Parsing the Windows prefetch records with “PECmd”
Now if we filter the results for “conhost.exe”, as show in below figure, we will get the last 8 times the application was executed including the last time it was run which was at “2022-10-25 10:37:03”.
Figure 14. The last 8 times “conhost.exe” was executed, extracted from its prefetch record
Now we will acquire the USN Journal, and enrich the output with The Master File Table and see how this can help us expand our knowledge of application execution using the prefetch for more than the last 8 times. We begin by filtering the results for the “.pf” file extension , “conhost.exe” file name, and for the “DataExtend|DataTruncation|Close” update reasons.
Figure 15. Output of the enriched USN Journal
As shown in the above figure we can see changes that occurred on the “.pf” file of the executable in question, with dates and times beyond the 8 records available within the “.pf” file itself. Here we see more than 54 records for the last 54 times the application was run, and reach the conclusion that it was lunched a month before the last execution date that we got from the prefetch record itself, which is at “2022-09-26 08:22:35”.
Conclusion
“$UsnJrnl” contains the change records of all files and folders in a volume. It has two main data steams which are “$J”, and “$MAX”. The “$J” data stream has a forensics value that help investigators gain more information about data within the file system and leverage that for advanced use-cases such as detecting deleted files, TimeStomping and extending the value of Prefetch in windows workstations. The value of the “$UsnJrnl” shrinks when conducting a big scale threat hunting in a big environment and extends when conducting an incident response on a contained set of windows machines.