Malware that uses PowerShell, the most prevalent use is the garden-variety stager: an executable or document macro that launches PowerShell to download another executable and run it.
Before You Begin
Set up environment to collect and process PowerShell logs:
1) Process Auditing and Command Line Logging. – Event ID 4688 on Windows gives you access to the command-line arguments used when processes are launched. – Another useful source of this data is Sysmon. 2) PowerShell Module, Script Block, and Transcription Logging. – Configure PowerShell to log all commands that it invokes, as well all script blocks that it invokes. – As with the content stream exposed to the Antimalware Scan Interface, this logging also includes code generated or transformed at runtime. 3) Centralized Event Log Collection / Forwarding. There are many techniques to collect and forward event logs across an organization. – Windows Event Forwarding: Monitoring what matters – Windows Event Forwarding for everyone (even if you already have a SIEM).
Most detection by taking these two major event sources and applying static signatures to them. This is usually simple string matches and regular expressions.
Obfuscation methods
Launch Techniques
1) Rely entirely on command-line logging (4688) event rather than PowerShell script block logs (4104) to detect malicious PowerShell.
* To break 4688-based logging
a. To obscure the name of the PowerShell executable, some attackers will create (or include) a copy of PowerShell.exe, but rename it to something less suspicious.
(Like plastic surgery, haha)
The 4688 command line logs, then, would show something similar to:
Unlink the command-line arguments from the code they deliver
b. An example of this comes from PowerShell’s (like most management / automation programs) ability to consume commands from the Standard Input stream.
Two examples of this are:
When viewed in the event log, the arguments to PowerShell.exe are no longer directly visible:
When this technique is chained several times, must correlate several layers of process command lines to understand the code that was invoked.
(Similar to Matryoshka Doll or makeup?)
Another technique, employed by the Kovter family of malware and others, is to store the malicious PowerShell commands in an environment variable and have PowerShell execute the content of that variable.
* https://blog.malwarebytes.com/threat-analysis/2016/07/untangling-kovter/
To storing content in environment variables, it is also possible to deliver content so that reassembling the command lines from the chain of parent processes offers little to no insight.
Obfuscating the Cradle
Obfuscation of the PowerShell script text itself can prove very effective against static signature-based detections.
This is a common battle ground between malware authors and antivirus vendors for all scripting languages, and this battle continues when applied to PowerShell scripts.
For the purposes of discussion, we will focus on an example download cradle as well as the static signatures that might be used to detect it.
An initial detection approach might attempt to match all of the following terms:
- Invoke-Expression
- New-Object
- System.Net.WebClient
- DownloadString(‘http
Obfuscation techniques:
- The URL is just a string, so can be concatenated and written in other ways such as “h” +“ttp”.
- System is optional in PowerShell type names, then System.Net.WebClient = Net.WebClient
- PowerShell can use either single or double quotes in strings, whitespace can be added almost anywhere. Then, DownloadString(‘ = DownloadString( “
- WebClient class offers many methods to download content, DownloadString = DownloadFile = DownloadData = OpenReadAsync
- Method names such as DownloadString can be included in quotes and have escape characters,
- Net.WebClient argument to New-Object can be obfuscated with escape characters, string-based obfuscation techniques, and concatenation across multiple variables
- Command names often have aliases, Invoke-Expression = iex
- Even when commands do not have aliases, the Get-Command command lets a script author query the PowerShell command list and invoke the result. This query can include wildcards, so invoking New-Object can look like this: & (Get-Command w-O). The invocation (&) operator in this example has an alternative, which is the dot (.) operator. The Get-Command cmdlet has an alias and can be dynamically invoked similarly, so is not safe to key on.
- Get-Command as a mechanism to query command names, PowerShell offers several API-style methods to query command names – such as $executionContext.InvokeCommand.GetCommand()
- & and . operators support string arguments. These can be used for string concatenation and string reordering
- Detection of Invoke-Expression suffers from the same challenges of command obfuscation that New-Object and Get-Command making false positives based on this indicator a significant challenge.
- Invoke-Expression can be Invoke-Command, Script Block invocation (such as & [Scriptblock]::Create(“Write-Host Script Block Conversion”) ), and dynamic script invocation APIs such as $ExecutionContext.InvokeCommand.InvokeScript(“Write-Host EXPRESSION”).
String Obfuscation
- String concatenation: “http” = “h” + “ttp”, PowerShell’s –join operator, String.Join() and String.Concat() methods from .NET.
- PowerShell’s –f string formatting operator, based on the C# String.Format method uses format tokens like {0} and {1} to identify the order of replacement strings: & (“{1}{0}{2}” -f ‘wOb’,’Ne’,’ject’).
- Strings can be reversed PowerShell’s array slicing operator (-join “detacsufbO”[9..0]), Array.Reverse ($a = [char[]]”detacsufbO”; [Array]::Reverse($a); -join $a), reverse regular expression matching (-join [RegEx]::Matches(“detacsufbO”,’.’,’RightToLeft’)), and others.
- Strings can be split by an arbitrary delimiter, and then rejoined: -join (“Obfuscated” -split “~~”)
- Strings can be replaced either to remove delimiters, or change the meaning of a string by the –replace operator or the String.Replace() method: “System.SafeClass” -replace “Safe”,”Unsafe”
Detecting Obfuscated PowerShell
Character Frequency Analysis
Example: The canonical obfuscation built into the Metasploit Framework uses random characters for all variable and function names.
The MSF-based stager had the most randomness in its variable names, and only 24% of the script came from the top four letters. For the “normal” scripts in that small-scale experiment, the top four letters accounted for 35% or more of the script (noemal > 35%).
Cosine Similarity
Rather than analyze the frequency of the top four letters, also can analyze the frequency of each letter in a script.
One approach used frequently in the information retrieval community to rank and compare lists of numbers is called Cosine or Vector similarity.
The number of occurrences of certain words, paragraph lengths, number of internal links, number of external links and more.
Cosine similarity builds on the same math that we use to measure the angle between two lines in geometry.
Building the PowerShell Corpus
Two major areas of weakness with the initial Cosine Similarity investigation were:
Lack of variety. Scripts shared in PoshCode have some degree of variety, but don’t represent the full breadth of author experiences and scenarios in the PowerShell ecosystem. PoshCode actively encourages copying, modifying, and re-sharing existing scripts, so there were many examples that were redundant with respect to other scripts.
Lack of labeled data. Without an exhaustive labeling of which scripts were in fact obfuscated vs. which were not, the accuracy of Cosine Similarity cannot be measured. Despite having a reasonable number of false positives when all of the items below a similarity value of 0.8 were reviewed, it was not possible to measure how many obfuscated scripts were missed by having a similarity score greater than 0.8.
Cosine Similarity produced great precision (89% of the items it considered obfuscated were in fact obfuscated), but suffered from poor recall (at scale, it only detected 37% of what was obfuscated). The F1 score, which incorporates both of these metrics, demonstrates the relatively poor overall performance.
Leveraging the PowerShell Tokenizer and AST
PowerShell engine includes two extremely powerful features to give tool authors deeper insight into the structure of PowerShell scripts: the PowerShell Tokenizer, and the PowerShell Abstract Syntax Tree (AST). These features are commonly used to enable syntax highlighting support for PowerShell editors (such as the PowerShell ISE and Visual Studio Code), as well as advanced code analysis features such as the detection of unused variables. The System.Management.Automation.Language.Parser class provides access to both the tokenization of a PowerShell script and the tree-like representation of the script. Tokenization provides access to PowerShell’s initial basic extraction of comments, variable names, command names, operators, and more:
PowerShell’s parser additionally creates a tree-like representation of the script, called the Abstract Syntax Tree. This representation provides access to rich structural data about the script, such as the nesting of commands within script blocks, variables used in parameter arguments, and more:
Logistic Regression with Gradient Descent
With this advanced access to the structure of any given PowerShell script, we can begin to extract features much more descriptive of a script’s composition than its character frequency alone. As part of this investigation, we wrote feature extractors to calculate and summarize 4098 unique script characteristics, including:
- Distribution of AST types
- Distribution of language operators
- Assignment, binary, invocation, …
- Array size ranges
- Statistics within each AST type
- Character frequency, entropy, length (max, min, median, mean, mode, range), whitespace density, character casing, …
- Statistics of command names, .NET methods, variables…
This classification approach directly identifies the likelihood that a script is obfuscated, rather than use another metric (like similarity being greater than a certain number) to determine that fact.
A common approach to classification of feature vectors is to apply a linear regression.
Most statistical and mathematical packages offer built-in functionality to create a linear regression.
Excel is one popular choice. A linear regression is based on the simple concept that you take each feature, multiply it by a weight, and then add all of those results together. If the result is over a certain amount, then the sample is considered part of the target classification (i.e.: “obfuscated”), while otherwise it is not.
In pseudo-code, this looks similar to:
To keep the result within a reasonable range of values and also to enable some slightly non-linear distributions, it is common to apply the Logit function to this calculation. Together, this forms the basis of a Logistic Regression.
The final step in creating the PowerShell Obfuscation classifier is to decide on the appropriate weighting for each of the 4098 unique script features.
One approach is to do it manually, but that’s error-prone and time consuming. Another approach is to employ a Gradient Descent algorithm, which is the approach we took.
* The fundamental concept behind the Gradient Descent algorithm is similar to the back-propagation approach used in neural networks.
Apply this error correction approach over many thousands of iterations over the labeled training data set, the Gradient Descent algorithm will generate a set of weights for the feature vector that minimizes error as much as possible.
When doing this training, it is critical to separate the set of data used for training from the set of data used to calculate the actual results. Without this partitioning, a given weight vector can become over-trained to the point that it is extremely accurate on the training data, but loses a great deal of fidelity on unseen data.
