What is a threat and which are the threats you should hunt for? All threats are bad but threat hunting is specific to certain types of threats. The following are some examples of threats a security operation center (SOC) can hunt for.
- Malware on endpoint
- Insider threat
- Compromised IoT device
- Unauthorized tunnel to the outside network
Threats that involves threat hunting are essentially bad stuff within the network. When a threat is found, an incident is created. Organizations need a way to respond to an incident, which is typically part of a SOC’s incident response program. To be clear, a security incident is an event that leads to a violation of an organization’s security policy and puts sensitive data at risk of exposure. A breach, however, is a type of security incident. Every security incident isn’t a breach. For example, if a server goes down, that would be a security incident, but not a breach. Hunting for threats can occur as part of responding to an incident that involves a breach. If a threat exists outside of the network, it is not a breach and the SOC doesn’t hunt for it.
Lots of threat hunting also occurs proactively meaning the SOC doesn’t know what to look for or if they are under attack. The SOC just knows where to look and will be able to identify something bad as a search is conducted. The characteristics of what is searched for can be referenced within the MITRE ATT&CK framework found at https://attack.mitre.org/.
Detecting Threats
When a breach occurs, the SOC needs a way to detect the associated threat or an incident will never be created hence the breach will go unnoticed. Threat detection can be proactive meaning the SOC develops a hypothesis and searches for it within the network or reactive, which means a security tool or user detects a threat and signals for the SOC to respond to it meaning the hunting starts based on a response to a security incident. Security detection occurs using one or more of three foundation security capabilities found withing security tools. Those capabilities are either
Signature: Matches a known pattern such as a hash file
Behavior: Matching known behavior such as port scanning
Anomaly: Identified as unusual behavior compared against a baseline of normal behavior.
The following figure represents this concept. Many security tools use a combination of these capabilities.
Breach Defense
Reactive threat hunting means having one or more breach defense tool(s) provide an indication that a breach as occurred. The most common methods to detect a breach are the following:
- IDS: Connecting an internal intrusion detection system (IDS) to a SPAN/port mirror to monitor internal traffic
- Network Analytics: Monitoring data within the network devices such as NetFlow
- Honeypot: Setting up a honeypot to lure threats to attack the honeypot rather than real targets
- Outbound Communication: Using tools that monitor outbound traffic looking for communication to high-risk resources such as the darknet, Tor, or known malicious resources
- Packet Capture: Capturing and analyzing packets from within the network
- Hosts Alarms: Your employees and their systems alerting you of a possible event
Malware developers know security tools and SOCs exist within organizations. Teams I have worked with and friends in the security industry have taken down perpetrators of attack campaigns and discovered that the attackers had research labs containing all of the security tools used in the industry. Malware developers typically have popular vendor firewalls models, IPS tools, various flavors of sandboxes, most modern antivirus offerings, and anything else organizations use to defend against cyber threats. This allows advanced threats to develop anti detection capabilities including encrypting files (packing), using stealth when communicating and defensive triggers, which will launch if the threat detects it is being analyzed by a security tool.
Threat Hunting
Proactive threat hunting means you are searching for something specific based on a hypothesis. Maybe you heard about a threat, been provided its indicators of compromise and looking to see if those indicators exist within your network. Maybe you know of specific malware files based on their hash and want to see if they exist within the network. Maybe you have had user’s complain and want to setup tests to see if the complaints are related to a breach. The following are common indicators to look for when threat hunting
Phone Home: Most modern malware communicates outside of your network when it successfully breaches a system. Ransomware using asymmetric encryption needs to communicate with the attacker to perform the public private key handshake. Remote Access Tools (RATs) need to inform the attacker that a port is open and ready for communication. Exporters will send data off the network.
Phone home traffic can occur over commonly used ports and can also be encrypted. The key to identifying phone home communication is using a blend of (1) “baselining” capabilities, which flags new and usual communication, (2) application visibility and controls to monitor for what types of communication is occurring and (3) segmentation to limit outbound communication from parts of your network containing sensitive systems.
Port Scanning: Most modern malware needs to spread. Insider threats also want to explorer the compromised network. In order to spread or move within a network, one must research the environment. Research includes scanning for open ports and protocols. The SOC needs to monitor for unauthorized port scanning as one way to detect an insider threat looking to learn the environment it has compromised.
IoC and Hash Scanning: Most security resources that speak about a threat provide mechanisms to detect if you have been impacted by it. This is commonly referred to as indicators or compromise (IoCs). Sometimes a specific hash is provided representing a signature of the threat. Organizations can use IoCs and hashes as search criteria within their security tools to see if a match is made. See the threat hunting example later in this post that includes the Cisco Talos blog for more details on this concept.
Other resources such as threat intelligence can provide additional data points to hunt for when threat hunting. MITRE ATT&CK provides different indicators of compromise that can lead to discovering a threat. Know that discovering a unsuccessful attack campaign or just the act of threat hunting can also improve the organization’s security based on the data that is collected.
Proactive hunting cannot be fully automated. A human must verify and adjust the hypothesis just like any science experiment. What typically occurs is you investigate a hypothesis, come up with either a ton of results or nothing and adjust the hypothesis accordingly. This allows for accommodating new data leading to identifying threats you didn’t know how to identify. You also may identify part of an attack, which by addressing the associated vulnerability, will prevent future exploitation from other attacks. This is why proactive threat hunting is essential to improving the security posture of a organization. It essentially allows for accommodating threats you don’t know how to identify, which tend to be the ones that have been on the network for a very long time.
The following represents different maturity states of a threat hunting service. More mature threat hunting teams will leverage large data sources, which require automation to evaluate. For example, a file can be compared not only against data within the organization, but also see how the same file has been used by other organizations around the work to help determine if it is a risk. This process, however requires a mature big data infrastructure to accommodate the large data set associated with a big data threat intelligence resource.
Containment, Eradication and Recover
When a threat is identified, there are three steps to properly respond. NIST SP800-61 calls these steps Containment, Eradication and Recover. You must first identify the scope of the breach. You can do this by developing a hypothesis of what to look for and evaluate all systems and networks against the hypothesis. As systems are identified as impacted, you must implement controls to limit the spread of the threat. The containment step is repeated until the entire scope of the threat is identified and contained.
When threat hunting during the containment step, you will aim to answer the following questions
- How is the attacker or malware moving around your network?
- How many systems have been impacted?
- What vulnerabilities need to be addressed to avoid future events?
- How is the attacker utilizing his or her tools?
- How is the attacker determining where he or she is going?
- Can the malware survive a reboot or simple remediation steps?
- Has the attacker stolen any data and, if so, what data was impacted?
Then remediation can occur following containment until the threat is eradicated. Eradication may include patching vulnerable systems, reinstalling operating systems on impacted endpoints, installing security tools designed to remove malicious files and resetting passwords. Containment measures must remain in place until the eradication step is complete.
Once the threat is considered eradicated, containment measures can be removed as part of the recovery step. Recovery means to return the organization back to normal post incident.
Threat Hunting Example
Cisco SecureX has a really cool threat hunting feature that targets the question “is this threat on my network?” hence proactively threat hunting for a published threat. Administrators can download an internet browser plugin that scrapes any IoC data from a website. The Firefox version of the plugin can be found at https://addons.mozilla.org/en-US/firefox/addon/cisco-securex-ribbon/ (Links to an external site.) and shown.
An example of a resource that provides IoCs is the Cisco Talos research blog found at https://blog.talosintelligence.com. (Links to an external site.) You can click any article about threats such as the weekly threat round up. Within such articles are various types of IoCs, IP addresses and hashes you could search within your network to see if you have been breached by the threats being covered in the article. The following is an example of file hashes posted in the weekly threat round up by Cisco Talos.
The Cisco SecureX plugin will scrap all of this and quickly run it against any security tool that is managed by SecureX. Maybe the malicious file came into your network through an email, which your email security tool would be able to flag based on matching IoCs from the Cisco Talos blog. Maybe the file was downloaded to a host, which your gateway security tool or host anti-malware would be able to flag. Maybe somebody plugged in a USB drive containing malware, which the malware will attempt to spread creating a footprint within the network as it attempts to pivot. Any security tool such as a NetFlow tool or IDS could be evaluated to see if it has seen any of the IoCs. The following show SecureX taking scrapped threat data and quickly searching the network for any matches.
Wrap Up
Threat hunting is a specific service used during a incident response program as well as a proactive service that evaluates a hypothesis. Every C level I meet with is concerned about threats living within their network. I always recommend to include both a proactive and reactive approach to threat hunting, which includes a combination of People, Process and Technology. Hopefully this post gives you a introduction to threat hunting concepts. You can learn more by visiting the Cisco Talos blog, registering for a Cisco Threat Hunting workshop and checking out the threat hunting exercises found within dcloud.cisco.com
