Click to learn more about author Thomas Hazel.

Security analysts have long been challenged to keep up with growing volumes of increasingly sophisticated cyberattacks, but their struggles have recently grown more acute.

Only 46% of security operations leaders are satisfied with their team’s ability to detect threats, and 82% of decision-makers report that their responses to threats are mostly or completely reactive – a shortcoming they’d like to overcome.

The reactive approach to threat detection that so many security programs continue to follow is nothing new. Throughout most of its evolution, the field of cybersecurity has adapted itself to each novel attack technique that cybercriminals have come up with. The first commercial antivirus programs operated by following in attackers’ footsteps, updating their databases of known malware signatures each time researchers discovered a new strain of malicious code. Enterprises followed suit, deploying additional perimeter-based technologies each time a new one came out – usually to counter the latest threat.

In today’s cloud-centric and data-intensive computing environments, however, reactive defense strategies simply don’t work. The legacy assumption that threats can be kept out of a trusted internal corporate network has been rendered obsolete by borderless network architectures and “anyone, anytime, anywhere, any device” connectivity needs.

The growing frequency and growing impact of advanced persistent threats (APTs) – coupled with the recognition that spending alone cannot sufficiently protect their organization – is driving a renewed interest in threat hunting. Cybersecurity leaders recognize that passive controls and existing security technologies are limited in terms of what kinds of malicious activity they can uncover, and how quickly and efficiently they can do so. In contrast, threat hunting is the proactive approach of uncovering the threats that linger within the environment. And like the threat adversaries that they are up against, threat hunting relies as much on human savvy as on technology.

The good news is that using existing SecOps personnel, and the technologies already in place, organizations can ramp up an effective threat hunting capability rapidly.

What Is Threat Hunting?

Threat hunting is a proactive approach to cyber defense that’s predicated upon an “assume breach” mindset. A threat hunter commences work with the operating assumption that a breach has already occurred; the hunt is a methodical search for evidence of the attackers’ presence.

Because it enables security teams to address gaps in their existing operational processes and tooling, security organizations that implement hunting programs can dramatically improve their overall security posture.

Let’s walk you through how modern threat hunting tactics can help your enterprise reduce risk and increase efficiencies, all while revolutionizing your data cloud strategy for optimal security.

Build Threat Hunting Capabilities

Unlike most other SecOps roles, the threat hunter will purposefully seek out evidence of malicious activities that did not generate security alerts, using a methodical approach and multi-dimensional data analytics tools. The primary objective of threat hunting is to intercept potential attacks before damage is done, or to mitigate the damage of an attack in progress.

Effective threat hunting relies on a mindset and a methodical approach that allows the security analyst to think like a threat actor and use that understanding to determine what clues to look for to identify an attack underway. While experience certainly helps, the ever-changing landscape of threat actors, and their sophistication, requires the threat hunter to take a disciplined approach that structures a methodical hunt based on updated tools, techniques, and procedures (TTPs) of top global threat actors. Today’s threat hunter relies on a repeatable framework that allows the hunter to think through the stages of an attack, and then determine the clues or evidence to search for.

In addition, a foundation in core SecOps concepts and technologies allows for efficiency in hunting threats, including: malware analysis, penetration testing, incident response and forensics. Formal training programs are also available, offered through industry organizations like IASCA and the SANS Institute. To further build threat hunting capabilities, you should take the following steps.

1. Think Like Your Adversary

Like a good CIA agent, top threat hunters begin by adopting the mindset of their adversary. Thinking like an adversary allows the hunter to think through how to stage a successful attack. This begins by understanding the common stages that a sophisticated attack might take.

Though adversaries are always seeking to enhance their capabilities by exploiting previously undiscovered vulnerabilities or honing new techniques, the majority of attacks follow the same general trajectory – from initial compromise that gives them a beachhead in the environment, all the way through to data exfiltration (or another means of achieving the objective, like using ransomware rather than exfiltration to monetize their efforts).

Cybersecurity experts and threat researchers have identified six common steps of a typical sophisticated attack, or advanced persistent threat (APT).

Understanding these steps allows the threat hunter to define the potential clues of malicious behavior that align with one or more of the stages. This becomes a process of thinking through the tools an attacker might use at each stage, and the trail of evidence – however faint – that would be left behind. In many cases, attackers will seek to exploit existing, sanctioned tools that are already in the environment in their attempt to avoid detection. Thus, when planning, threat hunters think through how to find unusual usage of these commonly used tools, in order to home in on potential cases in which they are being hijacked for malicious purposes.

2. Analyze Your Data in Depth

Threat hunting involves investigating a hypothesized attack scenario, rather than following up on an alert that existing security tools have generated.

Lacking the clear-cut evidence that would trigger an alarm, threat hunting requires the hunter to gather intelligence, by conducting various analyses on the data in the environment. Indeed, the most successful hunt teams rely on large-scale data aggregation and analysis that go beyond many other use cases of log data (e.g., IT monitoring).

One of the most important determinants of a security organization’s hunting ability is the quantity and quality of the log data it collects and makes available to the SecOps team. A majority of security professionals believe that enriching the systems in their security operations center (SOC) with additional data sources is the most important step they could take in order to enhance their threat hunting capabilities.

Broadly speaking, threat hunters need access to both host and network data sources as well as cloud application logs. Host logs can be collected via an agent or through native logging applications like Windows Event Forwarding, the Sysmon utility, auditing services for Linux architectures, or unified logging for MacOS. These logs should provide visibility into how configuration management utilities like PowerShell are being used within the environment, since these tools are commonly exploited by attackers seeking to maintain persistence while keeping a low profile.

Which specific data sources are needed for a particular hunt depends on the hypothesis that’s under investigation. Standard knowledge bases and frameworks such as MITRE ATT&CK associate a list of data sources that can be examined for evidence of each TTP they include.

Because of the ever-changing landscape of both the IT organization and the global cyberthreat landscape, data platforms hunters rely on must have the ability to ingest and index a wide variety of data types from a wide variety of sources at speed. They also need to be flexible enough to incorporate additional sources without needing to re-extract, transform, and load (ETL) the original data set.

Data Sources with Hunt Types: Examples of Attack Scenarios

Many different threat groups and adversaries make use of malicious PowerShell commands.

Attackers with elevated privileges can remain undetected for long periods of time while performing exploratory, command, and control (C2) and malicious file execution activities using this powerful Windows scripting environment, which can execute commands locally and on remote computers. Attackers often leverage PowerShell to maintain persistence in an environment without needing to install malware. Data sources required to hunt for PowerShell-based attacks include DLL monitoring, file monitoring, PowerShell logs, process command-line parameters, process monitoring, and Windows event logs.

In another illustrative example, hunters might search for signs that attackers are leveraging Microsoft’s Component Object Model (COM) – a set of standards that enable Microsoft Office products to seamlessly interact – to execute malicious code, manipulate software classes in the current user registry, and through these activities, maintain persistence without being noticed.

If you’re looking for evidence that this technique has been employed, you should include DLL monitoring, loaded DLLs, process command-line parameters, process monitoring, and Windows registry monitoring among your data sources.

The third case in point: To locate valuable data or other resources of interest, attackers typically undertake a discovery and exploration process, moving across the network to figure out the lay of the land.

While doing so, attackers often attempt to employ tools that allow them to authenticate to remote systems or execute commands on remote hosts. Windows Management Instrumentation (WMI) is frequently used for this purpose by attackers trying to gain remote access to Windows system components. Data sources that can reveal this attack technique to your hunt team include authentication logs, Netflow data, process command-line parameters, and process monitoring.

Access the Data: Central Log Management, SIEM Platform, or Both?

It’s common for security teams to rely on existing SIEM platforms for threat hunting. However, most SIEM solutions were designed and implemented for compliance and reporting purposes and thus may have limitations when it comes to threat hunting.

SIEM Drawbacks

1. Labor-intensive and complex to manage
2. Limited in the number of log data types or amount of contextual information they’re able to ingest
3. Bound by licensing models that make it cost-prohibitive to store data for longer retention periods
4. Subject to performance issues (slow search) as data volumes increase

To level up threat hunting capabilities, security leaders often choose to supplement SIEM with a centralized log management solution. These tools aren’t competitive but complementary.

This strategy has each work in parallel, with the data lake platform accepting logs that are forwarded from the SIEM. It’s also possible to split the data between the two using an automated data pipeline.

Plan Your Threat Hunt 

Once you’ve found the people and instituted the processes, next comes the technology that allows flexible, integrated access to your data.

Initially developed to serve as the user interface for the Elasticsearch search engine, Kibana has grown into one of the most widely used data analytic tools in threat hunting today.

Kibana is both powerful and flexible, allowing threat hunters to conduct a wide range of queries, perform data correlations, and create data visualizations that help uncover the hidden insights within the data sets. Its capabilities include drill-down dashboard building, time series analysis, and the ability to create a wide array of visualizations including bar and pie charts, tables, histograms, and maps. These visualization capabilities allow threat hunters to search through large volumes of aggregated data to quickly identify outliers in a manner that’s efficient and consistent.

Some organizations go further, coupling Kibana with data platforms that enable search and analysis of security log data at scale. This means even resource-constrained security teams can create a security data lake in the cloud to facilitate and accelerate threat hunting.

It is vital organizations retain large volumes of historical log data from a broad array of sources, so proactive threat hunting can take place at a rapid pace – uncovering the clues that even the most sophisticated of attackers inevitably leave behind.

Ready to start the hunt?