Cyber Defense Review

Active Defense: Security Operations Evolved

By Capt Mark Manglicmot, CPT Adam Tyra | May 01, 2015

Introduction

During the Hundred Years War, gunpowder artillery was regularly employed in sieges to reduce to rubble fortresses that previously could only have been taken through starvation of the defenders or betrayal. Contemporary cyber defense shares a number of characteristics with fifteenth century land warfare. Many enterprises today still attempt to engage in positional warfare, believing that they can build their networks into an impenetrable fortress. They install firewalls, antivirus, and intrusion detection systems. They hire armies of technicians to maintain and employ various technologies to monitor, detect, correlate, capture, and analyze security information on their networks. In spite of these efforts, networks belonging to the largest corporations and the most powerful nations are regularly compromised and raided with ease by attackers.

Cyber attackers have achieved a level of sophistication that cannot be matched by traditional defensive tactics that envision the network as a fortress with a guarded perimeter. Custom built cyber weapons mated with zero day vulnerabilities are the modern cyber equivalent to renaissance-era gunpowder artillery, except they can defeat defenses in a matter of milliseconds rather than the days required by their earlier fiery analogs. Further, detection often occurs too late. In the case of the high-profile breach reported by Sony near the end of 2014, attackers had been present for weeks or even months installing back doors and stealing data before announcing themselves to the world. A December 2014 Wired magazine article discussing the breach speculated that attackers had stolen, “ trove of sensitive data…possibly as large as 100 terabytes.” (Zetter 2014). They didn’t pull this off in a day, and an effective defense could have stopped them before they were able to cause lasting damage.

The stark reality of the contemporary cyber threat is that any organization that has a digital resource worth stealing, affecting, or destroying, whether it’s proprietary information or a computer controlled electrical transformer, has probably already been breached. They just don’t know it yet. A survey report of 567 executives at major US companies released by the Ponemon Institute in September 2014 stated that 43% of respondents reported a breach in 2014. Another report released by security incident response firm Mandiant in early 2015 stated that the average time elapsed between the earliest indicators of compromise and the discovery of a breach for their clients was 205 days. The longest elapsed time was a shocking 2,982 days! We can interpret these two figures by concluding either that the average organization was probably breached last year and has not discovered it yet or that it will be breached this year, and it will not discover the breach until next year.

Components of an Active Defense

Active defense is the fusion of timely threat intelligence with proactive measures that combat specific threat scenarios to yield a progressive reduction in enterprise risk. To attackers, the effect is a continuously increasing level of effort required to succeed in an attack against an enterprise that uses an active defense.

Active defense does not replace traditional security operations. Instead, active defense comprises a toolkit of tactics to be deployed by high functioning security operations teams after they have mastered basic security operations functions. Deploying the tactics described herein requires a functioning security operations center (SOC), a basic vulnerability management program, and a moderate level of data and asset classification. Practitioners will also need leadership buy-in, since active defense tactics can be disruptive or even invasive in some instances.

Knowledge

We have developed concrete tactics and techniques that can be employed by security operations teams to create active defense effects. However, before engaging in active defense, defenders must have knowledge. For, as Sun Tzu famously wrote in the Chinese classic, The Art of War, “[I]f you know your enemies and know yourself, you can win a hundred battles without a single loss.”

 

Figure 1: The Active Defense Lifecycle

 

Know thy self

Although Sun Tzu mentioned the enemy first, self-knowledge is the more important kind. The first step for organizations in defending themselves against today’s advanced attackers is to understand what makes them a target.

Thoughtful conversations between security practitioners and business leaders should result in identification of the top ten business functions, critical platforms, important applications, and/or sensitive data repositories. Relevant assets are generally those that risk serious or grave consequences for the enterprise should they be manipulated or stolen or become unavailable. Examples include intellectual property, research and development data supporting future innovation, employee or customer personally identifiable information, payment card information for clients, and the industrial control systems that support critical business functions. Once these items have been identified, they must be prioritized for defense utilizing a risk based approach. If the organization did not previously have a risk management strategy, they now have an ad-hoc strategy since they have de facto accepted some risks and begun laying plans to mitigate others.

Besides understanding what is valuable on the network and where it is stored, building an effective defense also relies heavily on an understanding of what “normal” means for the network. Typically, this is referred to as a “baseline” in the context of security. However, much of this baseline lives in the minds of the IT staff rather than in security monitoring tools. We will discuss this further in the anomaly analysis section.

Know thy attacker

Self-knowledge is a core component of any security program, but active defense is powered by insight about likely attackers. With an understanding of the organization’s valuable assets, defenders can develop a list of potential threat actors and determine the tactics that they might employ. Consider the answers to questions like, “Who would want our research data?”, and listen for statements by leaders similar to, “If our floor price for buying Company DELTA is revealed, we’ll surely get undercut by Company ECHO.” These statements reveal the likely targets of an attack and help paint a picture of the most likely attacker. Open source research is usually sufficient at this stage.

Intelligence

Once likely adversaries have been identified, the organization should attempt to map their tactics, techniques, and procedures (TTPs) to a kill chain. Any version of a cyber-kill-chain will do, but the version proposed by Lockheed-Martin in “Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains” came first and has become the de facto standard.

Data points that should be collected and mapped for all attack groups include:

  • Attacker source IP ranges
  • Malware metadata
  • Typical hardware/software leveraged by the attacker
  • Typical hardware/software targeted by the attacker
  • Typical times of attacker operations (i.e. typically active between 0400 – 0900 GMT on Thursday, Friday, or Saturday)

For each critical function of the business, the following characteristics should also be gathered:

  • Hardware/software used to access the sensitive data and business processes
  • Patch level and patching schedule for identified hardware and software
  • Previous attack information (If available)
  • Detailed identity and access information associated with the resources (i.e. who can access them, and what privileges do they have?)

Active defenders should also gather intelligence about current events in their industry to determine who is attacking their peers and for what purpose. Peers are a great source to develop first-hand insight about the latest TTPs used by attackers.

Tactics

Supplied with a thorough understanding of their organization’s weak spots and critical assets along with a vision of likely adversaries and their likely adversary courses of action, defenders can activate their cyber defense with a range of tactics. The following are descriptions of several tactics that could be employed against a notional adversary to counter a number of potential threat scenarios.

Active Hunting

The typical security operations team focuses on reactive detection based on signatures or simple event correlation. Thus, advanced attackers must structure their activities to accomplish two goals when operating in a defended environment. First, they must evade signature based detection. Second, their activities must blend in with normal user behavior.

Cyber-attacks are comprised of a chain of events and actions that evolve over time. Signatures don’t evolve. This means that evading signature-based detection is trivial for advanced attackers once they know a particular tool or tactic is “burned”. They just innovate something new. In response, advanced teams expand their monitoring to include trending analysis based on event type over a period of time. Although this method retains its effectiveness, it is entirely dependent on the idea of “normal” in the network. If attacker activity in the network is common, attackers will thwart trending analysis as well.

Anomaly analysis using signatures and trends can be evolved by increasing the velocity with which defenders absorb new intelligence and iterate novel types of behavior to hunt. Once a steady stream of intelligence has been integrated into the security operations team, defenders can continuously develop new signatures for bad behavior and improve their behavioral models. Not only will this improve defenders’ ability to detect malicious behavior, it will also reduce the amount of time that attackers can hope to use newly devised tactics.

Examples of Anomalous Activities

Anomalous activity is any activity that is strange, abnormal, or doesn’t belong in the context in which it is seen. This context could include the user who is engaging in the activity, the time when the activity is observed, the frequency with which the activity occurs, and other circumstances. The following are a few types of data that a security team should baseline (as much as possible) and monitor for deviations:

  • Traffic type by time
  • Traffic by business/geographic area.
  • Traffic/activities by user group.
  • Device-to-device behavior.
  • Data movement.
  • File compression usage (zip, rar, etc.).
  • Host level activity. Look for deviations from normal activity including:
    1. Running processes including the name, path, and parent process
    2. Autoruns – look at all of the processes that start automatically when the system starts
    3. Third party programs that may give themselves higher level privileges
    4. Kernel drivers – these have full access to system and its resources, so they must be interrogated

In addition to hunting for anomalous activity in new event streams, defenders should ensure that they apply newly developed rules to historical data as well. The time when defenders become aware of a particular malicious behavior is always after the time when attackers began using it. Thus, historical logs much be searched to ensure that a compromise hasn’t already occurred due using a tactic.

Spoiling Cyber Staging Areas

Attackers often form a beachhead within a compromised network. This is a host from which they launch sorties against other hosts on the network and on which they may store stolen data. Often this data is compressed, obfuscated to look like something it isn’t or even encrypted. In one instance, we saw a large data cache rolled into several encrypted and compressed RAR files which then had their file extensions altered to make them look like video clips.

This beachhead concept is important, because hackers must prepare a staging ground within one or two “hops” from a location on the network from which data will be stolen. Not only is this required in order to limit the amount of activity on a target host to prevent detection, but routing connections and data through additional systems is technically complicated and subject to discovery as well.

Defenders should search likely beachhead locations near sensitive systems for stolen data and stored tools. In enterprises that enforce data storage locations for users, such as those that require all personal files to be saved to a network-shared folder, this search can be straightforward. Searching may also be aided by enterprise file naming schemes. These often aren’t apparent to outsiders, so attackers may create filenames names that automatically signal themselves away as anomalous.

Cyber Clear and Hold

Clear and hold is a counter-insurgency strategy employed to prevent enemies from re-occupying territory from which they have been ejected by defenders. After the clearing stage, the holding stage is usually characterized by regular patrols, surveillance, and the improvement of defenses. The cyber equivalent is to maintain a high level of scrutiny on hosts that have been previously inspected for anomalous activity.

A clear and hold mission may be warranted due to a number of internal or external factors. Defenders may learn about an attack against an industry peer and may wish to apply clear and hold tactics to protect the data types that were taken in that attack. Another driver could be the discovery of an un-patchable vulnerability in a critical system. Hosts on the same network segment could then be cleared to ensure that they are not currently hosting attackers who could take advantage of the weakness.

Activities of this nature can usually only be sustained for a brief period of time before resources must be redeployed to other areas. For example, a clear-and-hold mission would be appropriate during the period when a merger/acquisition is being planned (from the earliest stages) and executed. Once the merger is announced publicly and completed, the protection provided by clear and hold tactics is no longer necessary around the systems containing merger data.

Clear and hold can be executed on any part of the infrastructure that must be kept free of infiltrators. Common examples across many industries include:

  • Research and development networks (short and long term)
  • Network segments and resources used by teams working on mergers and acquisitions
  • Network segments and resources used by VIP users (executives and their assistants)
  • Network segments used by industrial control systems
  • The network’s demilitarized zone
  • Network segments used by systems that process financial data

Clearing Hosts

To conduct a cyber-clear-and-hold mission for a set of hosts, first select a sensitive enclave. Then, interrogate the hosts within this area. Conduct a frequency analysis of artifacts across hosts to identify least frequency filenames, service names, and automatically executed processes. A typical attack scenario would only result in modification of a few hosts, so defenders should focus their investigative efforts on the items that only appear a handful of times. Also defenders should conduct active hunting of hosts based on the most recent intelligence-derived indicators. Consider a limited ‘surge’ deployment (limited in scope and time) of a different technology to monitor or scan from a different perspective than normal operations allow.

Clearing Network Segments

To clear a network segment, defenders should start by examining the connections coming in and out of the enclave. Do any connections seem out of place?  Are there remote desktop connections? Is FTP enabled where it should not be? Examine low density connection types (host-to-host) for evidence of lateral movement.  Implement additional network segmentation (new subnets, VLANs, etc.) to further isolate high risk segments. These changes could be permanent for long-term threat scenarios, or they could be shorter term for scenarios that will not persist into the future.

Investigate all matches or “hits” encountered during examinations even if they can’t be matched to malicious activity. Evidence of failed intrusions is just as useful as evidence of successful ones. Defenders should make an effort to ‘war-game’ potential attacks as they could have occurred if they had succeeded. This will provide some insight that can be leveraged to reinforce defenses in threatened areas of the enterprise.

For the ‘hold’ part of the tactic, consider implementing highly specific and targeted SIEM rules. Tightly focused correlations and reports can be created along with the proactive countermeasures described above. These can be applied to specific subnets, data files, or other behaviors unique to hosts or segments of concern.

Cyber Recon by Fire

Reconnaissance by fire is a tactic used by ground forces to “check” areas for enemy forces without exposing themselves to attack. Advancing troops fire into clumps of bushes, structures, and other areas of cover and concealment to force adversaries to either return fire or withdraw from the area. In this way, they can trigger ambushes or cause enemies to reveal themselves.

In the same spirit, cyber recon-by-fire allows defenders to hunt for malicious activity by making (not so) subtle changes to the network that could draw out an intruder. A form of signature-less security, it leverages manipulation of the enterprise infrastructure in conjunction with monitoring tools to enable detection. Naturally, these activities require close coordination between the security operations, networking, and leadership teams before being undertaken. Reconfiguring infrastructure devices and servers in-flight may impact the user experience of employees or temporarily degrade service quality.

DNS Manipulation

Malware authors typically use hostnames to configure malware command and control servers rather than IP addresses. This improves resiliency for the malware, since defenders typically block outgoing traffic to specific IP addresses (routers and switches don’t know about hostnames). Using a hostname allows the malware’s command and control server to be located at any IP address. The attacker just needs to register it, and DNS servers around the world will carry the news to his deployed malware. Defenders who have tried to squash a malware infection have probably seen this behavior before. They block outgoing traffic from beaconing malware only to see it shift to new destination addresses every few hours.

By resetting the network’s DNS cache, defenders force renewed resolution of every hostname across the network- including those used by malware. Within a few hours or days, defenders can then examine the contents of the DNS cache for low-density hostnames or hostnames that were resolved at odd hours. A boat-load of connections to www.google.com at noon on a Tuesday shouldn’t raise any eyebrows, but a single connection to www.malwaremothership.com at 2 AM on a Tuesday warrants closer inspection.

Credential “Crazy Ivan”

Cold War era submarines had an area immediately behind their propulsion system known as the “baffles”. This area of noisy churned-up water was a blind spot for the hull mounted sonar systems of the day, because the noise of the engine masked any other sounds in the baffles. In order to ensure that enemy submarines weren’t following in this blind area, Soviet subs would execute periodic random hard turns to one side or the other to get a glimpse into the blind spot. Similar unpredictable behavior on the network can reveal intruders.

Attackers who have reached an advanced stage of the kill chain after having persisted in the network and escalated privileges are unlikely to engage in further overtly malicious activity. However, their access probably rests on the use of legitimate account credentials that they stole or credentials for illegitimate accounts that they created. Consider depriving them of this access by randomly resetting a large amount of account credentials. This is one of the more disruptive tactics that we would propose outside the context of a confirmed intrusion. However, defenders can gain tremendous insight from this exercise.

Once the credentials are reset, watch for changes in network behavior. Closely monitor traffic to, from, and within the credential storage location for attempts to steal the new credentials. This activity can signal the accounts that were previously compromised and are thus in use by an intruder. If any user accounts that were active suddenly go dormant, then these accounts were probably exclusively used by an intruder and can be eliminated from the network.

Malware Starvation

Many types of malware emit a regular “beacon” or “heartbeat” to a command and control (C&C) server as long as they are active. This serves two purposes. First, it acts as a remote notification to an attacker that his access to the network is still available. Second, it provides automated control systems with an opportunity to deliver orders to fielded malware instances (implants).

Highly sophisticated attackers may employ multiple cooperating malware implants that watch each other to provide backup. If one implant sees that its partner has been eradicated or is no longer communicating on the network, it activates and takes over the beaconing and malicious activity. We saw one network that had primary implants installed on more than 20 servers with alternate or backup implants hiding on another 14. The alternates weren’t detected until after the primaries had all been eradicated- the point when an incident response team would usually close the case and go home.

Changes in network connectivity are usually the cause that results in the activation of dormant implants. Consider simulating this to “starve” malware of its network access and change its behavior. Defenders can accomplish this in two ways. First, network segments can be cut off from one another temporarily to prevent cooperating malware samples from seeing or interacting with one another. This can result in backup malware spinning up and trying to take over for what it thinks is an eradicated primary.

The second way to starve malware is to eliminate outbound connectivity temporarily for the entire network. This should be undertaken during periods of low network usage (nights and weekends)- not only because it’s disruptive, but also because identifying new beaconing activity will be difficult if it’s mixed with a high volume of active traffic.

We should note that cutting off the entire network isn’t ideal for detection, because new malware instances may activate on practically any host. If defenders don’t know where to look, they might miss new network traffic on any particular backwater segment that isn’t being monitored. In addition, enterprise leaders are unlikely to authorize a long-term network outage for any purpose outside the context of a confirmed security incident.

Conclusion

The U.S. Army’s Field Manual 3-21.8 (The Infantry Rifle Platoon and Squad) describes a listing of priorities for small units preparing a defense. Among these are stockpiling food and ammunition, digging trenches, and establishing communications with nearby allies. At the bottom of the list is, “Continue to improve positions.” Soldiers are advised to develop better camouflage, dig fighting positions deeper, and even to draw sketches of the area to use for reference when calling for artillery support. In cyber defense, just as in kinetic warfare, there is always more to do to prepare an effective defense. Moreover, persistent cyber defense requires progressive elaboration to maintain its effect.

In an era of persistent cyber threat, effective defense requires active and evolutionary defensive tactics. The active defense methodology we discussed meets the needs of modern security operators for an agile and effective framework for intelligence-driven security operations. The main innovation presented herein for most security teams will be the proactive search for intruders outside the context of an incident response effort. However, we believe that the deliberate use of intelligence to focus the defense will be innovative to many readers as well.

After developing a thorough understanding of the resources possessed by the enterprise, security operations teams can employ strategic and tactical threat intelligence to identify the most likely attackers that would covet and attempt to compromise those resources. Next the SOC can build a kill chain to identify the most likely courses of action and tactics that attackers would use to penetrate the enterprise, locate critical data, and steal or manipulate it.  These strategies are not meant to replace the traditional vulnerability management, continuous monitoring, and network operations functions, but they are instead intended to better integrate and enhance them.

By developing and using timely threat intelligence, the SOC can mature a defense specifically targeted at critical resources rather than focusing on the outer perimeter. An active, intelligence driven defense will also improve the effectiveness of security monitoring capabilities which will ultimately drive up the cost of a successful attack and drive down the probability of success for the attacker. The desired end-state is for the attacker to not only fail but also to determine that the enterprise that employs an active defense isn’t worth the effort in the first place.

 

 

Works Cited

  1. The Need for Pro Active Defense and Threat Hunting Within Organizations. Performed by Andrew Case. 2015.
  2. Hutchins, Eric, Michael Cloppert, and Rohan Amin. “Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains.” Lockheed Martin Corporation, 2011.
  3. Is Your Company Ready for a Big Data Breach? Annual Study, The Ponemon Institute, LLC, 2014.
  4. M-Trends 2015: A View from the Front Lines. Annual Report, Mandiant, A FireEye Company, 2015.
  5. Responding to Targeted Cyberattacks. Rolling Meadows, IL: ISACA & EY, 2013.
  6. Tzu, Sun. The Art of War. Filiquarian, 2007.
  7. Zetter, Kim. Sony Got Hacked Hard: What We Know and Don’t Know So Far. December 3, 2014. http://www.wired.com/2014/12/sony-hack-what-we-know/ (accessed February 18, 2015).