Cyber Defense Review

Critical Infrastructure Exercise 16.2 – A Transformative Cybersecurity Learning Experience

By Doug Rapp, LTC Ernie Wong | August 08, 2016

With an increased national awareness that the critical infrastructure which keeps our country running is surprisingly vulnerable—not just to physical attacks, but also to cyberattacks that can be initiated from anywhere in the world—the State of Indiana executed CRIT-EX 16.2 on the 18th and 19th of May, 2016, at the Muscatatuck Urban Training Center. This cyberattack readiness exercise focused on improving Indiana’s overall security and responsiveness of its critical infrastructure to face advanced cyber disruption of essential water utility services – presenting an extreme public safety threat. Indiana, like the rest of the country, understands it has a short window of opportunity to prepare for a major cybersecurity event that, if successful, could be as devastating as a major earthquake or tornado. In order to effectively prepare for such a scenario, Indiana’s cybersecurity stakeholders realized they had to build high-functioning, collaborative networks that span the public and private sector. By working to collaborate on high-risk cyber issues, organizations throughout Indiana are elevating their response postures, and preparing to ratchet up their ability to confront the threats of tomorrow [1].

CRIT-EX 16.2 attendees tourthe FBI’s national Mobile Command Center (photo by Ernest Wong)

“This exercise explored the intersection between critical infrastructure and cyber security,” explained Jennifer De Medeiros, Emergency Services Program Manager for the Indiana Department of Homeland Security [2]. The Indiana Department of Homeland Security (DHS) in conjunction with the Indiana National Guard, Indiana Office of Technology, Cyber Leadership Alliance, and over 16 other public and private partners developed this controlled functional cyberattack exercise allowing participants to deploy resources and communicate with response partners to mitigate adverse effects and expedite recovery. Additionally, CRIT-EX is the first joint public-private partnership simulating responses to cyberattacks on the Muscatatuck water treatment plant, with expert programming and cybersecurity teams acting as cyberterrorists who attack the facility’s Supervisory Control and Data Acquisition (SCADA) systems [3]. Because most of these systems are controlled by civilian organizations that are not tasked with the defense of the nation’s infrastructure, the SCADA systems are not adequately hardened or secured from cyber hackers who could access their data, compromise their systems, and cause serious damage. Consequently, such hackers can hit these systems with malware and false commands resulting in real damage in the physical world, causing machinery and other systems to malfunction or shut down. As a consequence, a small cyberattack on one part of the system can lead to a major disaster over a much wider area since much of our infrastructure is interconnected [4].

During the two-day exercise, members of Frakes Engineering, a systems integration company that specializes in designing and integrating control systems, and cyber security teams from Pondurance and Rook Securities acted as cyber terrorists intent on disrupting and damaging the Muscatatuck water treatment plant. Participants helped to clearly expose and vividly illustrate just how vulnerable our SCADA systems are to a persistent cyber adversary. Additionally, representatives from the FBI SCADA Fly Team, Industrial Control Systems Cyber Emergency Response Team (ICS-CERT), the 172nd Army National Guard Cyber Protection Team, and the Indiana Information Sharing and Analysis Center (ISAC) were on hand to practice coordinating initial recovery efforts and offer professional response guidance.

Members of Rook Security, Pondurance, and the Indiana NG Prepare for CRIT-EX 16.2 (photo by Brad Staggs)

Tony Vespa, board member of the Cyber Leadership Alliance, remarked, “Getting more than 18 public and private entities to willingly participate in an exercise such as this was a huge challenge, and the team had to essentially create a cyber stone soup to get everyone to the table.” Vespa highlighted that CRIT-EX had three very important aspects that differentiate it from other cyber exercises: use of a common language for all exercise participants, a strict adherence to keeping all uncovered cyber vulnerabilities confidential and private, and the employment of real-time and complex cyberattack vectors in which the participating organizations could experience not only disruptions in the cyber domain, but actually see resulting consequences in the physical domain as well.

In order for all of the different entities to communicate effectively with one another throughout the exercise, the participants all agreed on a common language. After months of struggling to understand the unique requirements of each of the various critical infrastructure sectors, the planning team decided that the US-DHS Homeland Security Exercise and Evaluation Program (HSEEP) would serve as the common and unifying exercise language. Ultimately, this proved invaluable to the success of the exercise.

As participation increased to include all six of Indiana’s major water companies, privacy issues soon began to surface. Participating organizations and companies became extremely concerned about the disclosure of cyber vulnerabilities during the exercise that would leave their systems exposed. Additionally, the risk of unmasking regulatory compliance deficiencies left many wondering whether a water utility would be willing to accept such a potential risk. The exercise planners tackled these serious concerns by addressing them at the outset. First, participating teams from the various water companies were staggered throughout the exercise so that no team would have the chance to observe the deficiencies of another team. Trained observer teams from Indiana University, Purdue University, and the Indiana National Guard were all vetted, and served as neutral third-party evaluators. Additional evaluators came from the ranks of the Indiana InfraGard, a cooperative undertaking between the FBI and an association of businesses, academic institutions, and state and local law enforcement organizations dedicated to increasing the safety and security of Indiana and US critical infrastructures [5]. Finally, Red Team debriefings and after action reviews (AARs) were conducted in closed-room session at the end of the exercise, where all teams were informed that all notes, recordings, and images pertaining to a team’s performance would be returned or destroyed at the end of each AAR. Creating, emphasizing, and delivering on this level of confidentiality for CRIT-EX 16.2 engendered greater trust between all the organizations involved.

The final aspect which truly set CRIT-EX 16.2 apart from other cybersecurity exercises was that it linked virtual cyber security disruptions to actual physical disruptions at the Muscatatuck Urban Training Center. Cyberattack vectors were conducted in real-time on a fully functional water treatment plant; the effects of those virtual attacks were monitored from a control room fitted with screens displaying not just systems controls, but also closed circuit monitors showing water main disruptions and spillage based on the cyber hacker’s manipulation of the utility’s SCADA system. The participating teams consisted of system operators, executives, and supervisors familiar with incident response plans, which provided a multi-echelon perspective on both the cyberattack and its physical effects. Adding to the exercise complexity was the selection of the evaluation criteria used to judge the effectiveness of each water utility’s in-place cybersecurity measures. To help alleviate concerns from the participating water utilities that new standards were not being created specifically for the exercise, the CRIT-EX 16.2 planners emphasized key principles of the Homeland Security 32 core capabilities [6], and evaluated cyber security controls and measures using National Institute of Standards and Technology (NIST) recommendations for the security of industrial control systems (ICS) [7] and the American Water Works Association’s (AWWA) standards [8].

 

Exercise Controllers Describe the Effects of a Cyberattack on a Water Company’s SCADA Systems (photo by Brad Staggs)

Water plant superintendents and operators who attended and participated in this real-world exercise left with a more comprehensive understanding of the importance that the AWWA places on key areas of process control, which include:

  • Governance and risk management,
  • Business continuity and disaster recovery,
  • Server and workstation hardening,
  • Access control,
  • Application security,
  • Encryption,
  • Telecommunications, network security, and architecture,
  • Physical security of process control system (PCS) equipment,
  • Service level agreements (SLA),
  • Operations security (OPSEC),
  • Education, and
  • Personnel security [9].

Additionally, the participants and attendees gained a better appreciation for the ten basic cybersecurity measures that the Water Information Sharing and Analysis Center (WaterISAC) has espoused to improve cybersecurity:

  • Maintain an accurate inventory of control system devices and eliminate any exposure of this equipment to external networks,
  • Implement network segmentation and apply firewalls,
  • Use secure remote access methods,
  • Establish role-based access controls and implement system logging,
  • Use only strong passwords, change default passwords, and consider other access controls,
  • Maintain awareness of vulnerabilities and implement necessary patches and updates,
  • Develop and enforce policies on mobile devices,
  • Implement an employee cybersecurity training program,
  • Involve executives in cybersecurity,
  • Implement measures for detecting compromises and develop a cybersecurity incident response plan [10].

 

Figure 1: Types of Computer Attacker Activities and Controls Designed to Thwart the Attacks [11]

With a “Cyber 9/11” and a “Digital Pearl Harbor” at the forefront of homeland security discourse today, Indiana’s CRIT-EX 16.2 has helped to illuminate not only how devastating it can be when we lose control of the computer systems that manage our critical infrastructure, but also just how easy it can be for adversaries to hack their way into our SCADA systems. The exercise was an eye-opening experience for many of the attendees to witness in real-time just how quickly and furtively an advanced persistent cyber threat can compromise key control systems and damage critical infrastructure. Perhaps most importantly, CRIT-EX 16.2 was a transformative learning experience that has helped to equip utility executives and operators with improved cyber control measures (see Figure 1) that make it more difficult for cyberattacks to gain and maintain access, improve readiness, and help bolster our national security.



US Army Comments Policy
If you wish to comment, use the text box below. Army reserves the right to modify this policy at any time.

This is a moderated forum. That means all comments will be reviewed before posting. In addition, we expect that participants will treat each other, as well as our agency and our employees, with respect. We will not post comments that contain abusive or vulgar language, spam, hate speech, personal attacks, violate EEO policy, are offensive to other or similar content. We will not post comments that are spam, are clearly "off topic", promote services or products, infringe copyright protected material, or contain any links that don't contribute to the discussion. Comments that make unsupported accusations will also not be posted. The Army and the Army alone will make a determination as to which comments will be posted. Any references to commercial entities, products, services, or other non-governmental organizations or individuals that remain on the site are provided solely for the information of individuals using this page. These references are not intended to reflect the opinion of the Army, DoD, the United States, or its officers or employees concerning the significance, priority, or importance to be given the referenced entity, product, service, or organization. Such references are not an official or personal endorsement of any product, person, or service, and may not be quoted or reproduced for the purpose of stating or implying Army endorsement or approval of any product, person, or service.

Any comments that report criminal activity including: suicidal behaviour or sexual assault will be reported to appropriate authorities including OSI. This forum is not:

  • This forum is not to be used to report criminal activity. If you have information for law enforcement, please contact OSI or your local police agency.
  • Do not submit unsolicited proposals, or other business ideas or inquiries to this forum. This site is not to be used for contracting or commercial business.
  • This forum may not be used for the submission of any claim, demand, informal or formal complaint, or any other form of legal and/or administrative notice or process, or for the exhaustion of any legal and/or administrative remedy.

Army does not guarantee or warrant that any information posted by individuals on this forum is correct, and disclaims any liability for any loss or damage resulting from reliance on any such information. Army may not be able to verify, does not warrant or guarantee, and assumes no liability for anything posted on this website by any other person. Army does not endorse, support or otherwise promote any private or commercial entity or the information, products or services contained on those websites that may be reached through links on our website.

Members of the media are asked to send questions to the public affairs through their normal channels and to refrain from submitting questions here as comments. Reporter questions will not be posted. We recognize that the Web is a 24/7 medium, and your comments are welcome at any time. However, given the need to manage federal resources, moderating and posting of comments will occur during regular business hours Monday through Friday. Comments submitted after hours or on weekends will be read and posted as early as possible; in most cases, this means the next business day.

For the benefit of robust discussion, we ask that comments remain "on-topic." This means that comments will be posted only as it relates to the topic that is being discussed within the blog post. The views expressed on the site by non-federal commentators do not necessarily reflect the official views of the Army or the Federal Government.

To protect your own privacy and the privacy of others, please do not include personally identifiable information, such as name, Social Security number, DoD ID number, OSI Case number, phone numbers or email addresses in the body of your comment. If you do voluntarily include personally identifiable information in your comment, such as your name, that comment may or may not be posted on the page. If your comment is posted, your name will not be redacted or removed. In no circumstances will comments be posted that contain Social Security numbers, DoD ID numbers, OSI case numbers, addresses, email address or phone numbers. The default for the posting of comments is "anonymous", but if you opt not to, any information, including your login name, may be displayed on our site.

Thank you for taking the time to read this comment policy. We encourage your participation in our discussion and look forward to an active exchange of ideas.