Cyberspace operations is not cyber security
The importance of cyber security is manifest across our traditional enterprise information technology systems. Our human and military affairs are network enabled, and therefore potentially vulnerable. The reach of cyber-physical systems is also expanding rapidly, going beyond infrastructure (e.g., the utilities, water resource management, and food supply protection) and entering everything from automotive technology, to household and healthcare networks. A major effort is underway to develop the next generation of cyber security experts through the establishment of ‘academic centers of excellence’ and cyber security programs at a great many US colleges and universities. This matter is particularly acute because the current workforce must possess a great deal of knowledge and a high degree of skill.
Cyber security as a work domain and commercial sector has been maturing rapidly over the past 20 years. Many of the skills and tools required for cyber security have application for cyberspace operations. However, the Department of Defense (DoD) cannot rely on cyber security solutions to satisfy its full range of operational requirements. Cyber security is primarily concerned with keeping computer networks operational and trustworthy. Maintaining network confidentiality, integrity and availability are the guiding principles for security policies and behaviors. Small groups of experts continue to operate and protect well-defined networks and data with their tailored cyber security tools and tradecraft.
Cyberspace operations, on the other hand, are synchronized military activities to identify, degrade and/or deceive threat actors in cyberspace. Joint Publication 3-12 defines cyberspace operations as “the employment of cyberspace capabilities where the primary purpose is to achieve military objectives or effects in or through cyberspace.” In 2009, the DoD established U.S. Cyber Command (USCYBERCOM) to centralize command of joint cyberspace operations, and in 2013 initiated a build-out of a robust Cyber Mission Force (CMF) that is manned, trained, and equipped by the Service Departments.
Cyberspace operations are inherently dynamic due to changing technology and tactics of malicious actors. Recent increases in the number and scale of cyber incidents have illustrated the need for improved coordination across the CMF as well as improved feedback and accelerated technology transition between operational research, and development communities.
Effective, operationally focused research is difficult
Cyberspace operations concepts are still in the formative stage. This spans everything from organizational design within the Service Departments, to joint force coordination, to operational design. Communication and collaboration methods are evolving. The increasing scale and tempo of cyberspace operations matches new decision support tools developed to assist in network mapping, netflow monitoring, and malware detection. Thus, there is a need for research to explore work methods and objectively evaluate new technologies. There is a need for performance evaluation, and metrics that are specific to the emerging technologies. What is a good baseline for performance in cyberspace operations? Information Technology provides necessary but insufficient benchmarks for metrics such as ‘detect in time to prevent’. Our cyber community requires effective measures that evaluate the cognitive workload and operational impact at the ‘systems of systems’ level.
Enterprise-wide expertise is critical, but ephemeral. Until the DoD adopts different personnel policies, the CMF will be staffed with predominantly novice military service members supervised by a few experienced personnel. To complicate matters, many of the most proficient practitioners find lucrative employment in the private sector. Training for cyberspace work requires experiential learning at realistic cyber work activities that have high “cognitive fidelity”. That is, the training experience must place the same demands on attention, reasoning, decision-making, and interaction as actual practice. To be realistic, cyber work simulations must provide abundant opportunities for trainees to engage in network analysis, vulnerability analysis, malware detection, and other cyber activities, on realistic (although often virtual) networks. However, it is not possible to develop scenarios and simulations without establishing (and maintaining) a repository of expert knowledge, and a library of lessons-learned.
Thus, operationally focused applied research is required for converging reasons. Research focused on cyberspace operations must be informed by a knowledge of how experts, journeymen, and apprentices work together to accomplish their tasks. This mandates cognitive task analyses and knowledge elicitation. The cyber professional’s workplace technologies (software systems, visualization systems) shape their strategies and knowledge. This mandates evaluations of the strengths and limitations of automation as well as human-system performance in terms of robust metrics.
Applied research not only dissolves the traditional distinction between basic and applied science, but it necessarily blurs the distinction between research and operations.[1] One cannot easily take actual military cyberspace work out of context, situate it in a laboratory, and study it under controlled circumstances. To achieve a science-based and yet genuine understanding of cyber work one must to some degree let the ‘fog of war’ intrude into the research setting.
In addition to the integration of research and operational communities in experimental applied research settings, the involvement of the development community who might ‘productize’ or engineer the final integrated product is critical to facilitating timely transition and sustainment of technological breakthroughs. The software developers who will eventually integrate the research into a system can provide the research community key insights from an engineering perspective that will enable more rapid technology transition. Because of their day-to-day hands-on involvement with various tools, the developer community may also provide operational use cases that the research or operational community may not always perceive.
Integrated research and training enables a decision support system that is sufficient for training and should also be sufficient for conducting the actual work, and vice versa.[2] Exploratory and evaluative work analysis conducted in the same context as cyberspace training, assures cognitive fidelity. Experience shows that research and training in a ‘virtual distance’ mode does not entail the complexities, practical issues, and networking issues involved in ‘actual distance’ mode. Simply saying, “Room 101 is our virtual Florida, and Room 102 is our virtual Kansas” is not sufficient to emulate actual network-based cyberspace work and the logistical challenges and surprises that it engenders. For example, distant communication may limit bandwidth, increase latency, or simple availability of personnel because of time zone differences—factors easily overlooked when creating a synthetic environment.
The Cyber Immersion Lab is closing the gap
The USCYBERCOM J9 (Advanced Capabilities Directorate) operates the Cyber Immersion Lab (CIL), which is closing the gap between operations and research. The CIL is an adaptable, human-centered facility that develops and assesses cyberspace capabilities as well as conducts experiments to identify technology requirements for the CMF. The CIL includes five project spaces, which can accommodate between ten and twenty-five persons each, and can conduct unclassified, secret and top secret projects concurrently. The CIL’s infrastructure can support off-network projects as well as remote connections to other labs, ranges or operational networks. USCYBERCOM or adjacent research partnerships internally fund CIL projects. The core strength of the CIL is its proximity and access to the CMF in the National Capital Region.
Figure 1 – Cyber Immersion Lab
A recent illustration of the importance of experimentation in cyber operations was a project to evaluate network-mapping capabilities for operational fit in the CMF. The Network Mapping Project, which was sponsored by OUSD(I) SIGINT-Cyber, included multiple complementary investigative strategies. It began with a field study of cyber teamwork to determine evaluation conditions and criteria. It also included market research to identify candidate capabilities and culminated with an experiment that involved 20 members of 10 Cyber Protection Teams. In addition to a report that described the relative performance of the sampled capabilities, the project generated a number of other insights to inform operations as well as research communities.
For example, a key insight from the project was that network mapping, or the rendering of data about networks into a graphical format supports decision-making and information sharing. In the past, network engineers used network maps for configuration management, property accountability, or network planning activities. In cyberspace operations, network maps must support:
- Shared understanding of topology, activity, and vulnerabilities,
- Shared information about adversary and friendly resources and actions,
- Military planning in cyberspace as well as physical military domains.
Because decisions in cyberspace differ across teams and echelons, so the maps to support those decisions must be commensurately varied. There is no universally reliable mapping activity, with cyberspace operations requiring federated sense-making at enterprise scale. Although the CIL network-mapping project did not identify a single existing materiel solution for the CMF, it did generate a rich description of the features that are required. More importantly, it serves as a proof of concept for experimentation to elucidate technology requirements.
Figure 2 – Cyber Team members participate in an experiment in the Cyber Immersion Lab
Campaign of experimentation
Traditional views of experiments – factorial manipulation of variables and tightly controlled circumstances – are unwieldy and under-informative in cyberspace experimentation. First, factorial designs are prohibitively resource and time intensive. For example, evaluating five tools with experienced and less experienced participants would require ten experimental conditions. Each condition would require unique, but equally representative environments to avoid the effects of learning the environment. Emergent operational conflicts challenge the sustainment of participant groups to execute such lengthy designs. Second, tight control of variables eliminates the important messiness of the real world. Isolating unpredictability and complexity of the world results in sterile, and therefore, unrealistic conditions. Cyberspace work is necessarily subject to numerous vagaries and unanticipated circumstances (e.g., the customer does not have a map of their own network; computers do not initialize efficiently, and so forth).
We recommend a ‘campaign of experimentation’ with select combinations of variables or conditions employed across a series of mini-experiments alleviating impractical factorial designs. The cyber community should collaborate in the conduct of more research to achieve the general methodological aim. Future experiments will involve multiple teams of cyber professionals located at multiple physically distributed locations, and orchestrated to conduct network analyses comparing a number of different network analysis tools, or to study different work process models. Going forward, it is crucial that USCYBERCOM and the DoD cyber community broadly, develop a robust methodology for conducting such a Campaign of Experimentation. A recent National Science Foundation report provides a similar call for empiricism to improve cybersecurity.[3]
The key to applied research and experimentation for cyberspace operations is the coalescing of researchers, developers, and the operational community in an environment that supports realistic live, virtual, and synthetic environments that are easily reconfigurable. These three groups enable the evaluation of the science, technology, human factors, engineering development and employment concepts, and the potential operational effectiveness of such research. Effective experimentation results in not only appropriate feedback for the research community but ultimately accelerates technology transition to the operational community. Without these three groups of people working together in an environment that supports loosely controlled experimentation and exploration, the ability to keep pace and advance ahead of our adversaries in the cyberspace domain will continue to be challenging as the adversary uses the Internet continuously to perform this type of research, development, experimentation, and technology transition.
Disclaimer: This paper reflects the views of the authors. It does not represent the position of the Department of Defense or US Cyber Command.
References
[1] R. Hoffman, and K. Deffenbacher, “A multidimensional analysis of the relations of basic and applied psychology”. Theoretical Issues in Ergonomic Science. (2011) DOI: 10.1080/1464536X.2011.573013
[2] R. Hoffman, G. Lintern, and S. Eitelman, (March/April 2004). “The Janus Principle”. IEEE: Intelligent Systems, (March/April 2004), 78-80.
[3] D. Balenson, L. Tinnel, L., T. Benzel, “Cybersecurity Experimentation of the Future (CEF): Catalyzing a New Generation of Experimental Cybersecurity Research”. National Science Foundation. (2015) at: http://www.cyberexperimentation.org/files/5514/3834/3934/CEF_Final_Report_20150731.pdf (accessed on 1 March 2016).