An official website of the United States government
Here's how you know
A .mil website belongs to an official U.S. Department of Defense organization in the United States.
A lock (lock ) or https:// means you’ve safely connected to the .mil website. Share sensitive information only on official, secure websites.

The Cyber Defense Review

 
Home > CDR Content > Articles > Article View

Big Data is Dead, Long Live Big Data

By Dr. David Thomson | March 10, 2016

PRINT

The Gartner Hype Cycle, which assigns emerging technologies into 5 regions: Innovation Trigger, Peak of Inflated Expectations, Trough of Disillusionment, Slope of Enlightenment and Plateau of Productivity. In 2014, Big Data was at the edge of the Peak of Inflated Expectations, where the hype has already generated an enormous amount of goodwill through amazing success stories, and on a descent towards the Trough of Disillusionment, where the rate of new successes relative to the Peak creates a depressed sense of its novelty.

Big Data fell off the chart in 2015.

So is Big Data dead? Not so says Gartner analyst Nick Heudecker, whose blog post is entitled “Big Data Isn’t Obsolete. It’s Normal’’ [0b]. An in-depth look of how Big Data fell off Gartner’s list is available for purchase (or via a Gartner account) [0c], but for those whose pocketbooks or (organizations’ pocketbooks) don’t allow, multiple avenues have indicated that the Internet of Things have emerged to take over Big Data’s hype [1-3].

I would posit a slightly different interpretation: as Big Data has normalized, considering Big Data as its own topic has been eschewed in exchange for considering its various theories (data mining, machine learning, natural language processing, etc.) and resulting technologies (Internet of Things, autonomous vehicles, etc.) as their own entities. And, in my opinion, rightly so.

Big Data may be off the hype cycle, but it maintains healthy funding:

The Defense Advanced Research Projects Agency (DARPA) has increased their spending on programs with a Big Data component from $200M in FY14 to $242M in FY15 to $243M in FY16, with 4 programs receiving more than $20M each:

  1. XData ($39M): Develop tools to analyze large data sets with noisy or incomplete data.
  2. Wireless Network Defense ($35M): Develop new technologies and protocols for robust control of wireless networks, focusing particularly on securing current and next-generation products.
  3. PlanX ($25M): A program to develop technologies to operate on the cyber battlefront in a similar way that the military has developed tools and technologies to support kinetic warfare.
  4. Multifunctional Materials and Structures ($23M): Create design methodology for synthetic materials, and build materials/systems that demonstrate multi-functionality value to DoD.

Focusing only on projects which are specified as a core Big Data Program shows DARPA budgeting numbers of $97M/$118M/$165M over FY14/FY15/FY16, respectively, despite removing PlanX from the budgeting; see [4].

Moreover, the National Science Foundation [5] has solicitations for full proposals in the related areas of:

  1. Critical Techniques, Technologies and Methodologies for Advancing Foundations and Applications of Big Data Sciences and Engineering (BIGDATA).
  2. Campus Cyberinfrastructure – Data, Networking, and Innovation Program (CC*DNI).
  3. Computational and Data Enabled Science and Engineering (CDS&E).
  4. Exploiting Parallelism and Scalability (XPS).
  5. Secure and Trusted Cyberspace (SaTC).

These sources of funding are being leveraged across the academic, government and private sectors.

The prevalence of so-called Open Data (open-source data often resulting from government studies) is allowing researchers across multiple disciplines to test hypotheses on relevant data sets and put new theory, tools and techniques or algorithms to the test. The website http://www.data.gov is the US government’s own open data repository, providing open access to nearly 200,000 datasets from across the various government departments.

All of which indicates that data is not dead, but due to ubiquity and maturity, it is becoming normal. Long Live Big Data.

 

 

References

[0]: A. Brantly and J. Frigo, Big Data and Cybersecurity, Available (Online) Big Data and CyberSecurity, accessed 03/2016, (2015).

[0b]: N. Heudecker, Big data isn’t obsolete. It’s normal, Available (Online) http://blogs.gartner.com/nick-heudecker/big-data-is-now-normal/, accessed 03/2016, (2015).

[0c]: N. Heudecker, et al, The demise of Big Data, its lessons and the state of things to come, Gartner Technical Report, Available (Online) https://www.gartner.com/doc/3115022/demise-big-data-lessons-state, (2015) 6 pages.

[1]: A. White, Internet of Things will dwarf Big Data, Available (Online) http://blogs.gartner.com/andrew_white/2015/05/21/internet-of-things-iot-will-dwarf-big-data/, accessed 03/2016, (2015).

[2]: G. Press, It’s official: the Internet of Things takes over Big Data as the most hyped technology, Forbes, Available (Online) http://www.forbes.com/sites/gilpress/2014/08/18/its-official-the-internet-of-things-takes-over-big-data-as-the-most-hyped-technology/#614d9b561aaa, accessed 03/2016, (2014).

[3]: G. B. Peddibhotla, Gartner 2015 Hype Cycle: Big Data is out, Machine Learning is in, KDNuggest, Available (Online) http://www.kdnuggets.com/2015/08/gartner-2015-hype-cycle-big-data-is-out-machine-learning-is-in.html, accessed 03/2016, (2015).

[4]: J. Lutton, DARPA is spending big on Big Data, FCW, Available (Online) https://fcw.com/articles/2015/04/15/snapshot-data-programs.aspx, accessed 03/2016, (2015).

[5]: Critical Techniques, Technologies and Methodologies for Advancing Foundations and Applications of Big Data Sciences and Engineering (BIGDATA), National Science Foundation proposal solicitation, Available (Online) http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=504767, accessed 03/2016, (2015).

PRINT


US Army Comments Policy
If you wish to comment, use the text box below. Army reserves the right to modify this policy at any time.

This is a moderated forum. That means all comments will be reviewed before posting. In addition, we expect that participants will treat each other, as well as our agency and our employees, with respect. We will not post comments that contain abusive or vulgar language, spam, hate speech, personal attacks, violate EEO policy, are offensive to other or similar content. We will not post comments that are spam, are clearly "off topic", promote services or products, infringe copyright protected material, or contain any links that don't contribute to the discussion. Comments that make unsupported accusations will also not be posted. The Army and the Army alone will make a determination as to which comments will be posted. Any references to commercial entities, products, services, or other non-governmental organizations or individuals that remain on the site are provided solely for the information of individuals using this page. These references are not intended to reflect the opinion of the Army, DoD, the United States, or its officers or employees concerning the significance, priority, or importance to be given the referenced entity, product, service, or organization. Such references are not an official or personal endorsement of any product, person, or service, and may not be quoted or reproduced for the purpose of stating or implying Army endorsement or approval of any product, person, or service.

Any comments that report criminal activity including: suicidal behaviour or sexual assault will be reported to appropriate authorities including OSI. This forum is not:

  • This forum is not to be used to report criminal activity. If you have information for law enforcement, please contact OSI or your local police agency.
  • Do not submit unsolicited proposals, or other business ideas or inquiries to this forum. This site is not to be used for contracting or commercial business.
  • This forum may not be used for the submission of any claim, demand, informal or formal complaint, or any other form of legal and/or administrative notice or process, or for the exhaustion of any legal and/or administrative remedy.

Army does not guarantee or warrant that any information posted by individuals on this forum is correct, and disclaims any liability for any loss or damage resulting from reliance on any such information. Army may not be able to verify, does not warrant or guarantee, and assumes no liability for anything posted on this website by any other person. Army does not endorse, support or otherwise promote any private or commercial entity or the information, products or services contained on those websites that may be reached through links on our website.

Members of the media are asked to send questions to the public affairs through their normal channels and to refrain from submitting questions here as comments. Reporter questions will not be posted. We recognize that the Web is a 24/7 medium, and your comments are welcome at any time. However, given the need to manage federal resources, moderating and posting of comments will occur during regular business hours Monday through Friday. Comments submitted after hours or on weekends will be read and posted as early as possible; in most cases, this means the next business day.

For the benefit of robust discussion, we ask that comments remain "on-topic." This means that comments will be posted only as it relates to the topic that is being discussed within the blog post. The views expressed on the site by non-federal commentators do not necessarily reflect the official views of the Army or the Federal Government.

To protect your own privacy and the privacy of others, please do not include personally identifiable information, such as name, Social Security number, DoD ID number, OSI Case number, phone numbers or email addresses in the body of your comment. If you do voluntarily include personally identifiable information in your comment, such as your name, that comment may or may not be posted on the page. If your comment is posted, your name will not be redacted or removed. In no circumstances will comments be posted that contain Social Security numbers, DoD ID numbers, OSI case numbers, addresses, email address or phone numbers. The default for the posting of comments is "anonymous", but if you opt not to, any information, including your login name, may be displayed on our site.

Thank you for taking the time to read this comment policy. We encourage your participation in our discussion and look forward to an active exchange of ideas.

Help & Support

Resources

Hosted by Defense Media Activity - WEB.mil Veterans Crisis Line