Big Data and Analytics in the U.S. Military

Each day, approximately 2.5 quintillion bytes of data are generated.

Imagine, if the massive amounts of data humans produce every day is shaping the way people live and conduct business, what is happening to the U.S. military?

For a deeper dive into today’s challenges and solutions, keep reading to discover how the U.S. military is handling the big data movement.

Big Data Movement

Data is a vital weapon system in modern warfare. It travels from computation to combat. Information is the highway for the economy, but also for the U.S. military too.

Command and control need big data as a domain. It will determine the results of current and future conflicts. Although the data itself will not decide the outcome but play a key role in how the Army operates and wins its future combats.

The new arms race across the globe is about information dominance, and it is about how the U.S. military uses the data they have.

What Is Big Data?

This is a term that all industries use, not just the military, to describe large volumes of data that are challenging to manage. It includes data that is structured and unstructured.

However, the volume of data is not as important as how you manage it.

What Are Analytics?

It is a complex process that examines big data to discover information. You can find correlations, hidden patterns, and other insights.

Analytics can include tools, methods, and applications that collect, process, and derive those insights from high-velocity data.

The Mismatch of Military Input and Outputs

The U.S. military is struggling with information overload. There is a mismatch between what is coming in, like global positioning systems, full motion video, network services, and more, compared to what is going out, like mission orders, intelligence estimates, target lists, and more.

With the tremendous waves of data that are drowning military operations, it can cause indecision, and ultimately, errors. You may have heard this issue referred to as the “input-output problem.”

While this is a big hurdle, that does not mean they cannot overcome it. With such an influx of data never seen before, the U.S. military will need to leverage artificial intelligence to decide. With big data, analytics, and AI, overwhelming data is no longer their vulnerability, but a strength.

Once they are able to use autonomous systems to their advantage, the military will enter a modern war with a clear vision instead of the fog of massive amounts of big data. When leaders take the big data movement and apply artificial intelligence, they can discover trends, break through a typical bureaucratic structure, and make quick decisions with the right information.

Structural Inefficiency

The U.S. military has staff that attempts to tackle processing their data. However, a human being can work at a human speed. This is a huge limitation.

However, to truly understand the “input-output problem,” you must consider these two critical factors. These factors are structure and process.

Think about it in terms of a plumbing system, for example, and water is flowing through it. How long will it take for the water to flow through? This is determined by two things: the width of the pipe, and the total distance it must travel.

Now, take the U.S. military. They are trying to get too much water through pipes that are narrow and long.

The U.S. Military Structure

The “plumbing system” for the military is based on a simple concept. All military branches concentrate their capabilities on an adversary at one time and in one location. Their goal is to overwhelm the adversary’s decision-making capabilities.

They can achieve their goal with the creation of a unified command, which is made up of all the service branches, and one commander. This structure has given the military enormous benefits, such as flexibility, unity of effort, and the power to overwhelm battle effects. However, there are disadvantages too, such as that it is often inefficient and costly.

Therefore, it contributes to the input-output challenge. The reason is that it forces too much redundancy, as each tier of leadership must process information. When you combine a tiered formal hierarchy, each with specializations, and a unified command, the result is often inefficiency.

Because the military is handling such structural inefficiencies with how their information flows, another result is a waste of time and effort. Hence, operating is costly, and a waste of funding.

Leveraging Joint Resources Is Challenging

In this structure, all commanders need to process requests that go up the command chain until it goes to the resource-controlling commander. A request for joint capability takes too long, and it shouldn’t need to take so long.

Hierarchical organization, such as what you find with tactical military units, has a significant feature where the coordination costs increase the farther down you travel in the structure.

The lowest echelon of command in the United States Army which, through a formal process, requests joint capability is the battalion level. The primary tool for U.S. military units is the target process. They request joint capabilities and request to generate an effect on targets.

A surgical raid, missile strike, or electronic warfare could take the form of an effect.

What Does This Mean For the Battalion?

Every echelon of the joint task force must conduct deliberate planning. Then, they would need to request scarce joint resources.

Between the joint headquarters and a battalion, there are six tiers of leadership at minimum. Before they can decide, a joint capability request will need to travel through every tier of leadership, which again, is at least six.

Now, apply some math here to the span of the army’s control. Consider three or five such requests per command tier. Hundreds, possibly thousands of target requests, are traveling through this hierarchy system at any time.

As the organization gets larger, the requests increase exponentially. The joint force commander’s staff cannot see or hear all such requests. However, even if it is too much, it still requires processing.

How Does the U.S. Military Process Big Data?

This is not a situation to take lightly. This tremendous amount of data processing. In fact, consider those entire units exist just to process joint capability requests at the highest levels in the US Army.

As it stands now, these requests are prioritized, sorted, and allocated by human beings, at a human speed. The ones processing the requests are up to the challenge, such as airmen, marines, sailors, and soldiers. However, what they are not considering is how long it takes to get through a six-step process to complete the joint targeting process.

Assessing an Executed Target

Now, after there is an execution of a target, the staff at the joint force command determines the target execution’s effectiveness. This would involve receiving and processing data from various sources, like unmanned systems, maneuver units, reconnaissance units, electronic/signal intelligence systems, along with many others.

The assessment data will need to process in the same structure that was outlined previously, besides the same costs associated with it.

Okay, so here comes an important question. Who can perform this work? Or rather, who has the time to, anyway?

The Problem: Time

So, can you guess the real culprit in the input-output challenge? If you said “time,” you are correct. This is a potentially dangerous scenario, though.

They can force tactical units into a dilemma when there is a scarcity of resources. They could take the time needed to process inputs but then run into a situation where time has run out to submit the joint target requests. Another scenario, to cut corners and trim time, is that the tactical unit generates its output while limiting the input processing time to speed things up, but instead, they create an error.

Regardless of either scenario, it will not lead to the outcome of the nation and its military need.

What Is the Solution?

This is the perfect time to insert artificial intelligence. It has the unique position to address the problem that the current input-output faces. AI has a superior ability to tackle the massive amounts of data that overwhelm the human workers in military staffs.

Look at autonomous systems. They must find trends and then analyze the patterns.

There are variations in AI, however, autonomous systems still follow the same basic process, which is as follows.

1. Collect data
2. Process data
3. Generate action

There are two building blocks that are critical to this process. These are data processing and data collection.

Data Collection Capacity

For a functioning autonomous system, abundantly available data is a fundamental requirement. Such data pools drive the machine-learning process.

A programmer “trains” a system by assimilating large amounts of data with labels. This is how they can find tiny correlations with the input-outputs.

For example, think of Amazon’s Alexa or Apple’s Siri, which is speech recognition, and chatbots, which process natural language. Even machine vision is a good example, like Google’s self-driving car.

The amount of data and the type of data that artificial intelligence needs depend on how complex the task is. Although, the magnitude is robust.

Gathering Data Can Be a Challenge

A significant barrier to entry can be the data available. Although, for the military, this mismatch is the number one reason artificial intelligence can help them succeed. By its iterative design and how it relies on enormous amounts of data, military operations have the data pools needed to jumpstart the process.

At a closer look, the basic process for autonomous systems mirrors military planning methodologies. As operations officers set their plans, this is gathering data. When they prepare information, this is like processing it.

Both autonomous systems and military organizations generate actions from the information data they gather.

Data Processing Speed

The military needs decisions quickly. This means processing data fast. There is a distinguishing speed at which an autonomous system can operate.

With data processing, machines are faster than humans. Thus, systems that are artificially intelligent have a huge processing advantage.

New Planning Methodologies

Where does the U.S. military go from here? A practical solution is to condense their planning methodologies. One way is to push orders down from higher echelons.

Currently, junior officers have the task of sifting through irrelevant, repetitive information that comes from higher orders. Choosing which items to analyze further is tedious and time-consuming. It is also costly.

If the military were to use artificial intelligence to assist and give “baseline courses of action,” this can speed up the process. Often, most low-level tactical tasks are routine enough that it wouldn’t be necessary to deviate from standard operating procedures.

Artificial intelligence can think faster, so it will buy the army more time to act.

At NSTXL, we are raising the bar with a unique approach. NSTXL combines end-to-end government acquisition services and commercial best practices to boost innovation for a safer future.