« COIN is Boring or: Th… | Home | Getting Orwellian: Na… »

The New On Violence Bailiwick: Data

While I was attending the Military Intelligence Career Course in 2009, I once badgered a professor about his use of a doctrinal term. We had to read the newest version of Field Manual 2-something “Intelligence Preparation of the Battlefield”. Some of the terms in the new manual didn’t match with slides the MICCC gave us. So I called it out; some of the instructors hadn’t read the new manual. One of them, offhandedly, described my emphasis on doctrinal terms as my “bailiwick”.

That became a running joke in my squad. Since bailiwick is a fairly obscure word (Eric C, though familiar with the word, had no idea how to spell it), my friends in the MICCC claimed I had a new bailiwick everyday. (Other bailiwicks: rebounding in college basketball, rules of engagement, and America’s email addiction.)

Today, I introduce a new bailiwick for this blog. You see, I fell in love last year. Despite the predictions of my quantitative GMAT score, I have fallen head over heels in love with statistics. Specifically, regression analysis. I came to the following conclusion:

Statistics/big data/regression analysis is literally the coolest thing ever.

While falling in love with statistics is still not normal for my generation, it isn’t too unusual either. Want to pwn your friends with sports knowledge? Read Bill Barnwell or Zach Lowe. Want to win election predictions? Read Nate Silver. Want to win elections? Use data to find voters, a la the Obama campaign. Want to win an Academy Awards prediction pool? Eric C has won our family’s Oscar pool for three years now using analytics.

Numbers are your friend.

A few technological innovations--computing power, the internet, Excel, and R--have made numbers relatively easy for any layman to gather and manipulate. And the nerds are using this knowledge to their advantage. But you know who doesn’t have a clue how to leverage advanced statistics, analytics or simulations? No surprise, the U.S. military and national security community. For instance:

Intelligence. How many terrorists do we kill in Pakistan? How accurate are we? Which intelligence methods work best? Sounds like a ready made example for Bayes Theorem. (And not just in the CIA, but in every intelligence organization.)

Human resources. Who gets promoted in the U.S. Army? And why? Sounds like a perfect candidate for regression analysis. (Not just for the Department of the Army, but for every Brigade in the Army.)

Combat. Does any form of regression work from the battalion on down? Most importantly, do senior leaders use data to make decisions, or ignore when it doesn’t fit their “gut feeling”? (Not just for Corps headquarters, but for every battalion S3 section.)

Spreadsheets. Can we teach all analysts how to use a spreadsheet? I mean, not just to sum numbers to but to run simulations, data tables, pivot-tables and the like. Again, 99% of the Army doesn’t know the difference between an “if” formula and “vlookup”. Excel, not PowerPoint, should be the most popular program in the military.

The U.S. Army (and probably the rest of the military) needs data. Not even so-called “big data”; the U.S. Army barely has a grasp on regular old “small” data. I say this because I know how much I didn’t know or use statistics/analytics, and I worked in military intelligence, the most data rich field. And it wasn’t that I was lazy or ignored statistics, I was actually ahead of the curve on Excel...and it stuns me how much I didn’t know. I literally couldn’t regress a single variable, and I was swamped in activity data. (Here are some of my recommendations for management books to read to improve data analysis from my recent Thomas Ricks guest post.)

A lot of military leaders have MBAs, so that means they took at least one statistics course. And a lot of officers took statistics in their academies or for their majors in college (I assume). But that doesn’t mean the military has a data culture. Unlike the financial industry, operations research, consultancies, or even parts of the sports world, data doesn’t drive the military. And guess what? It’s not going to drive it in the future, unless officers start rigorously using data to make decisions.

In his controversial op-ed on military leadership in the Washington Post, Bruce Fleming summed up the data problem in one line, “Rather than prioritizing decisions based on justifiable evidence, we’ve been training our high-potential officers to believe their internal compass is king.”

So consider this a bailiwick of mine for the near future. My goal is to convince whatever officers I can that they need to make their organizations data cultures.

Expect more posts in the future.

four comments

First off, l love what you do; creative and thought provoking. Anyway on to commenting; I’m sure you’ve seen “Fog of War.” McNamara was a data guy and good one right? I know he was politically leashed but—come on—the Missle Crisis, Viet Nam, the decimation of Japan. He had all that data and used it like a machine. Maybe “they” have a point about that internal compass. Our have I misinterpreted your point? Anyhoo keep this stuff coming guys!


Carl, that is an incredible point (from an incredible documentary as well). But data advocates like myself (roughly in the vein of Nate Silver) don’t think data will solve all problems. Specifically, using data for immoral purposes is always…immoral.

And some data/analytics will be wrong. The most common example is the Rand study of fire houses in NY in the 70s which made the fire situation dramatically worse instead of better.

Instead, I am saying find pieces of “common wisdom” or accepted “conventional wisdom” and test whether they contribute to success. The pinnacle of this is Moneyball. ERA was the be all-end all for success in baseball. Now slugging and OBP have superseded or added to it.

The Army needs its own Moneyball revolution, in short.


Less Excel, more SPSS. And before the technical aspects are executed, a foundation of theory needs to be established.


Less Excel, more SPSS. And before the technical aspects are executed, a foundation of theory needs to be established.

“Machines are like children. Like parents, we have to let them have the fun while we child-proof the environment (sanitize their inputs) and clean up after them (do whatever they are too clumsy to do and clean up any messes they create).” http://www.ribbonfarm.com/2013/07/10/you..