Product Development as Warfighting

Pop quiz. Closed book. Multiple choice. Complete the following.

John Boyd (1927-1997) was:

A. in the U.S. Army Air Corps and later U.S. Air Force and served in three wars.

B. the chief instructor at the Fighter Weapons School at Nellis Air Force Base in Nevada. He was known as Forty-Second Boyd for his boast that he could win any dogfight in forty-seconds. He was undefeated.

C. the author of Aerial Attack Study, the first text book on air combat. The book, in either its original or pirated form, became the principle text book on air combat for every air force in the world.

D. the inventor of Energy-Maneuverability Theory, which applied thermodynamics to aircraft performance. E-M Theory became an invaluable evidence-based tool in the design, specification, evaluation, and comparison of aircraft.

E. a key player in the design of the F-15, F-16, and A-1 aircraft, and was a fierce opponent of the initial flawed designs of the M1-A1 Abrams tank and the Bradley Fighting Vehicle.

F. the thought leader of the Defense Reform Movement at the Pentagon, applying an evidence-based approach to military procurement. He required for the first time in decades that competing defense contractors build working prototypes of weapons systems so that they could be directly compared.

G. a retired Air Force officer who synthesized centuries of strategic thinking to codify Maneuver Warfare. He lectured at Quantico and his work was eventually adopted by the U.S. Marine Corps as official doctrine. A statue of Boyd, in his Air Force flight suit, can be found at the Marine Corps Research Center at Quantico.

Of course it's a trick question. Would you expect less from me? If you or I had achieved any one of these accomplishments, we would have considered it the capstone of a brilliant career. Boyd achieved all of them.

That's not all. He desegregated the Las Vegas casinos in the late 1950s by insisting that his black officers be allowed to dine with the rest of his men in the restaurants. And for a time he commanded what was probably the most secret base in South East Asia, responsible for monitoring the Ho Chi Minh trail.

But his achievements did not come without a price. He barely spoke to his wife, although he apparently spent enough time with her to have five children, whom he also ignored. His career stalled at the rank of full colonel because he had become a thorn in the side of too many senior officers in all branches of service. In retirement, he refused to double dip, and his pension was just enough to house his large family in a tiny basement apartment in a bad part of town. If you saw him on the street, you might conclude from the way he was dressed and acted that he was among the ranks of the homeless instead of one of the greatest strategic thinkers of all time.

And although nothing I have read has ever suggested it, all of the descriptions of Boyd's behavior and mannerisms point to the symptoms of Asperger Syndrome. It's like checking off bullet items on a checklist. Economist Tyler Cowen has written that for some professions, Asperger's or related traits on the autism spectrum are not necessarily dysfunctions but can actually be a competitive advantage. Those with Asperger's frequently are able to bring an extreme mental focus, an obsessive attention to detail, and other cognitive abilities to bear on highly complex problems. What they may lack in the social skills considered to be naturally occurring by others can be learned.

One thing for sure, if Boyd did have Aspergers, he was probably in good company. It has been speculated that luminaries ranging from Thomas Jefferson to Albert Einstein to Alan Turing also lived on the autism spectrum.

There are several recurring themes in Boyd's work, briefings, and teachings.

People. Ideas. Things.

It took me decades to figure this out on my own. In any endeavour, people are more important than ideas. Ideas are more important than things. Things (or as Boyd put it, hardware) are the least important of all.

Boyd, who felt that the Air Force was too much a technocracy enamored with the latest new hardware, preached this time and time again to anyone that would listen. It is people who make the difference. It is the idea that can be more broadly applied. The hardware is just a passing fad.

This is the mistake that many technologists make in their personal lives. When given the choice between going out with your friends and sitting at home alone cruising the web, generally the best long term choice is to go with your friends. If trying to decide between spending money on travel with your spouse or buying a new home computer, take the trip. Learning a new concept is more important than learning how to use the latest gadget.

It's also the mistake made by many hiring managers, architects, and technical leads. A really good developer can more than make up for marginal or legacy technology. A developer who is good at synthesizing and applying new ideas is better than a developer that has the latest new technology on their resume.

This is also why design patterns are more important than specific tools, because the pattern is more broadly applicable. A developer who can intuitively apply design patterns is even better.

To be or to do.

Boyd saw that a lot of the officers at the Pentagon were careerists, more interested in getting promoted than doing the right thing. He also realized that in a bureaucracy it was frequently impossible to do both. Advancement in a bureaucracy typically depends on defending the status quo, not on shaking things up. Boyd lectured his followers on the need to decide whether they were going to be something or to do something.

George Benard Shaw once said "The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." The fact that Boyd and many of his followers were unreasonable men is why most of them saw their careers stall in mid-flight: they choose to accomplish something great rather than to become something great.

I should be the last one to give you career advice. But I will tell you that if you have any ambition at all, you will face this choice, probably many times, during your career in high-technology. I personally have always chosen to do rather than to be. But then again, I am living proof that you can spend a decade in an organization without getting promoted or accomplishing anything significant. If you find yourself in that situation, then both of your engines have flamed out and your plane is on fire. Consider bailing out.

Fast Transients.

It may seem strange, particularly to those who have served in the military, that a self-described "dumb fighter jock" would have revolutionized both air combat and ground warfare. But Boyd found many similarities. All of his work revealed the importance of fast transients: the ability, and the willingness, to change course quickly. It was this critical aspect of his concept of air combat that directly led to the design of the small, nimble, and very successful, F-16 fighter, when all Air Force doctrine at the time called for higher, further, and faster in a straight line. It also was the main idea behind maneuver warfare: move quickly, probe for weaknesses, don't get bogged down, don't worry about your flanks, let the enemy worry about his flanks. This can be applied to business, to development processes, and to architecture and design.

If your adversary is on your tail, not changing anything will not improve your situation. Changing too slowly will just give him time to adapt. You have to commit to change, to changing quickly, and to do so in a way that your adversary cannot anticipate. You must exploit what Sun Tzu referred to as cheng and ch'i, the expected and the unexpected. This is the core of what is innovation, doing the unexpected (and is what Apple in my opinion is very good at).

Be prepared to toss out a business plan or a technical design when new evidence comes to light that makes the current path unworkable. For sure, fix it if it can be fixed. But if it can't, don't spent endless meetings debating or fretting about it or moaning about the compromise of your beautiful aesthetic design. You're a craftsman, not an artist. Take the control stick in your hands and change course.

If during development you become blocked on a problem, don't sit around waiting for things to change. Find an area to work where you are not blocked. Make progress where you can. You will accomplish something while waiting for the blockage to resolve itself, and you may find that the progress that you made actually led to its resolution. This is the essence of blitzkrieg.

When designing complex real-time systems, expect emergent behavior. Be prepared to deal with it. Expect failures to cascade quickly through the layers of your architecture. Don't get so overwhelmed with a fire hose of log messages that you can no longer tell what is going on. More is not necessarily better. Eliminate detail. Reduce the logging level until the chaff and smoke and clutter is reduced so that you can clearly see your target. Achieve fingerspitzengefuhl, or finger-tip feel, an intuitive mental model of how your systems works and reacts.

Keep the end goal in sight. It is easy for technologists to become side tracked with the cool details of the latest new thing, to become tied up in dealing with a stack of non-critical bug reports, or to continue to refine and refactor that working code until it meets some arbitrary personal design aesthetic. Don't get bogged down in the thousands of tiny technical details so that you can no longer see the big picture. Make all decisions based on what will help you achieve your end goal, which is always shipping a product. Organizational guru Steven Covey says "begin with the end in mind", but among warfighters this is what is known as schwerpunkt: maintaining a focus on the objective.

Observe, Orient, Decide, Act.

Boyd's most famous idea (and probably the most misunderstood, including by me) is the OODA Loop. Boyd described how people process information to deal with the outside world. In his briefings he applied it to conflict, but it's much more broadly applicable than that. You observe what is happening. You orient, by which he meant you synthesize and interpret what is happening based on your own experience and cognitive biases. You decide on a course of action. You act on that decision. Because your actions change the circumstances in which you find yourself, you iterate and do it all over again.

When in a conflict with an adversary, both parties are iterating in their own OODA Loops. The key to success is to be able to iterate faster than your adversary. Boyd called this getting inside your adversary's decision loop. When applied on the battlefield, Marine Corps commanders remarked that it was like you were running both sides of the battle. The enemy spends all their cognitive bandwidth trying to figure out what the heck you are doing and reacting to old news.

Of all of Boyd's ideas, this is the one that has gotten the most press, mostly in the business world, for obvious reasons. It is the idea of fast transients writ large, taken from the realm of air combat and applied to conflict in general. Success in competition relies on not just making the correct decision based on the evidence at hand, but making that decision faster than the other guy.

What makes Boyd's OODA Loops go beyond just fast transients is the fact that you can negatively impact your adversary by controlling how and what he observes, and by exploiting his cognitive biases in how he interprets it. This is the role of most security and intelligence organizations (including business security and intelligence), but is crucial even among field commanders. You can force your adversary into making decisions based on the wrong information. Because you can do this to your adversary, you must assume that he is doing it to you.

Sometimes your adversary is an architecture, a design, or a legacy code base. You have to be aware of what your cognitive biases are, and whether they are blinding you to what can be done, or what needs to be done. This is one of the reasons I believe developers should learn new programming languages, particularly languages that are very different from what they use everyday, even if they never intend to use those different languages in production.

For example, if you spend most of your time using procedural languages like C, you should learn object oriented languages like C++. If you use an unmanaged language like C++, you should learn Java. If you are a C, C++ and Java expert, you should learn Prolog, Snobol, Lisp, Haskell, Smalltalk, or any number of other non-traditional languages, just to learn a radically different mindset and approach to solving problems. You may find idioms and patterns in those languages that you can productively apply to your everyday work.

Since ideas are more important than things, learning concepts like finite state machines, push down automata, and formal grammars, which can be applied in any language, are really useful too. I had several courses in graduate school on automata and formal languages that were ostensibly theory. Yet they completely changed, for the better, the way I view many programming issues I encounter in my daily work.

A point about OODA Loops that is frequently ignored, but is clear from Boyd's briefing slides, is that this iterative process is fractal: there are loops within loops at all different levels of scale. This fractally iterative process describes how I see the development process in general: you constantly iterate at all levels of scale: implementation, design, architecture, and requirements, with feedback from all levels constantly changing other levels as you decide what can and cannot be done, what must be done and what is optional, given the technology, resources, and schedule. This is really what Agile Development at its core is all about: fast yet effective iteration.

John Boyd is my hero.

I find the story of John Boyd to not only be inspirational, but also a cautionary tale regarding work-life balance and the fine line between genius and dysfunction. Because Boyd chose to deliver his message primarily through classified briefings, and seldom if ever published any of his ideas, he is largely unknown today. When his work is used, he typically goes uncredited. This is even though his ideas revolutionized much thinking in the military, in business, and for me, in high-technology development.

John Boyd died of cancer in 1997. He is buried in Arlington National Cemetery.

Sources

Robert Coram, Boyd: The Fighter Pilot Who Changed the Art of War, Little, Brown and Company, 2002 (This nearly 500 page book is surely the definitive biography of John Boyd. It is also one of the best non-fiction books I have ever read. Coram makes Boyd's story inspirational, compelling, and cautionary.)

James Fallows, National Defense, Random House, 1981 (Fallows frequently interviewed and wrote about Boyd and national security issues for the Atlantic while Boyd was at the Pentagon. This National Book Award winner covers a lot more territory than just John Boyd. Fallows' coverage leans neither left nor right, and is still worth a read today, although his remarks on nuclear policy makes me nostalgic for the Cold War when nuclear weapons were in the hands of professionals instead of radicals, thugs, and lunatics.)

Grant T. Hammond, The Mind of War: John Boyd and National Security, Smithsonian Institution Press, 2001 (This book was researched and largely written while Boyd was still alive. Coram states that Boyd prohibited Hammond from including much in the way of personal detail. It is an excellent introduction to Boyd's ideas, but Coram's book gives a picture of Boyd the man.)

Frans P. B. Osinga, Science, Strategy and War: The Strategic Theory of John Boyd, Routledge, 2007

U. S. Marine Corps Staff, Warfighting, MCDP 1, 1997