Sunday, April 24, 2011

Sturgeon's Law and Context Sensitivity

Hi. My name is Chip, and I've been a manager. More than once. It's not something I'm proud of, nor did I find those experiences personally fulfilling, and only rarely enjoyable. Some misguided folks seemed to think I was good at it. It was an opinion I myself didn't share.

But I tried to be the best manager I could be, and part of that effort was spent reading books that made the bestseller lists in the various business magazines. Over the years I came to the conclusion that, like pretty much everything else, business books obey Sturgeon's Law:"Ninety percent of everything is crap" [1].

An article by Matthew Stewart, "The Management Myth", published in 2006 in the magazine The Atlantic (and also as a book reviewed in the Wall Street Journal), speaks to this. In it, Stewart relates how he spent seven years as a management consultant, ultimately co-founding a management consulting firm that employed six hundred people. Stewart did not have an M.B.A. He didn't have a business degree at all. He had a Ph.D. in nineteenth-century German philosophy. So, was he a natural born genius in business management? Not so much. To hear him tell the tale, he pretty much faked it. He doesn't make it sound that hard. He also runs to ground many of the classic epic tales in management, many of which I'd read or been told about during my own management training, revealing many to have actually been epic fails.

Frederick Winslow Taylor is credited with creating management as a profession by writing the book The Principles of Scientific Management based on his experience with figuring out in 1899 how to get laborers to load train cars faster: he bribed them. His book however described a much more complicated theory of management, which some called "Taylorism", which could only be accomplished by someone with an advanced education, an expensive suit, and for sure a higher salary. Unfortunately, it was a theory whose application could not be replicated by others. Bethlehem Steel Company eventually wised up, fired him, and scrapped his new fangled management techniques.

Another anecdote, and one that I had also heard about before, involved Homer Hibarger, who in the early twentieth century was doing research into how to increase productivity of workers by increasing workplace illumination. Since this research was being funded by a manufacturer of electric light bulbs, you may assume the results were a forgone conclusion, for otherwise they would have never, ah, seen the light of day. Indeed, productivity among a group of woman on a assembly line at a Western Electric plant increased when the illumination of the work area was increased. It also improved when illumination was decreased. In fact, productivity increased when almost any change was made in the work environment. What change resulted in the greatest improvement? Apparently, that occurred when some of the women on the line were replaced with others who were easier to get along with. And the fact that the women earned extra compensation for participating in the experiment.

Stewart isn't shy about relating his own experiences as a management consultant, watching his fellow consultants massage raw data to fit the curve that they needed, altering conclusions to match those required by their client, or reiterating the latest fad from a recent business bestseller whether it was applicable to the situation or not. Eventually his disillusionment led him to leave his consulting firm, only to watch it self-destruct in the receding distance under the squabbles of the remaining partners.

I had a similar period of disillusionment as a manager. I was a fan the 1982 bestseller In Search of Excellence by Tom Peters and Robert Waterman. It had what appeared to be hard data on how companies succeeded in their industries through both innovation and good management. That list of excellent companies included 3M, Walt Disney, and Intel. And also Amdahl, Data General, and Wang Laboratories. As I read other books by Tom Peters, I began to see less and less hard data from Peters backing up whatever management fad he was pushing. And about those excellent companies: many of them later fell on hard times, and some of them ceased to exist all together. Some, like NCR and Xerox, didn't even produce great financial results at the time the book was published. Peters was later accused of fudging the data for the book, a charge he denies.

But I'm reluctant to be too hard on Peters, or for that matter on Hibarger or even Taylor. Because although I'm a lousy manager, I have a pretty good track record as a software developer. And one thing I've learned over the past three-plus decades from studying, applying, and being painfully subjected to the latest software development process fad: the application of all processes and techniques, regardless of the problem domain, is very sensitive to context.

Extreme Programming was widely publicized before it had ever had its first big success. It was first applied to a project at Chrysler which never came to fruition; the project was cancelled when Chrysler was acquired by Daimler-Benz. Large development organizations (like the kind I have found myself in much of my career) have found the principles of Agile Development difficult to apply to non-co-located staff or to mission critical applications (ditto). The U.S. government (as well as many large development organizations) have found the classic Waterfall Process to be way too heavy-weight and inefficient to work at the rapidly changing pace of software development, although it works for things like building nuclear submarines.

That doesn't mean that Peter's eight themes of excellent companies, Boehm's tenets of Extreme Programming, the principles of the Agile Manifesto, or even the hoary old Waterfall Process, are fundamentally wrong. On the contrary, all of those have nuggets of great truths in them. It just means any particular technique or process can't be all things to all people all the time. Expecting any of them to work in all circumstances is just plain naive. And that includes whatever article on the latest thing in business management or software development that you are reading today. You might even apply Sturgeon's Law to mean that any of these ideas will be applicable at most 10% of the time.

Besides, it is in dealing with the exceptions where managers, and developers, earn their pay. The exceptions are what require innovation, insight, experience, and improvisation. As much as upper management would like to have a recipe for guaranteed success, sometimes you just have to make it up as you go long. And to come back to Stewart's article in The Atlantic, it's almost always more about the people you have than the process you use.

If it were easy, anybody could do it.

(Many thanks to my old friend and occasional colleague Jacque Marshall for passing the Stewart article along.)

[1] I was fortunate many years ago to hear the late Theodore Sturgeon relate this anecdote in person: at a dinner party a woman asked what he did for a living. "I'm a science fiction writer," he replied. "Oh dear," she said, "isn't ninety percent of it crap?" "Yes", he explained, "but ninety percent of everything is crap."

No comments: