In Defense of Misbehavior

Some years ago I read Robert Austin’s 1996 book Measuring and Managing Performance in Organizations. It’s not the kind of book I would have normally chosen to read at that point in my life. But I was on a tear reading books on software engineering methodology and people management, and I kept stumbling across references to it. Reading it changed my whole perspective on Life, the Universe, and Everything. The recent death of Enron chairman Kenneth Lay inspired me to try to organize my thoughts on Austin’s topic of measurement dysfunction.

Austin’s thesis can be summed up as follows:

• To be effective, incentive plans must be tied to objective measurements of employees’ performance against objectives.
  
• To incent employees to produce the optimal desired results and avoid unintended negative consequences, all possible aspects of their performance must be objectively measured.
  
• In any but the most trivial of tasks, such complete measurement is at worst impossible, or at best too expensive to be practical.

The implications of this should be terrifying to managers trying to steer the ship of industry, because it says that the helm is at best only loosely coupled to the rudder. Steering inputs may have little effect, the opposite effect, or no effect at all. The linkage may be so complicated as to appear non-deterministic. Or, perhaps worst of all, the lack of complete objective measures may lull the person at the helm into thinking everything is fine when in fact they are steering blindly in a field of icebergs.

In my experience, this seems unbelievable to some, obvious to others. What Austin did, though, was apply agency theory to demonstrate this mathematically. Agency theory is an application of game theory, the very same branch of mathematics that made Nobel laureates of folks like Robert Aumann, John Nash, and Thomas Schelling. It is the basis of much of modern contract law and employment practices. Common employment practices, such as overtime pay for hourly employees, is based on the math behind agency theory.

What is so compelling about Austin’s work is that while you can disagree with his premise when stated as a bold fact, it is a lot harder to argue with the math. Okay, so you don’t like his results; what part of the math don’t you agree with? That the less free time employees have, the more valuable is their remaining free time? That employees’ production is a mixture of results measurable by different metrics (for example, quality, functionality, time to market)? That there is some specific mix of results that is optimal? That some metrics are expensive or impossible to measure?

We have all heard of examples of measurement dysfunction, possibly under different terminology in other contexts. Incentive distortion, unintended consequences, perverse incentives, and moral hazard are just a few I have come across in reading articles on economics, law, management, and ethics. Measurement dysfunction is so fundamental that once you grasp the basic idea, you start seeing it in the news (or experiencing it first hand) everywhere.

Amazon.com’s call center agents were measured by the number of calls they processed per hour, inciting them to hang up on customers in the middle of conversations.

Gateway’s technical support agents shipped whole new computers to customers with relatively minor (but time consuming) problems in order to make their monthly bonuses.

The incentives for corporate executives like Kenneth Lay and Jeffrey Skilling to lie, cheat and steal were so great that they overcame the intrinsic motivators towards honesty and good behavior and the extrinsic disincentives like heavy fines and jail time. In fact, if you can make tens of millions of dollars deceiving your employees, your shareholders, and your government, mightn’t some jail time seem worth the risk? When you face the prospect of forty million dollars in the bank and a few years in jail, paying off the Aryan Brotherhood for protection and bribing a few prison guards suddenly seems doable (whether it really is or not). It may not be until such corporate officers face the prospect of getting letters from their families written on toilet paper from the community soup kitchen, describing how they were sleeping in cardboard boxes because federal agencies froze all their assets, will the disincentives towards crime appear adequate. The incentives for finding loopholes even in the 2002 Sarbanes-Oxley Act are great.

Software developers and the organizations that employ them seem particularly prone to measurement dysfunction. As Joel Spolsky has pointed out (and as has been my personal experience), if you incent developers to write bug-free code, they will go out of their way to cover up the bugs they can’t find, which will hence be shipped with the product. If you incent developers to fix bugs, they will inflate their metrics by introducing more bugs for them to find. If you measure programmer productivity on lines of code delivered, don’t expect any efforts at code reuse or code optimization to succeed; you’re not rewarding them for the lines of code they didn’t write. If you reward developers for customer support, subtle bugs will increasingly appear in production code so that developers can take heroic action. Nelson Repenning has written much on the topic of how rewarding fire fighting in organizations leads to more fires to fight.

I recall talking to the head of a large software development organization who was asking for a magic quality filter through which code could be run in order to add that objective measure to the incentive program. Folks, software developers solve problems and reverse engineer complex systems for a living. If they’re good at it, it is like their brains are hardwired to the task. And they love a challenge. I am completely confident that the developers I work with on a daily basis are perfectly capable of gaming any incentive system that their employer puts in place, without necessarily actually achieving any stated goal of the program. Plus, some aspects of software algorithm quality, such as it does not contain an infinite loop, are actually proveably impossible to detect in all cases (the so-called "halting problem" from my graduate school days).

Which, finally, brings me to the real points of this article.

First: any incentive program is bound to drive dysfunction into an organization. If you must have an incentive program, expect to spend large sums of money and much time tuning it to minimize the dysfunction. Don’t expect to even recognize that dysfunction is occurring. When I have worked in an organization that employed forced-distribution of ranking of employees, and have brought the topic of measurement dysfunction up to managers, every single one of them said “Yes, I understand that this is a risk, but so far it isn’t happening here.” Folks, of course it’s happening here. You have merely provided incentives for your employees to hide it from you. Or maybe you’re in denial. Either way, I see the dysfunction every single day as suboptimal results are delivered in order to improve objective, but partial, metrics.

Second: don’t blame your employees for responding to your incentive program. It is the incentive program that is at fault, not the employee. Upper management says all sorts of things that come under the heading of “motherhood and apple pie”. The only way an employee really knows what upper management truly values is via the incentive program. You may say “quality is job one”. But if you can’t measure quality (and you cannot measure all dimensions of it), but you continue to terminate employees that don’t make their dates, then it is clear to everyone that “quality is job seven, or maybe nine, and besides anything past job four isn’t really important”.

At the time of his dissertation, Austin, now a faculty member at the Harvard Business School, was an executive with the Ford Motor Company Europe. The insight his work gave me motivated me to go so far as to order his Ph.D. dissertation on which his very readable book is based. One wonders what insight from his personal experience managing both technology and people was Austin able to bring to his research.

Sources

Douglas Adams, Life, the Universe, and Everything, Del Rey, 2005

Robert Austin, Measuring and Managing Performance in Organizations, Dorset House, 1996

Robert Daniel Austin, Theories of measurement and dysfunction in organizations, (dissertation), Carnegie Mellon University, University Microfilm, #9522945, 1995

Robert Cenek, "Forced Ranking Forces Fear", Cenek Report, 2006

Robert Cenek, "Forced Ranking Forces Fear: An Update", Cenek Report, 2006

Tom DeMarco and Timothy Lister, Peopleware: Productive Projects and Teams, 2nd edition, Dorset House, 1999

W. Edwards Deming, Out of the Crisis, MIT Press, 2000, p. 23

Jena McGregor, "The Struggle to Measure Performance", BusinessWeek, January 9, 2006

Jena McGregor, "Forget Going With Your Gut", BusinessWeek, March 20, 2006

Jefffrey Pfeffer and Robert I. Sutton, Hard Facts, Dangerous Half-Truths, & Total Nonsense, Harvard Business School Press, 2006

Nelson Repenning et al., "Past the Tipping Point: The Persistence of Firefighting in Product Development", California Management Review, 43, 4:44-63, 2001

Nelson Repenning et al., "Nobody Ever Gets Credit For Fixing Defects that Didn't Happen: Creating and Sustaining Process Improvement", California Management Review, 43, 4:64-68, 2001

Joel Spolsky, "Measurement", Joel On Software, July 15, 2002

People Are Our Most Important Fungible Commodity

I had lunch a couple of weeks ago with the department head from the R&D arm of a large multinational corporation. In the course of conversation over our rice bowls I remarked that his company's employee retention strategy seemed to depend upon the high-technology job market not improving. We both came up with several examples of the company's upper management doing stuff that just downright pissed people off.

My favorite example that he brought up: the company encouraged their engineering staff to forego vacations in order to make dates. Then they instituted a policy that forbad carrying over vacation into the next year. Then they generously offered to pay fifty cents on the dollar for any unused vacation. Then, to add insult to injury, they remarked that this policy was illegal in California, so those lucky bastards would be reimbursed at full price. Brilliant!

Completely coincidentally, I was poking around in the HR Policy Manual of this same company just a few days later and I noticed a section entitled "Retention Policy". The section had a single line which basically said "under review". Geeze, I didn't realize until then that what I had said was literally true. I was just being my usual smart ass Chip Overclock-self at the time.

There are lots of obvious reasons why this is a Really Bad Idea. Like: if your business is developing high technology products, and if you have no manufacturing or you have outsourced it (as this company had), then most of your capital is intellectual capital. That is, the knowledge and experience inside your employees' heads. Most of this knowledge is domain specific stuff that you can't hire off the street, but can only acquire from your competitors or create anew, either at relatively high expense. And when I say "capital", I really mean it. The accounting trend appears to be to capitalize the cost of software development so that it appears in the financial report as an asset instead of an expense. Watching intellectual capital walk out the door should be like watching part of your factory burn down, from the point of view of your financials.

I had seen some talented people walk out the door of this company in the past year. Bad enough that four engineers that I knew personally had recently left, but just a few months before the company's chief product-line architect had walked out the door to work at their biggest competitor. His parting email said they had made him an offer he couldn't refuse. The rumor mill said he had felt marginalized and unappreciated by upper management, who had probably been too busy flying customers around in one of their corporate jets to realize that the guy with their entire product-line roadmap in his head was about to defect.

No one probably really knows the true cost of these defections, but for the engineers you can at least estimate the cost of replacing them. In the second edition of their book Peopleware, Tom DeMarco and Timothy Lister write about "assessing the investment in human capital". There is the cost of interviewing and selecting a candidate. There is the lost productivity of other employees as they valiantly try to temporarily cover the workload. There is the ramp-up cost of the new employee during the time they are relatively unproductive and they require a lot of mentoring and hand-holding. Figuring a six month ramp-up time, this 1999 book figured the cost at around $150,000. The ramp-up time may be worse; another manager at this same company estimated it at about two years for some product areas. Note that none of this covers the cost of having your architects and engineers working for your competitor. We're talking probably at least a few hundred thousand dollars. Makes cutting back on office supplies seem kind of pointless.

Let me says this as simply as I can: whether or not you have an employee retention strategy, for sure your competitors all have an employee acquisition strategy targeted at you.

In the high-technology job market, your competitors include a lot of companies outside of your market domain, most of whom you have probably never heard of, although some of them may be your biggest customers. Think outsourcing overseas is going to solve this? Sorry, your competitors for intellectual capital have already thought of that too, and are right this very instant hiring the best that Bangalore, Prague, and Dublin have to offer. Or they'll wait until you train them, then they'll hire them away from you. Right now, IBM is hiring all the best tech talent in India to the tune of six billion dollars over the next three years. So long, and thanks for all the fish!

Of course, if you really are banking on the job market not improving, then what you're really saying is that you hope the economy surrounding your pool of current and potential employees doesn't improve. If this is the same economy into which you sell your goods and services, this is probably not a growth strategy. Rule of thumb: success should not hinge on two mutually exclusive conditions. Just my opinion.

Relax. It could be worse.

Back in 1995 I was privileged to spent a month working, lecturing, and traveling around mainland China. I remember vividly the director of a laboratory in Beijing, funded by the PRC's equivalent of the National Science Foundation, showing me a bar chart that displayed the age distribution of the researchers at the lab. There was this huge hole in the chart. It slowly dawned on me that this was the impact of Mao's Cultural Revolution. For about a decade, the PRC's production of scientists fell to zero. (The director himself gave a group of us a tour of the farmland he once worked as part of his reeducation as a freshly minted Ph.D.) The director knew that eventually the most senior researchers would retire and there would be no next tier to take their place, to lead the younger Ph.D.s, to fill the mentoring and research needs of the lab.

Want to achieve a similar effect? Freeze hiring in R&D for several years while times are lean. Then wonder what upper management is going to do when all the fifty-and-older baby boomers burn out and there aren't enough senior people left to provide the leadership you need for the younger folk.

Of course, Mao would have told you, there's always farming.

Sources

HR Policy Manual of nameless multinational corporation whose products you may be using

Douglas Adams, The Hitchhikers Guide to the Galaxy, Random House, 1981

Tom DeMarco and Timothy Lister, Peopleware: Productive Projects and Teams, 2nd ed., Dorset House, 1999

Barry Karafin, Business Management for Technologists, seminar, 2005

Paul McDougall, "How 6 Billion IBM Dollars Helped Chase Apple Out Of India", InformationWeek, June 6, 2006

Jeffrey Pfeffer and Robert I. Sutton, Hard Facts, Dangerous Half-Truths, & Total Nonsense, Harvard Business School Press, 2006

The End of Civilization As We Know It

In 1974, I was an impressionable young man just starting college, seeking an undergraduate degree in the exciting new field of computer science. That was also the year that I picked up a copy of the Club of Rome's Limits to Growth, a book chock full of dire warnings of global catastrophe, based on then-current rates of resource consumption, estimates of available global resources, and some models of growth and our planet's capacity for environmental recycling.

Thirty years later I found myself a cranky old man thinking of leaving software development to the offshore outsourcing firms and getting a job at Starbucks. I was seriously disappointed that the predictions hadn't come true, and I wondered what I was going to do with the contents of that six-foot-high gun safe in the basement. The Club of Rome did not foresee the economic forces that lead to the discovery of new resource deposits, that would push technology to give us greater energy efficiency, and just flat out eliminate the need for much of our energy consumption. See, there is this new fangled thing called the Internet. Some folks think it is going to be big.

The Club of Rome and prognosticators of their ilk were not much better at predicting the future than my beloved science fiction authors from the 1940s through the 1950s who missed the boat on mobile phones, laptop computers, and the World Wide Web. Yes, some folks hit the mark here and there (Vannevar Bush comes to mind), but we still don't have our flying cars. Here's something else they all missed: the coming global catastrophe of declining population.

The United Nations predicts, based on national population censuses and current trends, that the global population, currently around 6.5 billion, will slow in growth, and peak at 9.1 billion around 2050. Probably to no one's surprise, all growth will be in the least developed countries, with the most rapid growth in developing countries. The populations of the fifty least developed countries will double, while the populations of the most developed nations will, for the most part, shrink (except for immigration).

The flattening of the growth curve has two reasons: decreasing fertility rates, and increasing mortality rates. This is no surprise to anyone who knows a shred of arithmetic, but what might raise a few eyebrows is where these trends are occurring. Fertility rates are dropping in the developed world even in traditionally Catholic nations like Spain and Italy. And mortality rates are rising in Eastern Europe. The U.N. credits this latter effect to the spread of HIV, although other studies have also suggested that the young men of the former Soviet Bloc are drinking themselves to death.

The populations of just eight countries are expected to account for half of the world's population between now and 2050: India, Pakistan, Nigeria, the Congo, Bangladesh, Uganda, the United States of America, Ethiopia, and China (listed in order of the size of their contributions to population growth).

These projections may be just as shaky as the Club of Rome's, and for similar reasons. And maybe this is just the direction of the current swing of the pendulum. But it is interesting to think of the consequences of these trends, should they come to pass.

• Reduction in the number of consumers in the developed countries, and a general slowing in the production of new consumers globally: probably good for the environment; probably bad for the economies of the developed world.
• Larger pools of skilled workers in countries like India, Pakistan, the United States, and China: probably good for high-technology firms; probably bad for skilled workers native to the developed countries.
• Larger pools of software engineers available to develop and support open source solutions: probably bad for vendors of closed/proprietary technology solutions.
• Growing influx of immigration from the developing countries to the developed countries providing cheap manual labor: probably good for producers of hard goods and labor services, probably bad for laborers native to the developed countries.
• A shift in national identity of the developed countries, due to immigration, and a change in global culture, depending on where population growth occurs: this suggests immigration and fertility may be used as weapons of ideology. (However, in at article in Wired, Stuart Luman documents a drop in fertility rates in Islamic nations.)
• An increase in the mean age of the population (which has been going on for some time in the developed world): a change in cultural emphasis and in national priorities to deal with an aging population.
• The United States continues to grow while Europe shrinks: the marginalization of the European Union as a world power.

Think even further ahead and consider what kind of world may exist if population levels continue to decline to below their current levels of today. This will happen much sooner in the developed nations (unless immigration really steps up) than in the developing world. Some countries are taking this very seriously. In an editorial in Newsweek, Robert Samuelson remarks that Russian president Vladimir Putin has proposed a baby bounty to remedy the situation.

Fortunately, Al Gore, Colin Campbell, and others have suggested that global catastrophe may still be within reach in my life time, thanks to global warming and oil depletion.

There's still hope.

Sources

Vannevar Bush, "As We May Think", Atlantic Monthly, July 1945

Donella H. Meadows, et al., The Limits to Growth: A Report for the Club of Rome's Project on the Predicament of Mankind, Pan Macmillan, 1974

"Population Growth Slows", The Futurist, 37.4, July-August 2003

Stuart Luman, "The Decline of Civilization", Wired, November 2004, p. 78

United Nations, "World Population to Grow from 6.5 Billion to 9.1 Billion by 2050", press release, February 24, 2005

United Nations, World Population Prospects - The 2004 Revision - Highlights, February 2005

Robert J. Samuelson, "The End of Motherhood" , Newsweek, May 29, 2006