It's Not Just About Moore's Law

Almost a decade ago I was working in a group that maintained a hierarchical mass storage system responsible for managing data to and from a facility full of CRAYs and other supercomputers, as well as to the desktops of several hundred researchers. Concerned about the scalability of the system over time, I was pondering Moore's Law. Gordon Moore, co-founder of Intel, observed that the complexity of integrated circuits doubles with respect to minimum component cost every twenty-four months. This is typically re-stated, somewhat inaccurately, that processor speed doubles every two years, and is sometimes misquoted as every eighteen months.

It occurred to me that processor speed could not be the only thing changing. After poking around a bit in a variety of sources ranging from NASA to the National Science Foundation to industry trade magazines, I built the following table of geometric progressions that I have tried to keep updated over the years.

Microprocessor speed doubles every 2 years.
Memory density doubles every 1.5 years.
Bus speed doubles every 10 years.
Bus width doubles every 5 years.
Network connectivity doubles every year.
Network bandwidth increases by a factor of 10 every 10 years.
Secondary storage density increases by a factor of 10 every 10 years.
Minimum feature size halves (density doubles) every 7 years.
Die size halves (density doubles) every 5 years.
Transistors per die doubles every 2 years.
CPU cores per microprocessor chip double every 1.5 years.

I plugged some of these into a spreadsheet and generated the following logarithmic graph. Note that some of the curves are hard to see because they fall on top of one another. (Click on the graph to see a larger version.)



What we have here is a set of mostly disparate power curves that illustrate how the performance of major components change with respect to one another over time. And not much time, either. For example, in a decade, processor speed will increase by a factor of thirty, while bus speed will merely double, and network bandwidth will increase by an order of magnitude.

Admittedly, some of the numbers from my original research may have gotten a little shaky as manfacturers have had to resort to more and more arcane measures to maintain their rate of improvement. But if I can at least convince you that different technologies are on very different curves of performance improvement, then you are pretty much forced to concede that the balanced system architecture you design today might not cut the mustard just a few years down the road as components and surrounding infrastructure are upgraded. The design decisions that made sense at the time, and the software implementations that were based on those assumptions, may not seem as wise later.

In fact, depending on the half-life of the system that is based on these assumptions, its basic architecture may have to be revisited more than once during its lifetime.

Sources

Computerworld

EE Times

NASA

National Media Laboratory

NSF Engineering Research Centers

The Total Cost of Code Ownership

In his book Object-Oriented and Classical Software Engineering, Stephen Schach describes the relative proportion of cost for each of the phases of the life-cycle of a software product. He comes to conclusion that some may find startling: 67% of the cost of software is in its maintenance: changes made to the software after the project is deemed complete. Some organizations with experience in maintaining large code bases, and by large here I mean millions of lines of code, place this number closer to 70% or even higher.



In "Does OO really match the way we think?", Les Hatton breaks this 67% into three categories: half of code maintenance is corrective (fixing bugs), and the other half is either adaptive (modifying due to changing requirements) or perfective (improving working code). So even if by some miracle we write perfect code (and in these days of compressed schedules, it would indeed be a miracle), the maintenance portion of the life-cycle pie is only reduced by half.

In his paper "Software Aging", computer scientist David Parnas writes about software entropy: software systems become more and more expensive to modify over time due to the cumulative effect of changes. Then perhaps it is no surprise that the bulk of the cost of developing software is in making these changes. This is as much a limit to system scalability as processor speed or network bandwidth. Increases in efficiencies in tools and processes must be applied not just to new code development, but to long-term code maintenance as well.

This is an area where code refactoring – the ability to substantially improve the design of code, including improving its ability to be maintained, without altering its external behavior – will continue to play a major role. Likewise, this calls for more thought into designing new code to be easy to modify, since the effort spent in changing code after the fact is the bulk of the cost of its development.

In my experience, this issue is largely ignored among software developers except among the refactoring proponents. Most developers -- and truth be told their managers as well, if widely used development processes are any indication -- are happy if code passes unit testing and makes it into the code base anywhere near the delivery deadline. Long term maintenance is someone else's problem.

I find that this long term cost is seldom taken into account, and its omission arises in sometimes surprising ways. I once heard a presentation from a software development outsourcing company. It happened to be based in India, but I am sure that there are plenty of home-grown culprits too. The company described several cost estimation techniques used by their ISO 9001 certified process which was assessed at SEI CMM Level 5 and used Six Sigma methodology. None of the cost estimation techniques addressed the long term cost of code maintenance. Code maintenance after delivery of the product was billed on an hourly basis.

I almost leaped from my chair, not because I was angry, but to go found a software development company based on this very business model. The idea of low-balling the initial estimate then making a killing on the 67% of the software life-cycle cost pie was a compelling one. Only two things stopped me. First, I had already founded a software development company. And second, I had read a similar suggestion made by Dogbert in a recent Dilbert cartoon strip, so I knew that everyone else already had the same idea.

It is as if once we deliver a line of code to the base, we think that the investment in that code is over. In fact, the numbers tell us it has just begun. Every single line of code added to a code base contributes to an ever increasing total cost of code ownership.

Sources

Les Hatton, "Does OO really match the way we think?", IEEE Software, 15.3, May/June 1998

David Parnas, "Software Aging" , Proc. 16th International Conference on Software Engineering, IEEE, May 1994

Stephen Schach, Object-Oriented and Classical Software Engineering, McGraw-Hill, 2002

The Value Proposition of Large Development Organizations

There are huge potential markets in the developing nations, e.g. India, China, and most of the rest of Asia, as well as most of the former Eastern Bloc countries (which used to be part of the developed world, but so it goes). But for these markets, labor is cheaper than capital. It is easier to devote a dozen engineers to a job than to buy a solution, because hard currency is difficult to come by, but motivated and talented engineers are numerous. So: in these potential markets, the ratio of availability of highly skilled labor to capital is the inverse of what it is in the developed world.

Nearly ten years ago I recall sitting in a lab late one night happily debugging a problem with an embedded project, using what I estimated at the time to be maybe $500,000 worth of diagnostic equipment. I was surrounded by scopes and analyzers and such. I thought: "This is great! There is no way I could do this kind of work on my own."

Today, my current project requires a laptop and a broadband Internet connection. All the software tools I'm using -- except for Windows XP, because that's what I have on my laptop -- are open source.

This really alters the value proposition for large development organizations. They may still fulfill a crucial role, mostly in terms of project management and interfacing to the marketing department, but in terms of providing infrastructure in support of development, not so much. So: open source changes the value proposition of the development organization, and drastically lowers the cost of entry into product development.

If a corporation in the developing world wanted a technology Thing, say, a PBX, it would be a lot cheaper for them to use an open source Thing, like, say Asterisk, even if it meant it had to devote a bunch of technical folk to support it. An added benefit is that these technical folk learn this open source Thing, and can build an industry around this open source thing, because the cost of entry into an open source industry is so much less than it used to be for the traditional Thing. So: instead of being an emerging market replacing the saturated U.S. and E. C. markets, they become a competitor to the traditional supplier of the Thing.

If your product has become commoditized, it is time to find a new product. If the tools you use to create your product have become commoditized, then maybe it is also time to find a new product.

Sources

Jim Van Meggelen, Jared Smith, and Leif Madsen, Asterisk: The Future of Telephony, O'Reilly, 2005

United Nations, World Population Prospects - The 2004 Revision - Highlights, February 2005

Extrinsic Motivators and the Subjugation of Humanity

Many studies have shown that intrinsic motivators (pride of workmanship, desire to please, etc.) are far more powerful than extrinsic motivators (money, awards), and that extrinsic motivators tend to override the intrinsic ones. In a fit of clarity, or possibly cynicism, it occurred to me that a very clever company could boost the effect of intrinsic motivators to their benefit by making the extrinsic motivators appear unattainable; hence the hypothetical existence of extrinsic motivators would be a moot point in many people's minds.

One way to do this would be to institute a forced-ranking system in which ratings are continuously lowered seemingly regardless of what the employee did. As employees felt more and more unvalued, such a system would gradually cease to have any motivational value. The continual reduction in rating leading to eventual termination would appear inevitable. A person in such a system might then return to more powerful intrinsic motivators as they did their best because of their own internal goals, having correctly figured that trying to reach the carrot on the stick was impossible.

This brilliantly evil plan is not to be mistaken for a dysfunctional forced-ranking system in which the lower 10% are laid off, and no new hires are made. This results in ratings compression as the remaining staff are fit to the the same 20-70-10 distribution. This is not brilliantly evil; it shows a lack of the most basic understanding of mathematics, the same sort of stupidity as when a manager (and you know who you are) complains that half of the employees are below the median.

Which system you may be in is left as an exercise to the reader.

Sources

Robert D. Austin, Measuring and Managing Performance in Organizations, Dorset House, 1996

Jeffrey Pfeffer and Robert I. Sutton, Hard Facts, Dangerous Half-Truths, & Total Nonsense, Harvard Business School Press, 2006