A Rambling Rant by an Old Man

I have had a good deal of career success. I attribute that success to a lot of things. One of those things is being evidence-based. Don't guess. Test. Measure. Look and see. Ask. If you can avoid guessing, do so. Sure, there are times you have to make a decision with incomplete information. But a lot of the time you can either defer that decision until more information is available, or improve the information on which you base your decision.

Not all opinions have the same value. For example, I have worked with people who have shared in a Nobel prize for their work in climate physics. Their opinions on climate change carry a lot more weight than yours. When you offer an opinion, on climate change or any other topic, I am going to assign an information content score to what you say. The definition of information is data that reduces uncertainty. The score I assign to your opinion might be zero.

The older I get, the more I seek out the original research and data on which people base their opinions. Often they either heard something second hand, third hand, or N-hand -- this is the "Chinese Whispers" problem -- or maybe the original study was flawed or biased, or maybe they are just full of crap because they believe what they want to believe. Most of the postings on Facebook fall into this latter category.

I like to read stuff that reinforces my opinions. I have read articles suggesting that this is common. I also like to read stuff that challenges my assumptions. I have read articles suggesting that this is rare. But it's the stuff that challenges my assumptions -- like education is a universal good, or that better communication unambiguously improves life -- that I find the most useful, and, really, the most interesting.

I like to read fiction, and have done so for decades. But increasingly I find myself more drawn to reading non-fiction. I find reality to be a lot more interesting, and stranger, and more compelling, than most of the fiction I read. It's as if reality exists just to entertain me. If you don't think reality is a damned interesting thing, then you aren't paying attention.

I'm old. Really old. I'm at an age where I have to read the obituaries, not just to make sure I'm not listed, but because I've lost many close friends and valued colleagues to accidents, strokes, cancer, heart disease, suicide, and even murder. I have a strong sense of my own mortality, and that time is short. Here's the thing: it's short for all of us, regardless of age. So you'll have to excuse me if I just don't have time for your bullshit.

I like people that agree with me. I also like people that don't agree with me. And I really like people for whom, when they don't agree with me, there is at least an even chance that they are right and I am wrong.

I find that, generally speaking, people act in the way in which they are incentivized to act. So if people aren't acting the way you want them to act, you should examine their incentives. Often, you will be the one (mis-)incentivizing them.

It's becoming increasingly difficult to find a really good soft pretzel.

How do you get to Carnegie Hall?

So I told him the same thing I tell all of them.

It's taken me forty years to get where I am. Two college degrees. Long periods of not making much money but getting a lot of great experience. Mentoring by many generous people smarter than I am. A lot of hard work. Many late nights and more than a few weekends. Even a skipped vacation or two. Occasional good judgement. Some luck. It helps a lot that I love what I do so much that I do it even when I'm not getting paid for it. There could be a faster way to get here, but you're asking the wrong guy what it is.

This is never a popular answer.

(Thanks to my friend and colleague Doug Young for suggesting the title.)

Hard Power Off, Solid State Disks, and Flash Memory: The Story Continues

Two more articles have come to my attention on the reliability of flash-based mass storage devices, from two organizations that should have some experience with such devices under their collective belts: Facebook and Google.

From Carnegie Mellon University and Facebook, Inc.:

J. Meza, Q. Wu, S. Kumar, O. Mutlu, A Large-Scale Study of Flash Memory Failures in the Field, ACM SIGMETRICS '15, June 15-19, 2015, Portland, OR, USA

Snippet:

Sparse data layout across an SSD's physical address space (e.g., non-contiguously allocated data) leads to high SSD failure rates;  dense data layout (e.g.,  contiguous data) can also negatively impact reliability under certain conditions, likely due to adversarial access patterns.

From University of Toronto and Google, Inc:

B. Schroeder, R. Lagisetty, A. Merchant, Flash Reliability in Production: The Expected and Unexepected, USENIX FAST '16, February 22-25, 2016, Santa Clara, CA, USA

Snippet:

In summary, we find that the flash drives in our study experience significantly lower replacement rates (within their rated lifetime) than hard disk drives. On the downside, they experience significantly higher rates of uncorrectable errors than hard disk drives.

Nearly five years after writing the first of the articles cited below regarding my own experience with incorporating SSDs and other flash-based mass storage in products, I continue to find challenges with their application. Not that I don't use them myself; semiconductor-based mass storage has replaced magnetic media in virtually all of the more recent systems that I deal with on a regular basis. But I find that the design practices of both hardware and software architects still haven't caught up with the implications of their use.

References

C. Overclock, Data Remanence and Solid State Drives, 2011-06-20

C. Overclock, The Death of Hard Power Off, 2012-07-17

C. Overclock, Hard Power Off Is Dead But Not Buried, 2013-03-04