Odds Are ...

I spent a large part of my professional life in software testing. There was some hardware testing in there too. At various points I played the role of statistician, calculating odds: mean time to failure, earned value, binomial models, markov chains, coverage statistics, mutation testing, usage models. I even invented methods such as Orthogonal Problem Calculation (patent pending). During the final decade of my career my primary role was statistician, although the key role I played was simply a quality advocate and evangelist — and did I ever preach from the pulpit.

I suspect most people think the job of a software tester is to find the bugs before the users. It is true that finding bugs is a main goal of a tester and that the process of finding and fixing bugs becomes the myopic focus during the final stages of development before the “great event,” GENERAL AVAILABILITY! But testing is a lot more than just finding and fixing the bugs. That was when my job started.

My focus was always more on error prevention rather than error detection, and I combed through testing statistics looking for trends and methods of improvement as well as evidence that recent process changes and new tools were working as expected. You call that engeneering “best practices,” and my job was to find and promote the best practices to encourage efficient and effective, high quality software development. It was outward focused on the customer and inward focused on the overall process. I even worked to improve the quality of the testing itself.

Knowing how to perform distribution analysis and regressions as well as methods of presenting statistical results were my day-to-day tasks. I used my teaching skills frequently as I presented results and recommendations to the workers in the trenches as well as the captains, colonels, and even the generals and admirals back at corporate. (You know corporate headquarters; where the rubber meets the sky.)

Now I spend my time doing a deadly statistical regression. What are the odds of cancer reoccurrence? Cancer survivors count the years. That is obvious that you would treasure the days that are left to you on this terrestrial orb and to celebrate each additional one like a birthday that had the odds against it. I’m new to this counting, and I wonder what anniversary you keep track of. You don’t know when the cancer started. You know when it was discovered and, in my case, I know when my cancer was surgically removed. We are not quite to the first anniversary of that event, although it is close — just a month away.

Doctors are good statisticians. Mathematical analysis is a major part of their profession as researchers determine the efficacy of treatment methods. Methods of medical experimentation are very refined and most medical news we hear in the press is based on the results of some study or statistical survey. They even do studies of the studies, something called a combined study.

When I was first diagnosed, I dug deep into the cancer literature and found many, very technical and intense statistical expositions. One key was a classification of a cancer by its stage. At the time the doctors thought my cancer was stage one, which is where you want to be, even though it may be sort of like a “little bit” pregnant. It was after my surgery and the biopsy of the tissue removed that I got the bad news: my cancer had spread and I was elevated to stage three cancer. Now there is worse news, stage four, but it was not my goal to score the highest on this particular exam.

So, I went to the literature. Stages can be used to calculate your life expectancy. All the tables use stage as a parameter in calculating longevity. The math is based on stage to predict the probability of the number of years of no reoccurrence of cancer — doesn’t that have a positive ring: YEARS of no reoccurrence. That means both that the cancer has not returned, at least not yet; but it also means additional birthdays. That’s the secret to long life, you know, it has been proven in study after study that people that have the most birthdays live the longest. That’s my goal.

So here is what the tables and charts tell me. In my case, if there is no reoccurrence of cancer after three years, then only 70% of the stage three prostate cancer patients have a reoccurrence. If you can make five years without a reoccurrence, then the odds improve  to 50%. Now I’m not a gambler and don’t go to Vegas for my vacations, but I got to say that those are pretty long odds. Don't get too discouraged, even if the prostate cancer reoccurs there are additional treatments such as radiation and hormonal treatment. That guy that Scotland released from prison for the Lockerbie bombing lasted years beyond what the doctors had predicted. As they say, “it is hard to make predictions, especially about the future.”

Also skewing the stats is the fact that prostate cancer is a disease of old age, and plenty of victims die of other causes before the cancer can get them. Prostate cancer is typically a slow growing malignancy, so that further improves the odds, although in my case there are concerns I have a rare, fast growing version. Wouldn’t you know it, the statistician suffers from a black swan (look it up on Wiki — the theory, not the movie).

So I’m tracking the numbers and making the graphs. Two more anniversaries and my odds of no reoccurrence jump up to 30%. Those are slim odds, but I don’t spend a lot of time worrying about it. I think the best defense is to live, love, and laugh. No-one knows the future, and people beat the odds all the time. I often feel like the guy with no shoes, when all around me are people with no feet. I originally wrote about cancer in the context of four men that I knew — I was the fourth. Two of them had died shortly after the discovery of cancer and two of them — again I was one of those two — still survived. Well, my dear friend ‘F’ succumbed last month (and I sure miss him). So it is now down to me. That’s 25% for those calculating. I don’t suggest you graph it with years as the abscissa. It is not a happy graph. No, I prefer the “bathtub curve.” After all, it makes a “smiley face.”

I think the foggy and dreary day led me to these musings. Please don’t worry about me. The last thing in the world I am is depressed. Just the opposite. Life has been a continued shower of blessings on this humble traveler, and I will go to my maker with the same smile I wore when I walked into the conference room with a stack of statistics under my arms and a message about quality in my heart.

Originally written Sept. 6, 2011.