Saturday, April 4, 2015
Jorge Soto's cancer detection kit
Early detection would be beneficial only if the cancers detected are those that will either eventually kill us or cause us suffering. If it cannot distinguish between nonprogressive, gradually progressive and aggressive cancers, early detection can lead to harm via overtreatment. Secondly, no detection method is perfect. Unless the test has 100% sensitivity and specificity, it will produce false positives, and the less prevalent the disease the worse the test will fare.
Soto says @9:35 their microRNA test is "accurate" but accuracy can mean different things. Does he mean that their microRNA test will, for instance, correctly tell us 95 out of a 100 times that cancer is present when given blood samples from patients who we know have cancer? If so then he's talking about the test's sensitivity. I'm inclined to believe this is what he means by accuracy. He might also be defining accuracy in another statistical sense: as the ratio of correct results to the total number of people tested: accuracy = [true positives + true negatives] / population.
Either way, the accuracy figure would be bloated and deceiving, particularly if he's using "accuracy" in the second sense as the example computation below shows. I've taken the liberty of giving Soto's test a very high sensitivity and specificity. As can be seen accuracy increases even as the prevalence of the disease decreases! That's the way to mislead with statistics.
In contrast to statisticians, the "accuracy" that doctors and patients are interested in are the positive and negative predictive values, which are, respectively, the probability of having the disease if the test result comes out positive and the probability of not being sick if one tests negative. And because different cancers have different prevalences, the rarer the disease the more prone to false positives this and any test will be. And so in the example, if the type of cancer is such that one out of every 100 people has it we see that PPV = 49%, meaning if we test positive then we might as well just flip a coin to find out if we have cancer or not, because for every 1000 people who test positive 510 don't.