| ||What Your Doctor Is Saying || |
How To Understand When Your Doctor Talks Statistics
Last reviewed by Faculty of Harvard Medical School on January 13, 2011
By Robert H. Shmerling, M.D.
Beth Israel Deaconess Medical Center
"There are three kinds of lies: lies, damned lies and statistics." This quote, attributed to Benjamin Disraeli, summarizes the suspicion many people have about statistics. I've often heard it said, only half in jest, that statistics can be used to prove anything you want. While it may be true that in many situations, numbers can be "spun" or manipulated to serve a particular purpose, it is also true that we are comforted by the way something that can be measured and quantified somehow, it helps us accept an observation as "real" if it can be expressed in numbers.
This suspicion of and simultaneous reliance on statistics makes it important to understand the basics well, especially when it comes to what your doctor is telling you. While nearly all doctors are required to learn about statistics during their training, most patients are not, so here's your chance!
Statistics in Medicine
A "statistic" can mean different things to different people. According to my Merriam-Webster dictionary, a statistic is defined as "a quantity (such as the average of a group of measurements) that is computed from a sample." Your doctor is dealing with numbers all the time, and much of what he or she tells you is based on principles of statistics. For example, if your blood pressure is elevated, should it be treated? That decision depends on information derived from thousands of other people who have had similar blood pressure results and what happens with their health over time compared with others who have normal blood pressure or received treatment to lower it.
What does it mean if your blood test returns "abnormal"? While the answer depends on many factors (such as which test, why it was ordered, and how severe the abnormality), it is key to understand how "abnormal" is defined in the first place and that depends on statistical measures. What if your doctor discovers a completely unexpected test result? Or if you have a family member with heart disease, how does that affect your risk and does it mean you should take a medicine? Again, an appreciation for some statistical principles will go a long way toward understanding your doctor's answers to these questions.
Back to top
What Do You Mean, "Normal"?
You may think you know "normal" when you see it, but for hundreds of measurements and tests, normal is defined by the results found among healthy people. A common way is to consider normal those results that include 95% of healthy people. For example, a blood test that detects anemia (called the hematocrit) is normal in our hospital's lab if it falls between 37% and 45%. That's because if you measured hundreds or thousands of perfectly healthy people, 95% of them would fall in that range. But that also means that 5% of healthy people's test results fall outside of that range: 2.5% above it and 2.5% below it. Usually, the healthy folks who get abnormal results have values very close to the cutoff that is, they are nearly normal. Still, that's one reason why your doctor may say a test needs to be repeated or that a confirmatory test should be checked, but that the abnormality is probably nothing to worry about.
Back to top
Statistical Significance: Power in Numbers
Research studies have to deal with the issue of normal people having abnormal results as well. But, they also must deal with the possibility that within two groups of people, differences may be due to statistical chance (or random variation) rather than truly different effects of treatment. For example, if 1,000 people were divided into two groups, with each group receiving a different medication for blood pressure, differences in blood pressure could be because one drug is better than the other, but they could also be due to chance. Blood pressure varies day to day in all of us and maybe it happened to fall a bit more in one group than the other.
This also applies to side effects. Imagine you are taking a medication to lower your blood pressure, but soon after you start taking the medicine, one of your blood counts is found to be abnormally low. Before blaming the medication and abandoning the study, the researchers will want to figure out if you are just in that 5% of healthy people with an abnormal test or if it is more likely to be a side effect of the medication. Through a complicated formula analyzing data from a large group of subjects, a determination can be made that the observed results are likely to be due to the treatment, or that it's even more likely that a difference is related to chance alone.
Your doctor may use the term "statistically significant," a phrase common to medical journals that publish research studies or news articles about them, to describe the results of such an analysis. It implies that the results, whether good or bad, are more likely due to a treatment rather than just a matter of chance. In most situations, we cannot rely entirely on tests to provide certainty. Whether it's your test results or those of a large scientific study, "truth" often remains a matter of statistical probability.
It is critical in this sort of analysis that there be enough patients to make valid comparisons that is, the study must have enough "power" to properly calculate statistical significance. That's why your doctor may dismiss a study you hear about in the news as preliminary or not enough to change recommendations for you research is often reported in the early phases when only a few patients have been studied, and, because of these statistical principles, it is hazardous to rely on research with too little power to detect real health effects.
Back to top
It All Comes Down to Probability
If multiple tests are ordered, one or more is bound to be abnormal. That's because, as above, most are defined by 95% of the population, and, sooner or later, you are going to fall in the other 5%. If you order 20 tests, there's an excellent chance that at least one of them will be abnormal (because one in 20 is 5%, the approximate fraction of healthy people whose test results are abnormal). If 100 tests are run, five are likely to be abnormal!
Abnormal results in people who are healthy are called "false positives," and they are common in medicine. False-positive results can be the source of unnecessary worry, additional tests and, sometimes, inappropriate treatment. A false-positive result is one of the many reasons that "extra" testing (testing that is not really necessary) is discouraged. It's also why many people are surprised to see abnormal results in their own medical record that no one ever mentioned their doctors probably decided that it was so close to normal and unlikely to be important that it represented a false positive and should simply be repeated or ignored.
Of course, results that are markedly or repeatedly abnormal are more likely to be important and to deserve further attention that is, they are more likely to represent "true positives." For example, if you have a severe sore throat and fever, a positive (abnormal) strep test is unlikely to represent a false positive.
One way to predict the importance of abnormal results is to estimate the chances of the illness or condition before the test was ordered. The situation that led to the test being ordered in the first place can increase the chances that an abnormal result truly reflects disease. That is, the better the reason for the test or the higher the suspicion of a particular condition, the more likely that an abnormal result is not a false positive. The weaker the reason for the test (including tests ordered for no specific reason, called screening tests), the higher the chance that an abnormal result represents a false positive.
The level of suspicion that your doctor has for a particular condition before he or she orders a test is called the "pre-test probability." Most tests are particularly helpful when the pre-test probability is in the middle range, neither very high nor very low. That's because if your doctor is quite sure about the diagnosis whether surely present or surely absent the test result may not sway his or her opinion. In the mid-range, however, a reasonably accurate test can provide powerful information.
Back to top
Consider this analogy: If a metal detector is installed in an airport, it will screen every person whether there is a specific reason (or suspicion) or not; when the alarm goes off, the chances are excellent that it is a false positive. Fortunately, there are other methods to confirm this accurately and quickly for example, by repeating the test after removing one's watch or using a wand to identify the source of metal. Now consider another situation in which a man wearing a ski mask and dark glasses races through a courthouse lobby and sets off the metal detector; it could still be a false positive, but this time the security guards will have a higher suspicion that it represents a "true positive," an abnormal reading that is significant.
Similar principles apply in medical tests. The likelihood of a disease before a test is ordered helps interpret its results, and there are statistical measures that can describe these likelihoods.
Back to top
The Bottom Line
How your doctor uses statistics in his or her everyday practice may sound mysterious or intimidating, but knowing some of the basics can show these terms for what they really are tools of the trade. Although you are an individual and each person is different, it is helpful to know about information collected from hundreds or thousands of people like you in a similar situation. The most likely diagnosis, the meaning of a test result, the reason for an examination finding, even how much you should weigh are based on statistical measures that often become a routine part of what your doctor is saying.
Fortunately, statistical measures are not the only thing backing up what your doctor is saying: judgment, objectivity and compassion are things no statistic can provide.
Back to top
Robert H. Shmerling, M.D., is associate physician at Beth Israel Deaconess Medical Center and associate professor at Harvard Medical School. He has been a practicing rheumatologist for over 20 years at Beth Israel Deaconess Medical Center. He is an active teacher in the Internal Medicine Residency Program, serving as the Robinson Firm Chief. He is also a teacher in the Rheumatology Fellowship Program.