Short-term studies are not reliable measures of the long-term. Unless the disease or symptom is a short-term issue, the inadequate follow-up of many studies is a significant limitation and source of error.
Yet, there are major challenges to performing long-term research, including lack of personnel (who change jobs or graduate), difficulty keeping track of the volunteers (who move away or grow weary of the monitoring), or changes in technology. For example, it may be impossible to compare the results of an MRI today with results from 15 years ago because the quality of images obtained has improved over that span of time.
Back to top
Could You Repeat the Question?
Research begins by asking a question and then designing a way to answer it. But because of how research is performed, the question may change to be something related that is easier to study, but not exactly the ideal question. For example, there is a lot of focus on cholesterol, but it's cardiovascular disease and death that we really care about. Knowing that there's a connection between cholesterol and these other important outcomes has led researchers to focus on cholesterol results. So, instead of asking, "Does this medication prevent heart attack and prolong life?" we ask, "Does this medicine reduce cholesterol?"
Relying on research that uses "proxies" (a variable that "fills in" for the one you're really interested in) is not as reliable as those that directly look at the key endpoints of interest.
Animal studies are another way that the question is revised for the sake of study design. If you want to ask, "Does this medicine prevent age-related changes in the brain?" you can actually examine the brains of rats and mice in a way that you cannot for humans. So, the question becomes, "Does this medicine prevent age-related changes in the brains of mice?"
Finally, the question may be shaped based on who is paying for the research. While you may want to know, "Does this medicine work better than another medicine?" a pharmaceutical company may change it to "Is our medicine better than nothing?" Why? Because that's the minimum requirement of the U.S. Food and Drug Administration for approving a drug and allowing it to be marketed in this country.
That question may be much less interesting — do you really want to take a medicine that's "better than nothing" or the best one among all options for treating your problem?
Back to top
Medical research is vitally important, but it has some significant limitations, and some of them contribute to how the medical news you hear today may be contradicted by other research next week or next year. Here's my advice: Remain skeptical, recognize that it's rarely a good idea to bank on a single study, and keep in mind that there could always be error, no matter how good the research (or the researcher) sounds.
Someday, there may be major changes in our system of funding and executing medical research. We need funding sources that do not alter the question that is asked, we need better ways to minimize error, and we need research designs with a long-term vision. Increasingly, regulators (such as the FDA) are asking pharmaceutical companies to study new drugs even after their approval. This is a promising development. Still, medical research will never eliminate all sources of error. So, it's best to take your medical news with a grain of salt — or perhaps a couple. It's likely that your doctor already does.
Back to top
Robert H. Shmerling, M.D., is associate physician at Beth Israel Deaconess Medical Center and associate professor at Harvard Medical School. He has been a practicing rheumatologist for over 20 years at Beth Israel Deaconess Medical Center. He is an active teacher in the Internal Medicine Residency Program, serving as the Robinson Firm Chief. He is also a teacher in the Rheumatology Fellowship Program.