Bottom Line: Even researchers don’t understand the mistake they often make
When it comes to health studies, the bigger the sample of people the better, right? That’s what medical researchers have long believed—and it’s big studies that seem most convincing to laypeople, too. Yet a recent Unversity of California, Berkeley, analysis found that applying “big-data” results could be causing more harm than good. That’s because in real life, typical “group” results—the holy grail of most modern research—end up applying to far fewer individuals than anyone expected.
In fact, one reason doctors may swing and miss so often with so many treatments is that the big-data research model provides deceptive information, says Aaron J. Fisher, PhD, assistant professor of psychology at UC Berkeley and lead author of the analysis. Fisher looked at studies of mental health treatments, but he thinks that what he found would hold true for many or most types of health studies—when you rely on the statistics of a large group, you sacrifice anything you could have learned about the individuals in that group, and the result is poor health advice for many people.
Fisher and his research team conducted six separate behavioral studies on hundreds of individuals including healthy people as well as those with generalized anxiety disorder (GAD), depression, post-traumatic stress disorder (PTSD) and panic disorder. They utilized data from studies that had asked participants to take online and smartphone self-reporting surveys, pinging them for information four to 10 times per day, depending on the study.
They confirmed that, in most cases, what was true for the group often wasn’t even close to what was true for the individual. While there has been the acceptance in the medical community that one form of treatment may not help everyone but should help most people, this study contends that this is a huge overestimation—considering how widely symptoms varied from one person to the next, the number of individuals unlikely to benefit from generalized findings is far greater than assumed.
For example, exposure therapy is the gold standard for getting over a phobia, the concept being that if you can be comfortable around the object or situation you’re afraid of and have been avoiding, you can conquer your fear. But one of Dr. Fisher’s studies showed that avoidance wasn’t tied to fear for some people, and because of this, those people won’t be helped by exposure therapy.
Should we lose faith in health studies, whether psychological or physiological? Fisher says big-data studies still are important, but it’s the research on the individual that can lead to precision and personalized medicine in which a person’s genes, health history and lifestyle all factor into the appropriate prevention, diagnosis and treatment for him/her. For example, just a few years ago, any cancerous growth that could be surgically removed would have been surgically removed, says Dr. Fisher. Now personalized medicine can help doctors figure out whether a tumor will threaten a person’s life and needs to be removed…or if it can be left alone to avoid treatment complications with little upside.
Bottom line: Be very skeptical that treatment findings from any given health study will actually apply to you. They might, but now we understand that there’s a bigger chance than we used to think that they won’t.
To learn more about the advantages of personalized medicine, read Bottom Line’s “Personalized Genetic Medicine” and “Personalize Your Heart Disease Prevention Plan.”
