The Trouble with Symptom Checkers

Date: December 15, 2016      Publication: Bottom Line health      Source:  Ateev Mehrotra, MD, MPH, Harvard Medical School      Print:

Computers are amazing. They can beat world-class chess champions. But they can’t yet replace your doctor. Case in point—online symptom checkers.

The checkers, free at sites such as WebMD, the Mayo Clinic and the American Academy of Physicians, and through free smartphone apps such as AskMD and iTriage, are powered by pretty sophisticated medical algorithms.

If you’re like millions of others, you may have turned to one of these to figure out what’s triggering your chronic cough or repeated headaches, your rash, your insomnia or unexplained weight gain or loss. These programs are easy to use—you just plug in one or more symptoms, the program asks you a few questions—and then it suggests what condition or conditions you might have. The checkers make recommendations, too, such as what to do to feel better and whether to see your doctor—or head immediately to the ER. While symptom checkers may not explicitly promise to give you a diagnosis, millions of people turn to them to try to figure out what’s causing their symptoms.


Here’s our recommendation: Don’t rely on these to help you figure out what’s wrong with you. While they may have some educational value, the latest research finds that not only are they unlikely to give you the most probable cause of your symptoms, but they’re also prone to putting people in a panic and sending them for urgent medical care when it isn’t needed. They do have a place, however—if you know the smart way to use them.


In the first study to compare the diagnostic accuracy of these online tools with the accuracy of live MDs, researchers at Harvard Medical School asked 234 doctors trained in internal medicine, family practice or pediatrics to evaluate 45 “clinical vignettes”—symptoms and medical histories of hypothetical patients—and give the most likely diagnosis for each patient plus two additional possibilities. The conditions ranged from common to uncommon, minor to life-threatening, from canker sores to pulmonary embolism (a very dangerous blood clot in the lung).

Results: The white coats won by a mile. Computerized symptom checkers identified the most likely correct diagnosis only 34% of the time…doctors, 72%. Counting all three suggested diagnoses, checkers included a correct diagnosis only 51% of the time—but docs included a correct diagnosis 84% of the time. (In case you were wondering, no particular symptom checker was better than the others.)

Would you go to a doctor if you knew that he/she made correct diagnoses only one-third to one-half of the time? Didn’t think so!

More findings from the study…


  • For the mildest diseases, the automated symptom checkers did a little better, nailing 40% of the cases on the first try. But docs nailed 65%.
  • For the most common diseases, checkers got 39% versus 70% for doctors.
  • For acute illnesses—the kind that really need immediate attention—checkers guessed right only 24% of the time…doctors, 79%. That’s a huge difference and could make a big impact on a patient’s life.
  • For the rarest diseases, checkers got it right only 28% of the time versus 76% for doctors.


Clearly, the computers are missing some of the subtleties of clinical diagnosis. Current symptom checkers, both on websites and apps, ask you for symptoms, then ask for related symptoms, in a step-wise process to lead you on a logical path toward possible causes. But in practice, it’s not very accurate. One key reason is that they rely on a rigid “branching logic,” explains Ateev Mehrotra, MD, the lead investigator of the study and an associate professor of health-care policy and medicine at Harvard Medical School. “Right now, their databases aren’t big enough. Better diagnoses come from a more complex pattern recognition that is not yet part of the symptom checkers out there.” Doctors, on the other hand, look through a wider lens, he explains. They consider not just age and gender and symptoms, but also the most common ways that symptoms tend to show up in patients and what they observe in any particular patient—and then they filter that through their clinical experience.

Say you’re experiencing pain and stiffness in your joints, perhaps some swelling and tenderness in your hands and wrists, maybe even your feet, and your joints feel warm to the touch—the classic symptoms of rheumatoid arthritis. You click on the iTriage symptom checker and find “joint pain” and “joint stiffness” on its preset list of symptoms and choose joint pain, as iTriage allows only one choice at a time. Once you make one choice, it asks you questions and gives you more choices. When you’re done, up pops a ranked list of potential causes—bursitis (a common, nonspecific inflammation) is first, and rheumatoid arthritis is ninth, right under “pseudogout.” Had you chosen joint stiffness, rheumatoid arthritis would have moved up to number seven. The app then describes the many tests your doctor may order to diagnose you.

In other cases, a symptom checker may send you directly to urgent care or even the ER. In a 2015 audit study of symptom checkers, Dr. Mehrotra’s team found that in two-thirds of cases where self-care was all that was needed, symptom checkers recommended medical attention. “This was one of our main criticisms,” said Dr. Mehrotra. “I worry that symptom checkers that almost always recommend health care might add confusion and possibly drive more people to get care when it might not be necessary.”

Another risk: Excessive worry. Unfortunately, many people tend to focus on the direst causes listed by symptom checkers, said Dr. Mehrota. (Yes, indigestion could be a sign of cancer, but it’s much more likely to be caused by too much meat loaf.) That heightened anxiety has given rise to a new disorder—cyberchondria. It’s old-fashioned hypochondria—multiplied by the ability to find new health worries every time you look at your phone.


It’s not that doctors have reached diagnostic perfection, either.  While they beat the technology tested in the Harvard study, they still gave the wrong diagnosis in about 15% of the cases, which is consistent with previous estimates of human misdiagnoses.


One goal of the study was to see whether computers could help doctors get better at diagnosis. They’re not quite there yet. “The technology will inherently improve as they build more complex ways of making the diagnosis,” said Dr. Mehrotra. One way, he suggested, is for the programs to build in data from epidemiology—real-time information about the frequency of illness in the community. After all, if you see your doctor with digestive ills and he or she knows that there’s a GI infection going around town, that may help target the diagnosis. Ideally, as the checkers improve, they’ll help doctors improve their diagnostic skills—and help the public, too.

In the meantime, we patients should exercise extreme caution in using these programs, said Dr. Mehrota. They can play a role, he suggests, if they help you educate yourself about possible causes of your symptoms—and that prompts a discussion with your doctor. That is, use them to educate yourself so you can ask smart questions. But for a real diagnosis, there’s no substitute for a real doctor.

Sources: Study titled, “Comparison of Physician and Computer Diagnostic Accuracy” by Ateev Mehrotra, MD, MPH, associate professor of health-care policy and medicine, Harvard Medical School, and hospitalist, Beth Israel Deaconess Medical Center, both in Boston, and colleagues, published in JAMA Internal Medicine.