You might think that having a job that keeps you active all day is good for your health.

That makes perfect sense given everything we’re told about the benefits of physical activity.

But based on a new research finding, it might be completely wrong—in fact, unless you take specific steps (we’ll explain), being in a physically-demanding job might actually shorten your life.

Past studies have hinted at this “physical activity paradox”—one in which people who exercise during their leisure time get healthier and live longer, but those who “exercise” by working in physical jobs are harmed. To find out whether it’s real or not, an international research team at the VU University Medical Center in Amsterdam gathered mountains of data from 17 studies conducted between 1960 and 2010, covering 193,696 men and women. They applied statistical techniques to weed out possible confounding variables such as chronic health conditions, body-mass index, smoking and alcohol use, so they could level the playing field between office workers and physical workers. And they didn’t take into account job-related accidents—which you might reasonably assume would happen more often in highly physical jobs—because what they were studying was health, not injuries.

Eye-opening finding: Men in jobs requiring high levels of physical activity had an 18% higher risk of early death than those who were in largely inactive jobs.

In contrast, women in these kinds of jobs were no more likely to die prematurely than those in more sedentary jobs. The reasons aren’t known, but one likely contributor is that even in our more gender-enlightened times, the active jobs women have tend to be different than the active jobs men have.

Working Hard, But Not in a Healthy Way

What’s so unhealthy about the kinds of labor-intensive jobs men tend to do? A leading hypothesis is that there are big differences between physical activity on the job versus during leisure hours—with very different effects on the body.

Example: A laborer who spends 40 hours a week doing manual, repetitive tasks while standing in a static position. That could easily elevate his blood pressure and heart rate—without providing the cardio-fitness benefits that come from sustained aerobic activity. In addition, he’s doing these physically demanding tasks all day long—often without a sufficient recovery time. Chronic exhaustion and high blood pressure both contribute to heart disease—and thus, early death.

Contrast that with leisure time activity, whether it’s walking, working out at the gym or playing golf or tennis, hiking or swimming. You’re active for a shorter bout of time, engaged in aerobic activity that elevates your heart rate in a sustained way—and you have time to recover afterward. No wonder it leads to a healthier outcome.

There are exceptions, of course. Elite professional athletes, whose occupations require that they devote long hours to physical activity in the pursuit of fitness and competitive edge, tend to live longer lives than the general population, the researchers note. One could speculate that certain occupations—say, a mailman who walks his route—may be pretty healthy, too. Unfortunately, most of the studies didn’t identify specific jobs but rather by exertion levels, there’s no resulting list of jobs to avoid…or seek.

But one take-away is clear: If you’re in a physically-demanding job—whether you’re a man or a woman—you can’t necessarily rely on your work-a-day activity to keep you fit and healthy. To do that the human body needs resistance (muscle) exercise, aerobic exercise and recovery time—so you might need to use your leisure time to get and stay fit, too. That’s particularly important because research finds that people in physically demanding jobs tend to get less leisure time activity than those in more sedentary jobs. For the rest of us, it’s more evidence that staying physically active in our leisure time is key to a healthier, longer life.