Images of joyous late-in-life fathers and their newborn babies have danced across the pages of celebrity magazines and into our shopping carts for many years. Mick Jagger, Steve Martin, Robert DeNiro and Alec Baldwin are just a few of the A-listers who have discovered that you’re never too old to delight in creating a “mini me” and revel in the sweetness of your infant’s smile.

However, becoming a dad later in life is not confined to the rich and famous. While four decades ago, only 4% of babies were born to men over 40, today that number has more than doubled—and not in small part due to all those celebrity cover stories.

While the health risks for babies of older moms have been known for some time, a study done at Stanford University suggests that a man’s age can have a completely different set of concerns, not only for their babies but also for the moms as well.

The study tracked the maternal records of 40.5 million live births in the US and organized the babies’ dads into five age groups, ranging from under age 25 to over 55. The researchers also controlled for factors such as smoking history, access to medical care and the mother’s age—anything that might skew the results.

What they found was stunning. Babies born to men over age 45 were 14% more likely to be born prematurely, 14% more likely to have a low birth weight and 14% more likely to be admitted to an intensive care nursery for fetal monitoring than when the dad was under age 35.

Babies born to fathers age 55 and over were an additional 10% more likely to need a ventilator to help with breathing upon birth and an additional 28% more likely to be admitted to an intensive care nursery.

Surprisingly, the women giving birth to older men’s babies were 28% more likely to develop gestational diabetes, a condition that’s dangerous for both the mom and her unborn baby during pregnancy and possibly beyond, even when the gestational diabetes itself goes away.

The reason for all these risks is uncertain, but several likely possibilities exist. Along with wrinkles in their skin, older men get the equivalent of wrinkles in their sperm. Once a man begins to produce sperm, he begins to accumulate two new mutations to his DNA every year. Those mutations may accumulate in genes relevant to creating a baby.

What can you do? The connection between the dad’s age and the baby’s (and the mom’s) health needs more study. But for now, older men should consider these findings and discuss them with their doctor and their partner before deciding if and when it’s time for a child. And younger men and their partners should also weigh them as they start family planning—and perhaps not wait as long as they thought they might to have children.