Charles Darwin, who was married to his first cousin Emma Wedgwood, was the first experimentalist to demonstrate the adverse effects of inbreeding. He documented the deleterious consequences of self-fertilization on progeny in numerous plant species, and this research led him to suspect that the health problems of his 10 children, who were very often ill, might have been a consequence of his marriage to his first cousin. Because Darwin's concerns regarding the consequences of cousin marriage on his children even nowadays are considered controversial, we analyzed the potential effects of inbreeding on fertility in 30 marriages of the Darwin–Wedgwood dynasty, including the marriages of Darwin's children, which correspond to the offspring of four cousin marriages and three marriages between unrelated individuals. Analysis of the number of children per woman through zero-inflated regression models showed a significantly adverse effect of the husband inbreeding coefficient on family size. Furthermore, a statistically significant adverse effect of the husband inbreeding coefficient on reproductive period duration was also detected. To our knowledge, this is the first time that inbreeding depression on male fertility has been detected in humans. Because Darwin's sons had fewer children in comparison to non-inbred men of the dynasty, our findings give empirical support to Darwin's concerns on the consequences of consanguineous marriage in his own progeny.