Are scientists less rigorous about applying proper research standards when their subjects are mice? That’s the question Jennifer Couzin-Frankel investigates in the latest issue of Science—and her findings are disturbing. According to her paper, “When Mice Mislead,” scientists working with mice routinely use small sample sizes, select their subjects unsystematically, and even lose track of their data or—most ominously of all—leave out results that don’t support their research. The problems with extrapolating findings from rodents to humans are well-known, but if the research is sloppy, the links could be even more tenuous than scientists realized.
Ulrich Dirnagl, a researcher in Berlin, told Couzin-Frankel he was reviewing a paper examining rodents’ response to a new drug to treat strokes when he realized something was off: The study tested 20 mice—but only 17 were represented on a graph of the final results. When Dirnagl wrote to the paper’s editor, he learned that the three missing mice had suffered strokes and died. Rather than dealing with the implications of this for the drug, the researchers decided to drop them out of the study.
"This isn't fraud," says Dirnagl, who often works with mice. Dropping animals from a research study for any number of reasons, he explains, is an entrenched, accepted part of the culture. "You look at your data, there are no rules.… People exclude animals at their whim, they just do it and they don't report it."
That sounds a lot like fraud. Understandably, the scientists Couzin-Frankel interviewed were wary of tarnishing the public’s perception of animal research—but their revelations are worrying nonetheless.
"I was trained as an animal researcher," says Lisa Bero, now a health policy expert at the University of California, San Francisco. "Their idea of randomization is, you stick your hand in the cage and whichever one comes up to you, you grab. That is not a random way to select an animal." Some animals might be fearful, or biters, or they might just be curled up in the corner, asleep. None will be chosen. And there, bias begins.
Another issue is small sample size: “For ethical and cost reasons,” writes Couzin-Frankel, “researchers try to use as few animals as possible, which can mean minuscule sample sizes…. A survey of 76 influential animal studies found that half used five or fewer animals per group.”
Malcolm Macleod, a neurologist at the University of Edinburgh who began publishing on lax standards in animal research ten years ago, believes the problem is systemic and self-perpetuating: Researchers don’t think they’re doing anything wrong, because they’re emulating the less-than-rigorous techniques of their mentors. It looks like steps may be taken to improve lax standards: This year, some of the leading research institutes, such as the Environmental Protection Agency, announced new guidelines for animal researchers, and journals like Nature and Science Translational Medicine are requiring more detailed information on how researchers select subjects, and whether they can justify leaving some of them out of their results.