Don’t fall for these seven myths of research
Clinical journal highlights UC professors’ cross-college collaboration
A new publication by two University of Cincinnati researchers contends that adjusting how researchers approach their statistical analysis has the potential to change the lives of children and adolescents struggling with mental health issues across the world.
Jeffrey Mills, PhD, and Jeffrey Strawn, MD, have been collaborating on interdisciplinary research and data analysis for years. Their latest paper, “Myths of Randomized Controlled Trial Analysis in Pediatric Psychopharmacology,” was selected as an Editor’s Pick for the Journal of Child and Adolescent Psychopharmacology.
Improvements in biostatistics have deepened researchers’ understanding of some of the limitations of analytical methods, especially when it comes to nuances in pediatrics.
The “myths” referred to in the paper’s title are common misconceptions within the field of child and adolescent medicine, specifically in regard to treatments for mental health studied through randomized controlled trials.
“As a physician-scientist, you hope your work is both rigorous and clinically useful. Being selected as an Editor’s Pick means that we’d accomplished this,” said Strawn, professor in the Department of Psychiatry and Behavioral Neuroscience in UC’s College of Medicine.
Strawn, who is also a UC Health adolescent psychiatrist, collaborates with Mills to make sure data analysis in child and adolescent psychiatric clinical trials is clear and accurate.
“The myths in the paper are problems within statistical analysis, things researchers should keep in mind and errors repeatedly being made,” said Mills, professor of economics in UC’s Carl H. Lindner College of Business. “We're trying to point those out for people so it’s top of mind, not only when they read the journals, but when they are producing their own clinical analyses.”
Strengthening research through cross-collaboration
Mills and Strawn have been teaming up for clinical analysis for years. Photo/UC Marketing + Brand
Strawn emphasized the importance of interdisciplinary research, or team science, for improving medicine. “Imagine you’re a parent whose child is struggling with anxiety or depression. You’re told a medication doesn’t work because the trial showed ‘no significant effect.’
“The work Mills and I have done together over the years shows that those ‘no effect’ results might not mean the treatment failed,” he said. “Instead, the results might mean the analysis failed to see an effect.”
“In psychiatry, and especially child and adolescent psychopharmacology, trials are hard. The sample sizes are smaller. The statistical noise is greater. And the stakes for young patients are high. We need sound trial design, advanced statistical methods and variance in perspectives,” Strawn said. “It’s no longer enough to ask: ‘Did the drug work?’ We have to ask for whom, when and why.”
Mills said that it is incredibly difficult to squeeze in data analysis while also working as a medical provider, which is one of his motivations for getting involved in adolescent psychiatry as an econometrician.
His expertise in collaboration with Strawn has them questioning the typical approaches to biostatistics, such as five percent being the threshold for statistical significance. Mills argues that five percent is an arbitrary level and that the threshold of significance varies depending on context. “In clinical trials, you have all that expense, time and effort of all these people involved in your research, and the results are important in terms of helping people. So you should do the best you can with the statistical analysis,” he said.
Strawn said that in research on children and adolescents, there are so many variables that are easy to overlook, such as age, economic status and even specific symptoms. “By improving how we interpret trials, we’re not just tweaking statistics, we are developing ways to get more accurate answers, more personalized treatments and better outcomes,” he added.
Mills and I speak different scientific languages — psychiatry and economics — but we have the same goal.
Jeffrey Strawn, MD UC College of Medicine
The myths of randomized controlled trial analysis
A patient’s environment, including their school, socioeconomic status and potential other conditions are all factors in a treatment’s effectiveness. As it is impractical to control all of these extraneous variables, randomization works better for large samples where these additional factors are less influential on results.
The five percent level of significance is a convention, but not a law. The broader context and possibilities for error are important to consider in pediatrics, where sample sizes are typically small with high levels of variability.
Time and budget constraints often lead researchers to gather a large amount of data with multiple varying factors instead of a smaller, more “reliable” set. But a small, well-designed trial can prove to be much more reliable, even if it takes more time to complete.
With a better understanding of biostatistics, flaws in the “mixed-model for repeated measures” (MMRM) analysis method become exposed. In child psychopharmacology, there are complex datasets that may involve nonlinear relationships that could benefit from newer, dynamic methods of analysis.
It is common to adjust the statistically significant level for a dataset depending on the number of tests conducted. Correcting for multiple comparison bias can be overly conservative, potentially resulting in missed effects in the data.
Prediction is one of the most crucial steps of the scientific method of evaluating a hypothesis. By only testing if data from a researcher’s own sample supports a given hypothesis, it is easy to miss whether that hypothesis can predict treatment results on patients that are not part of the current study.
Refusing to revisit data as new theories emerge can be counterproductive to scientific advancement. In fact, more data and new and improved analyses can enhance the reliability of clinical findings even further.
The duo hopes that their work will continue to forge the path ahead for transdisciplinary research and collaboration. “There's a lot of cross-fertilization that has happened in terms of what I've learned from working in econometrics and what Dr. Strawn knows working in psychiatric medicine,” Mills said.
“Team science is critical in modern clinical research,” Strawn said. “Mills and I speak different scientific languages — psychiatry and economics — but we have the same goal. That is to say, we are both committed to getting better treatments to patients faster and improving the treatments that we already have.”
Read the full report on these myths in the Journal of Child and Adolescent Psychopharmacology.
Next Lives Here
The University of Cincinnati is classified as a Research 1 institution by the Carnegie Commission and is ranked in the National Science Foundation's Top-35 public research universities. UC's medical, graduate and undergraduate students and faculty investigate problems and innovate solutions with real-world impact.
Featured image at top: Strawn and Mills examine data on a computer together. Photo/UC Marketing + Brand
Related Stories
Make Hoxworth Blood Center’s special holiday events part of your family celebrations this December
December 12, 2025
This December, Hoxworth Blood Center, University of Cincinnati, is inviting families across Greater Cincinnati to add something truly meaningful to their holiday traditions: giving the gift of life. With festive community events, beloved local partners and special thank-you gifts for donors, Hoxworth is making it easier, and more heartwarming than ever, to roll up your sleeves and help save lives close to home.
Ohio nurses weigh in on proposed federal loan rule
December 12, 2025
Spectrum News journalist Javari Burnett spoke with UC Dean Alicia Ribar and UC nursing students Megan Romero and Nevaeh Haskins about proposed new federal student loan rules. Romero and Haskins, both seniors, were filmed in the College of Nursing’s Simulation Lab.
UC awarded nearly $1 million to help fight infant obesity spike
December 12, 2025
University of Cincinnati researcher Cathy Stough spoke with Spectrum News1 about a nearly $1 million National Institutes of Health grant awarded to UC to help prevent infant obesity through early nutrition support and family-based interventions.