A journal article authored by York University psychology Professor Christopher Green and nine of his students has been named the Best Article in Canadian Psychology in 2018 by the Canadian Psychological Association (CPA).
The journal Canadian Psychology is the flagship publication of the CPA.
Titled “Statcheck in Canada: What Proportion of CPA Journal Articles Contain Errors in the Reporting of p-Values,” the study used innovative digital methods to discover that many articles in CPA journals incorrectly report their statistical results (viz., p-values).
Christopher Green
“Our article followed from an article that had found the same for Euro and American psychological articles the previous year,” said Green.
He, along with students Sahir Abbas, Arlie Belliveau, Nataly Beribisky, Ian J. Davidson, Julian DiGiovanni, Crystal Heidari, Shane M. Martin, Eric Oosenbrug and Linda M. Wainewright, found the number of reporting errors was reduced when they hand-edited the statistics that had been digitally scraped.
A 2016 digital survey of several prestigious American and European psychology journals was conducted using a computer program called Statcheck. Results showed that the p-values reported in research articles did not agree with the corresponding test statistics at surprisingly high rates. It was determined nearly half of all articles contained at least one such error, as did about 10 per cent of all null hypothesis significance tests.
Researchers investigated whether this problem was present in Canadian psychology journals and, if so, at what frequency.
“We discovered similar rates of p-value errors in Canadian journals over the past 30 years,” said Green.
However, they also noticed a large number of typographical errors in the electronic versions of the articles. When they hand-corrected a sample of their articles, the per-article error rate remained about the same, but the per-test rate of errors dropped to 6.3 per cent.
The matter, he said, ties in with the broader “replication crisis” in psychology and other social/medical sciences in recent years.
The study recommend that, in future, journals include explicit checks of statistics in their editorial processes.
The article was published in 2018, and the award was recently announced.