The peer review system is fundamental to academic publishing. It has been set in place to ensure the quality and credibility of research. However, as publication numbers have continued to grow, so have concerns about reviewer availability and workload distribution. While an increase in the number of publications could mean a comparable increase in the number of reviewers, real-world data suggests that the number of expert reviewers, with more than five years of experience, continue to remain in short supply. This is the point senior data scientist at Digital Science, Hélène Draux, explored in her recently published article, “The reviewer paradox: more publications, fewer peers?”
By using Dimensions on Google BigQuery (GBQ), Dr Draux analyzed peer review trends from 1950 to 2023 and revealed that the reviewer demand has outpaced reviewer supply in many research fields, particularly in Humanities and Social Sciences. Dr Draux’s results revealed a discipline-specific disparity in peer review workloads. While the pool of researchers has grown, the number of reviewers has not done so evenly across disciplines, her analysis revealed.
So why doesn’t the reviewer pool scale with publications? Draux’s article reveals many factors that lead to short supply– from lack of experience, to multiple submission cycles requiring repeated reviews, further increasing the burden and reviewer fatigue.
A 2008 survey found that researchers reviewed an average of eight papers per year, which falls short of the estimated demand across all disciplines. However, Draux points out that the survey may have underestimated the strain on reviewers, as it does not account for fatigue, competing professional obligations, or subject-matter specificity.
A broader pattern of structural imbalances in academia
The imbalance in peer review capacity is just one example of structural inadequacies in academic publishing. An analysis of AI research trends using Dimensions tools, including Dimensions GBQ, found that AI publications and funding are heavily concentrated in a few advanced economies, such as the US and China, with Africa, South America and many Asian countries contributing to less than 5% of the research outputs. Just as peer review burdens are unevenly distributed across disciplines, AI research outputs and funding is disproportionately concentrated in a few regions.
By using tools such as Dimensions GBQ, academics and publishers can move towards having a better understanding of structural patterns in academic publishing. The peer review analysis, for example, shows that more researchers and research publications do not automatically translate into a sufficient reviewer pool, and that review demand varies significantly across disciplines.
Do you want to know more about how Dimensions GBQ can be used for detailed, data-driven insights? Then don’t hesitate to contact the Dimensions team.
For further reading: