Scientific Peer Review During the Pandemic and Beyond: Changes, Benefits, and Impacts

Posted on by Michael Goldcamp, Melinda Sinkule, JoAnne Fairbanks, and Allen Robison

 

The COVID-19 pandemic began significantly impacting the operations of the National Institute for Occupational Safety and Health (NIOSH) in March 2020. Travel restrictions were implemented, and CDC employees ordered to work from home. The Office of Extramural Programs (OEP) Review Team had a confirmed in-person peer review meeting for World Trade Center Health Program (WTCHP) research applications occurring within a week. Suddenly it was impossible to convene this, or any, in-person panel to review applications.

NIOSH/OEP adapted quickly to move from in-person peer review meetings to online review meetings. NIOSH was able to conduct the WTCHP meeting using the Virtual Meeting (VM) module within the NIH Internet Assisted Review (IAR) system, which was already in use for data collection in face-to-face meetings. Thus, no additional infrastructure was required.  The IAR system was selected over teleconference because of the large size of the meeting and limited access to teleconference lines.

Panelists were notified of the change and travel, hotel, and conference space were cancelled. The 32 reviewers were immediately given orientation to the IAR Virtual Meeting system and the original meeting dates remained unchanged. The meeting successfully provided a high-quality review of the 35 applications.

Since March 2020, all 42 additional review panel meetings have been virtual, either online (VM or Zoom) or telephone. In FY2022, all 15 review panels were held using Zoom.

Benefits and Challenges

Benefits associated with this change include reduced time burden on reviewers and significantly lower meeting costs due to the absence of travel, meeting space, and conference services. Honorariums for participation continue to be paid to panelists. New meeting formats and procedures have increased the efficiency of the review process.

There are some challenges, however, in managing the large number of reviewers needed with large numbers of applications, especially if they are complex and multi-component. To address the challenge of managing these large meetings virtually, OEP initiated the use of multiple panels to complete high quality reviews in a timely manner and reduce peer reviewer burden.

Minor drawbacks discovered with using virtual meetings include managing quorum and providing opportunities for reviewers to interact before and after meetings. With reviewers from different time zones, ensuring that a minimum number of reviewers are present to achieve quorum can be challenging.  If reviewers turn off their video when taking breaks, the Scientific Review Officer assigned to the meeting will know the number of reviewers present at the meeting at any given time.

Reviewers see networking opportunities associated with face-to-face meetings as a benefit of participating in reviews. While some have expressed concern about the lack of networking opportunities with virtual reviews, it has not impacted recruiting, most likely because traveling is not required. While not a replacement for face-to-face networking, NIOSH has allowed time after the meetings for virtual discussion among participants.

Conclusion

As we continue using virtual meetings, we will refine our approaches and adopt new tools and technologies. Upon return to in-person meetings, likely on a limited basis, lessons learned, such as the use of panels to manage large meetings, will be applied. A paradigm shift has occurred, is now accepted, and will continue to develop.

Please share the benefits or challenges you have experienced associated with the use of virtual meetings in the comment section below.

 

Michael Goldcamp, PhD, is the Review Team Lead in the NIOSH Office of Extramural Programs.

Melinda Sinkule, MA, is a Grants Technical Assistant with the NIOSH Office of Extramural Programs.

JoAnne Fairbanks is a Committee Management Specialist with the NIOSH Office of Extramural Programs.

W. Allen Robison, PhD is the Director of the NIOSH Office of Extramural Programs.

 

Posted on by Michael Goldcamp, Melinda Sinkule, JoAnne Fairbanks, and Allen Robison

One comment on “Scientific Peer Review During the Pandemic and Beyond: Changes, Benefits, and Impacts”

Comments listed below are posted by individuals not associated with CDC, unless otherwise stated. These comments do not represent the official views of CDC, and CDC does not guarantee that any information posted by individuals on this site is correct, and disclaims any liability for any loss or damage resulting from reliance on any such information. Read more about our comment policy ».

    Scientific peer review is an essential aspect of the scientific process. It is a system in which experts in a particular field evaluate the quality and validity of research papers before they are published in academic journals. Peer review helps ensure that only the most rigorous and well-supported research is disseminated to the scientific community.

    During the COVID-19 pandemic, the scientific peer review process has undergone significant changes. With many researchers unable to physically meet and discuss their work, there has been an increase in the use of online platforms for peer review. This shift has allowed for more flexibility and efficiency in the process, as researchers can review papers from anywhere with an internet connection.

    There have also been efforts to expedite the peer review process for research related to the pandemic. This has been done in order to get important findings out to the public as quickly as possible, as the pandemic has highlighted the need for timely dissemination of information. However, it is important to note that this accelerated review process does not compromise the rigor of the review itself. Peer reviewers are still expected to carefully evaluate the quality and validity of the research being considered for publication.

    Overall, the changes to the scientific peer review process during the pandemic have had both benefits and impacts. The increased use of online platforms and expedited review processes have allowed for greater flexibility and efficiency, but it remains important to maintain the high standards of the peer review process in order to ensure the quality and integrity of the research being published.

    sincerely
    Kipang Payakumbuh

Post a Comment

Your email address will not be published. Required fields are marked *

All comments posted become a part of the public domain, and users are responsible for their comments. This is a moderated site and your comments will be reviewed before they are posted. Read more about our comment policy »

Page last reviewed: October 26, 2022
Page last updated: October 26, 2022