Peer Review: When and How Will the NIH Process Change?

For the past year, staff and committees of the National Institutes of Health have carefully examined the two-tiered peer review system through which NIH rates and ranks grant applications.

For the past year, staff and committees of the National Institutes of Health have carefully examined the two-tiered peer review system through which NIH rates and ranks grant applications. At this time last year, NIH Director Elias Zerhouni, MD, established two peer review working groups, whose charge was to help get input from all stakeholders about what works well, and works less well, in the peer review process. These groups include:

  • The Advisory Committee to the Director Working Group (ACD WG), co-chaired by Dr. Keith Yamamoto, Executive Vice Dean, School of Medicine, UCSF and Dr. Lawrence Tabak, Director, National Institute of Dental and Craniofacial Research, NIH; and

  • The Steering Committee Working Group (SC WG), co-chaired by Dr. Tabak and Dr. Jeremy Berg, Director, National Institute of General Medical Sciences, NIH.

During the summer and fall of 2007, the working groups collected input and ideas for enhancing the peer review system from all stakeholder communities (i.e. extramural community, advocacy groups, professional society groups, and NIH staff). This process included an online Request for Information (RFI), an NIH-internal survey, an interactive website for liaisons, analyses of previous and existing peer review experiments and practices at the NIH and other agencies (international and domestic), direct communication with stakeholders through teleconferences with Deans, emails, letters, and a series of internal and external consultation meetings and regional meetings across the nation.

See the following links for past SPIN pieces explaining APA’s participation in the process of gathering scientific comments to help inform changes: Feedback on Peer Review from Behavioral Scientists, Your Opportunity to Speak Out, and Psychologists Speak Out on Peer Review.

The Advisory Committee to the NIH Director will meet again next week, on June 6, 2008, to consider implementation of the type of changes in process that have been recommended. Two psychologists, Nancy Adler, PhD, of the University of California San Francisco and Alan Leshner, PhD, Chief Executive Officer of the American Association for the Advancement of Science, serve on the Advisory Committee to the NIH Director, and are heavily involved in discussions about the future of peer review.

What types of changes are being discussed? They fall into broad categories at this point in the deliberations, with the details not fully settled. First, there is support for the idea that the grant application form should be shorter, focusing more on ideas and impact, and less on preliminary data. Many NIH comments about this focused on the need to enable quicker reading of grant applications so that more reviewers may read each application and therefore they may be more effectively ranked. Second, five elements in each grant proposal are likely to be rated separately: impact, investigators, innovation, climate, and environment, although each grant will get an overall rating. Many comments supported a review model closer to that of an editorial board. Many also reflected the need to provide incentives to encourage top scientists to serve on peer review panels. And new mechanisms will continue to be tested that improve support for new investigators and that encourage and reward transformational and interdisciplinary research (e.g. Pioneer Awards and other Roadmap programs).

You may read a transcript of the most recent discussion of these issues by the Advisory Committee to the Director and review the slides about the broad recommendations.

In the following weeks, The NIH Director will carefully consider the recommendations of the Advisory Committee to the Director and his internal committee and institute any changes. APA’s Science Government Relations Office and experts in the Science Directorate are monitoring this process carefully and will communicate quickly with APA scientists about any additional opportunities for comment, as well as any changes that are adopted.