Download the guide (PDF, 82KB)

The purpose of this document is to provide psychology students who aspire to research or academic careers with the background information necessary to understand the peer review process, and make informed decisions about when and how to become involved in it. This document is based on a symposium on this topic presented at the 2005 Annual Convention of the American Psychological Association and sponsored by the APA Science Student Council (Hoyle, Hall, & Tang, 2005).

Download the guide (PDF, 82KB)
What is the peer review process?
Quite often aspiring scientists think of the preparation of a manuscript as the final step in the research process; however, a finished manuscript is oftentimes only the beginning step in the process of having research findings published (Kazdin, 1995). Scholarly journals provide the usual outlet for research manuscripts, and most of these journals operate under the system of peer review. Generally speaking, this system involves the examination of a manuscript by a number of individuals with a record of publication in the substantive area that the manuscript addresses. (The Guidelines for reviewing section of this document further elaborates on the process that peer reviewers should follow when examining a document). Following a thorough review, each reviewer is asked to prepare a letter with suggested changes to a manuscript. In addition, the reviewer will make a recommendation about whether to accept a manuscript as is (a rare occurrence), accept with revisions, suggest that the author revise and re-submit the manuscript, or that the manuscript be rejected without the possibility of re-submission. An editor or assistant editor of the journal will then incorporate all of the reviewers’ suggestions into an editorial decision letter. A sample decision letter that includes reviewers’ comments is presented in Appendix A.
Why should graduate students become involved in peer review?
The development of skills in reviewing and the management of involvement in the review process are most often thought of as an early and mid-career process (Taylor & Martin, 2004). Indeed, some have suggested that individuals with established careers should be more selective in accepting invitations to review and pass some of these on to early to mid-career colleagues (Roediger & Balota, 2004). Graduate students who, having met all other expectations of their training, also obtain some experience with peer review may be able to more easily transition into the greater involvement with peer review that is necessary during their career. Experience with peer review may also be a distinguishing characteristic to an employer selecting among many candidates who are otherwise similarly qualified.
How can graduate students become involved in the peer review process?

The most typical ways in which students become involved in the peer review process is by being asked to review for a journal in which they have already been published, or by being asked to assist in an advisor’s (or other faculty’s) review. Students have also become involved as reviewers by contacting an editor or associate editor about the possibility of doing so. It must be noted that the chances of being successful with this last approach increase when a student has prior experience publishing their own work. For this reason, it is most often advanced graduate students (i.e., post-master’s or equivalent) that are involved in the peer review process.

There are also a number of journals, such as Cultural Diversity and Ethnic Minority Psychology and the Journal of Social Issues, that have student spots on their editorial board. These positions are often appointed by the current journal editor and are typically given to students with some publishing experience. Finally, there are some journals that are entirely student run. An excellent example is Representative Research in Social Psychology, a journal that for over thirty years has been run by graduate students at the University of North Carolina and includes graduate students from other institutions on its editorial board.

Guidelines for reviewing and managing involvement in the review process

Focus of the review

Reviews of psychological manuscripts should be focused on answering three main questions (Kazdin, 1998):

  1. Does this paper make an important substantive contribution to this area of research?
  2. Does the methodology (design and execution) permit one to draw the conclusions the author wishes to make?
  3. Is the paper well organized and complete in explaining what was done and why and how it was done?

Aside from answering these questions, a good review should point to specific revisions that are necessary in order for each of these answers to be in the affirmative. These revisions may be as minor as changes in the structure of an introduction or discussion, or as involved as the conduct of a follow-up study that will clarify limitations that can not be overlooked in the current study (see Sternberg, 1997, for a comprehensive guide to reviewing scientific works in psychology). The example presented in Appendix A is adapted from several actual reviews and may be a helpful guide for determining the tone and focus of a review.

Tone of the review

It is helpful to think of reviews as an opportunity to provide the authors with constructive feedback (Taylor & Martin, 2004). Good reviews are thorough and balanced, highlighting both the limitations and strengths of a paper. In addition, reviews should provide specific feedback as to changes that may improve upon the manuscript’s limitations (Roedinger & Balota, 2004). For instance, the statement “the use of an analytical technique such as Latent Growth Modeling seems more appropriate for the authors’ data, and would better support their conclusions” is descriptive and helpful. The statement, “the authors’ use of an outdated and inadequate analytical technique such as cross-lagged analyses is an embarrassment” to address the same concern is more evaluative and not as helpful.

It is also important to avoid ad hominem attacks (Sternberg, 2004), which are imputations about an author’s character based on the manuscript (e.g., “the authors use of arcane analytical methods and patent inability to grasp the most prevalent theoretical perspectives are an insult to this field of study”). Finally, it is worth noting that a common tendency of most new reviewers is to focus too much on grammar or spelling (Taylor & Martin, 2004). Reviewers are discouraged from attending to this aspect of the writing and encouraged instead to focus on answering the three main questions posed in the Focus of the Review section. Decisions about language use are typically the responsibility of the Editor or Associate Editor.

Managing involvement in the review process

One rule of thumb for early and mid-career scientists is to never be reviewing two manuscripts at the same time and to cap the number of reviews that one accepts at no more than 20 per year (Taylor & Martin, 2004). Most graduate students with aspirations to a research career may do well to consider having a much more reduced involvement in the peer review process and devoting most of their time to the production of scholarly output in the form of manuscripts (Hoyle et al., 2005).

Graduate students who chose to involve themselves in the peer review process should ensure that this involvement does not detract from their research and their educational requirements (such as qualifying / comprehensive examinations and the dissertation), as these have more influence toward the establishment of an academic career than involvement in peer review. Because of differences in training requirements and individual abilities, a rule of thumb is difficult to muster; however, involvement in one review per academic semester should be sufficient to provide an advanced graduate student adequate experience with the peer review process.

Students are urged to ensure that any decision about involvement in the peer review process is an informed one, and one that fosters, rather than detracts from, their professional development. This brief document has provided some of the basic information to consider. Consultation with advisors, other faculty, and senior colleagues may also be beneficial in gathering more information and in seeking reviewing opportunities.

References

Hoyle, R. H., Hall, C. G. N., Tang, S. (2005, August). Foot in the Door: How Students Can Become Journal Reviewers. Presentation delivered at the 2005 Annual Conference of the American Psychological Association. Washington, D.C.

Kazdin, A. E. (1995). Preparing and evaluating research reports. Psychological Assessment, 7, 228-237.

Kazdin, A. E. (1998). Methodological issues and strategies in clinical research, Second edition. Washington, D.C., USA: American Psychological Association.

Roediger, H. L, & Balota, D. A. (2004). Managing your career: The long view. In J. M. Darley, M. P. Zanna, and H. L. Roedinger (Eds.), The compleat academic: A career guide. Second Edition (pp. 393-408). Washington, D.C., USA: American Psychological Association.

Sternberg, R. J. (Ed.) (1997). Reviewing scientific works in psychology. Washington, D.C., USA: American Psychological Association.

Sternberg, R. J. (2004). Psychology 101 ½: The unspoken rules for success in academia. Washington, D.C., USA: American Psychological Association.

Taylor, S. E., & Martin, J. (2004). The academic marathon: Controlling one’s career. In J. M. Darley, M. P. Zanna, and H. L. Roedinger (Eds.), The compleat academic: A career guide. Second Edition (pp. 363-392). Washington, D.C., USA: American Psychological Association.