Refereeing

Nearly everyone has wildly different expectations of the goals, duties, and time expenditures expected of a referee. At times this leads to combative encounters, complaints to editors, editorial decisions thought to be bizarre by both the author and referee of the same paper, and incorrect results being accepted.  While every paper and process is somewhat unique, many of these problems likely stem from a lack of any guidance and the lack of a consistent set of expectations within the community.  A relatively large collection of postdocs/young faculty have no idea what the expectations are for the referee process, with the only guidance being the very wide range of referee reports we have received over the years. Below, we attempt to provide some of this lacking guidance.

Roles of the referee:

• decide the "reasonableness" of the paper
• point out obvious errors, if any
• evaluate major claims of paper for reasonable-ness
• suggest changes that would improve the readability and accessibility of the paper
• judge whether the paper explains its methodology to a sufficient degree to be reproducible and falsifiable

Things referee is NOT responsible for:

• redo all calculations
• identify and fix grammatical or typesetting errors
• suggesting major additions. Resist the urge to turn the paper into what you would've written. Be a referee and evaluator, not a co-author.

Things we wish referees did not do:

• Demand gratuitous references to their own papers.
• Take three months to return a cursory report that shows they barely read the paper.
• Be mean and condescending to authors. Try to write in the same (respectful) tone you would hopefully use if the author was standing right in front of you.

 Links

• Tales from the Other Side of the Fence | Professor in Training - good advice for responding to referee reports.
• Nature's Peer-Review Policy - nice advice and description of the system
• Quick Guide to Writing a Solid Peer Review - Nicholas and Gordon in EOS Transactions.
• Peer Review: The Nuts and Bolts - from Biologists, but very good.
• How to be an effective referee - Slides from an Astronomical Society of Australia Early Career Researcher training workshop
• How to become good at peer review - Jennifer Raff's advice, quite complete and with references.
• Peer Review Advice - Luigi Foschini
• Slides from AAS Workshops on the Refereeing Process - presentations by Richard de Grijs (ApJL) and W. Butler Burton (ApJ)