For Reviewers
due August 26th, 2022 at 11:59pm
Ideally, the review process presents an opportunity for you to reflect on others’ work in addition to your own. Given the breadth of submission topics and methods, there is no one-size-fits-all approach to reviewing. We recommended looking through the APA Review Guidelines for Graduate Students > and the Gender & Society Review Guidelines >. Here are a few additional suggestions to get you started.
Give yourself enough time to read the abstract carefully and to offer constructive, actionable guidance to the author. (We recommend forty-five minutes, especially if you are new to reviewing abstracts for conferences.)
Your review should reflect on:
(1) the abstract’s engagement with related work
(2) motivation
(3) soundness, particularly in relation to methods and claims
(4) alignment among literature, questions, methods, designs and/or findings
(5) broader impacts
(6) presentation and organization of abstract
When reviewing an abstract, it may be tempting to lead with and focus on criticism and critique. Instead, we suggest an approach common in ethnographic research that involves alternating between emic and etic stances, that is listening to understand an author on their own terms and, then, applying your own perspectives (e.g., Rogoff, 2003). For instance, it may be helpful to an author (and to you as a reviewer) if you try to reflect back what you think the author is communicating. Focus feedback on what you find to be the most salient aspects of the abstract. After summarizing for the author what you took from their piece, acknowledge both stronger and weaker points of the submission from your perspective as a reader, ideally with recommendations for how to strengthen the abstract.
While we are facilitating an anonymized review process, to help foster transparency, accountability, and a greater sense of community, you may choose to sign your review with your name so that the author(s) may contact you if they would like to follow up with any questions.
It may be helpful to explore additional resources to get started, especially if reviewing is a new process for you.
​
Resources
Review FAQs
What can a review process achieve?
The review process is a learning opportunity for both authors and reviewers. Reviewers can return to key relevant literature, think with others in the field (including their advisors who may be resources for participating in a review process), and reflect on what they know regarding theory, methods, and argumentation. Further, by communicating their feedback to authors in writing, reviewers are practicing putting ideas down and crafting arguments. For authors, the review process can be an opportunity to hear from a reviewer how their ideas are being heard. Many of the submissions for this conference are works in progress; reviewing work at this particular stage should aim to offer guidance to improve the work with acknowledgement that analysis may not be final or complete. For additional thoughts on what a review process might achieve, see Wang’s 2018 editorial in Human Development.
How are reviewers assigned for this conference?
Each corresponding author is asked to review at least two abstracts for each item they submit. Our aim is to assign reviews on topics/methods that you’re comfortable with and knowledgeable about, such that feedback can be most valuable to the authors whose work you’re reviewing. We group abstracts by similar topic based on selections authors made in the EasyChair submission portal. Once we put abstracts into subgroups, we pay attention to keywords and titles, as well as read through abstracts. Although it’s difficult to ensure perfect matches, we do our best to assign abstracts to review based on your own abstract topics and/or methods. We do not assign reviewers from the same university to try to maintain anonymity.
Who’s assigning these abstracts for review?
The Submissions and Review Committee, a group within the larger LSGS Conference Planning Committee, goes through all submissions and assigns two abstracts to each person who submitted an abstract of their own. This year, the Submissions and Review Committee is comprised of graduate students at the University of Wisconsin–Madison, University of Illinois Urbana-Champaign, Indiana University–Bloomington, University of Iowa, Stanford University, Northwestern University, Worcester Polytechnic Institute, University of Maryland, American University of Beirut, University of Minnesota–Twin Cities, and University of Calgary.
What if I have a conflict with the abstract I was assigned to review?
(e.g., I know the author and their work well)
Please let us know by replying to the email you received via EasyChair! We will do our best to re-assign the abstract and find another abstract for you to review.
​
Have questions?
Email Fangli Xia [ fxia24@wisc.edu ] &/or Kelsey Schenck [ keschenck@wisc.edu ]
​
An example review
Example Review (before)
This abstract describes a qualitative study on students’ use of a computational model of climate change and makes the claim that use of the model enhanced students’ abilities to use NGSS scientific practices. The authors engage with appropriate literature but might also consider connecting their argument to theories of conceptual change (see papers by diSessa, Vosniadou and others) to align discipline-based research on scientific practices with more cognitive perspectives. Descriptions of the demographics of the student participants and school context would make the results more contextualized so that the generalizability of findings can be better assessed. The authors made causal arguments that student behaviors were different after using the model but did not provide sufficient evidence on what student behaviors were like before using the model nor did they use a control group, which limits the usefulness of the study. Reducing the causal claims would provide more nuance and accuracy.
​
Meta-Commentary on original review
-
This review begins with a reviewer’s summary of the key purpose of the piece. The reviewer then offers constructive criticism that includes a suggestion for a useful concept from research literature and recommendations for further reading.
-
The review focuses on the core contributions of the abstract, rather than only offering suggestions on word choice or grammar. While word choice and grammar are important in communicating ideas (and having a reader understand and trust you as an author), a reviewer should try to provide feedback that focuses on substantive contributions an author might make.
-
To improve this review, the reviewer might have used emic and etic stances by acknowledging both the author’s intentions and the reviewer’s interpretation of the aim of the abstract. The current phrasing frames generalizability and causality as assumed goals for research, which may not be appropriate for a qualitative study.
-
The reviewer also offers advice on what could be added to the submission but does not offer suggestions for what could be removed from the submission or why it could be removed; authors have limited space, so helping them prioritize what to include is more beneficial than only requesting additional information.
​
Review example (after revision)
This abstract describes a qualitative study on students’ use of a computational model of climate change and makes the claim that use of the model enhanced students’ abilities to use NGSS scientific practices. The authors engage with appropriate literature but might also consider connecting their argument to theories of conceptual change (see papers by diSessa, Vosniadou and others) to align discipline-based research on scientific practices with more cognitive perspectives. There seem to be some inconsistencies between the methodology used and the types of claims described in the abstract. For a qualitative study like the one described, more descriptions of the demographics of the student participants and school context would make the results more contextualized so that readers could more appropriately interpret the findings and consider when and how they might provide insights for other contexts so that the generalizability of findings can be better assessed.
​
The authors made causal arguments that student behaviors were different after using the model but did not provide sufficient evidence on what student behaviors were like before using the model nor did they use a control group, which limits the usefulness of the study. Reducing the causal claims would better fit the methodology of the study provide more nuance and accuracy. I know space limitations are a challenge; I would suggest including fewer details about the development of the coding system and measures of interrater reliability. They are important for the study but not necessary to include in an abstract.
Meta-Commentary on Revised Review
Remember, reviews do not need to be perfect, or even necessarily this long. What is most important is for reviewers to engage with the proposal based on its own goals and merits and provide useful feedback for supporting the learning of the author(s). Remember all authors at the conference are students at different phases of their programs, and many proposals are works in progress.