How to review papers

​

Step-by-Step Guide for an In-Depth Medical Journal Review

​

Step 1: Preliminary Reading & Understanding Context

Abstract Analysis: Does the abstract provide a clear, concise summary of the key components? Evaluate whether it captures the research question, methodology, results, and conclusion succinctly. It should offer a snapshot that allows the reader to grasp the study's main findings and significance.

Context within the Field: Assess the relevance of the paper to the broader field of study. Does it address a significant gap or offer a novel approach to existing research? Review the cited literature to determine how well the study builds upon or contrasts with previous work. Consider who the target audience is: clinicians, researchers, or policymakers, and evaluate the potential impact of the findings.

 

Step 2: In-Depth Review of Each Section

Introduction: Framing the Research

• Research Background: Does the introduction provide enough background information? It should summarise key studies and current knowledge in the field, showing why the research is important.

• Clear Hypothesis: The introduction should clearly state the research question or hypothesis. Ensure that the primary and secondary objectives align with the study’s design and outcomes.

Methods: Assessing the methodology

• Study Design: Is the design appropriate for the research question? Consider the type of study—randomised controlled trials, cohort studies, case-control studies, or cross-sectional studies. The design should match the complexity and nature of the hypothesis. 

  • Example: for randomised controlled trials, check whether participants were randomly assigned, if blinding was adequate, and if control groups were used effectively. Assess how well the study design minimises bias

• Study Approach: Examine the study’s approach—Intent-to-Treat (ITT), Per-Protocol (PP), or As-Treated. The choice of approach affects both internal validity (how well the study controls bias) and external validity (how applicable the results are to other settings).

  • Example: ITT preserves randomisation and reduces bias, providing a realistic estimate of treatment effects in real-world conditions. PP, though more focused, may introduce bias if participants drop out or fail to adhere to the intervention.

• Population and Sampling:

  • Was the target population well-defined? Assess if the sample is representative of this population. How were participants recruited? Were inclusion/exclusion criteria clearly justified?

  • Sample Size Calculation: Was a power analysis conducted to determine the adequate sample size? Small sample sizes may result in underpowered studies, leading to inconclusive findings.

  • If multiple groups were used, check whether it was well-matched regarding factors such as age and gender.

• Detailed Methodology: Were the methods described in enough detail to allow replication? Look for clear descriptions of how interventions were administered, how measurements were taken, and what instruments or tools were used.

• Statistical Methods: Were the statistical methods appropriate for the type of data? Confirm whether the statistical analysis was detailed enough to assess the type of data used (e.g., continuous or categorical). Were confidence intervals and p-values provided to demonstrate the precision and significance of the results?

Handling of Missing Data: Were missing data handled properly? Common techniques include imputation, complete case analysis, or sensitivity analysis.

 

An easy framework to remember and use in for the methods section is PICO: 

• Population

  • Who are the patients or the population under study? This includes characteristics such as age, gender, condition, or disease (e.g., adults with hypertension).

• Intervention

  • What is the intervention or treatment being studied? This could be a medication, a surgical procedure, a lifestyle change, or any other form of therapeutic intervention (e.g., a new drug or dietary change).

• Comparison

  • Is there a control or comparison group? This could be a placebo, standard treatment, or another form of intervention for comparison purposes (e.g., comparing a new drug to an existing one or to no treatment).

• Outcome

  • What are the desired or measured outcomes? These could include improvements in health, reduction in symptoms, or other relevant clinical endpoints (e.g., reduction in blood pressure).

​

Step 3: Results Evaluation

• Presentation of Data:

  • Were the results presented clearly and logically? Check for appropriate use of graphs, charts, and tables.

  • Were the results presented in a way that facilitates understanding? Look for relative risk, odds ratios, and number-needed-to-treat measures where applicable.

• Statistical Significance:

  • Were the results statistically significant (typically p-value < 0.05)? However, bear in mind that statistical significance doesn’t always equate to clinical significance—consider the real-world impact of the findings on participants’ lives.

• Consistency with Study Objectives:

  • Were all primary and secondary outcomes addressed? Did the results answer the research question adequately, or were there gaps or unclear outcomes?

​

Step 4: Discussion and Interpretation

• Interpretation of Findings:

Did the authors interpret the results properly, considering the study’s limitations and existing literature? Look for comparisons with other studies and whether the findings were consistent or divergent.

• Addressing Limitations:

Were the study’s limitations clearly acknowledged? Transparency about limitations adds credibility. Consider potential biases (selection bias, measurement bias, or confounding), and assess if the authors discussed how these limitations may have impacted the findings. Were any steps taken to mitigate these limitations?

• Clinical or Practical Implications:

Are the results meaningful for clinical practice or future research? Assess whether the conclusions were overstated, or if they provide a balanced view of how the results might influence medical practice or policy.

• Future Directions:

Did the authors suggest future research avenues? A well-conducted study often opens new questions for further exploration.

• Ethical Considerations:

  • Was the study ethically sound? Check whether the paper mentions approval from an Institutional Review Board (IRB) or Ethics Committee, and if informed consent was obtained from participants.

  • Ensure that potential conflicts of interest were disclosed.

​

Step 5: Conclusion 

• Summarising Key Points: Does the conclusion align with the data presented? It should not introduce new information but rather synthesise the key findings. Were the clinical implications or theoretical contributions clearly articulated? Ensure that the conclusion is appropriately cautious and not overstated.

​

Step 6: References and Citations

• Appropriate Citation of Sources: Are the references up-to-date and relevant to the study? Ensure the authors cited foundational research and recent studies where applicable.

• Balanced Referencing: Did the authors reference a range of studies, including those that might contradict their findings? A balanced review of the literature reflects a critical approach to the field.

​​

Step 7: Final Review

Based on your review, would you recommend acceptance, minor revisions, major revisions, or rejection of the paper? Offer specific suggestions for improvement, whether related to methodology, data presentation, or interpretation.