Evaluating Evidence: A Critical Appraisal of Study Designs and Risks of Bias

Presentation Description / Summary: What impact can or should a research study have on clinical practice? Recommendations for diagnostic tests and therapeutic interventions are crucially dependent on the available evidence for the condition of interest. Evidence-based medical guidelines are expected to incorporate effect sizes from studies providing high-quality evidence (randomized controlled trials [RCT] for interventions and single-gate designs for diagnostic tests). However, the evidence provided by research studies can vary, depending on the study design and corresponding as well as additional risks of bias (RoB). The strength of recommendations in medical guidelines has to align with the overall quality of the evidence. Using practical examples, we will review study designs for diagnostic test assessment (paired vs. unpaired, randomized vs. non-randomized, single gate vs. two gate) and treatment efficacy testing (randomized vs. non-randomized, controlled vs. uncontrolled, prospective vs. retrospective). We will discuss how different types of bias (e.g., spectrum bias, selection bias) can impact the evidence derived from clinical research studies. We will also highlight the specific items of the Cochran RoB2 algorithm to evaluate RoB in randomized-controlled treatment trials with examples from veterinary medicine.