p 7 and 8
https://ies.ed.gov/ncee/wwc/Docs/ReferenceResources/wwc_dp_protocol_v3.0.pdf
Baseline Equivalence
For the Dropout Prevention topic area, RCTs with high attrition or QED studies must demonstrate
equivalence of the intervention and comparison groups before the intervention. The onus for
demonstrating equivalence in these studies rests with the authors. Sufficient reporting of preintervention data should be included in the study report (or obtained from the study authors) to allow
the review team to draw conclusions about the equivalence of the intervention and comparison
groups.
Important pre-intervention characteristics can include measures that are highly related to the
outcome measure(s). Other important pre-intervention characteristics can include outcome(s)
measured prior to the intervention. However, when the unit of analysis is the student, many
outcome(s) of interest for this review, such as dropout status or high school graduation status, are
not defined or are not informative when measured prior to the intervention.
Studies for which the unit of analysis is the student must show that the groups are equivalent in
terms of race/ethnicity and sex. Additionally, they must demonstrate equivalence of the research
groups in at least one measure of degree of disadvantage including:
• Free and reduced-price lunch status, poverty status, family income
• Being from a single-parent family
• Parent’s education
• Immigrant or English learner (EL) status
Special education or disability status
• Teen parent status
Finally, these studies must demonstrate equivalence of the research groups in at least one
measure of academic performance. These measures can include:
• Standardized test scores
• Whether behind in grade level (could be measured by age among students in the same
grade)
• Frequency of behavior or discipline incidents in school
• Rate of school attendance
• GPA
Because these measures of academic performance are not defined or typically not available for
students who have dropped out of school, studies of interventions for students who have dropped
out may demonstrate equivalence based on the proportion of students in each research group who
are dropouts.
Studies for which the unit of assignment is the school must show that the groups are equivalent
in terms of outcome(s) measured prior to the intervention. Additionally, they must demonstrate
equivalence in race/ethnicity and at least one measure of degree of disadvantage or academic
performance denoted above in bold text.
Groups are considered equivalent if the reported differences in pre-intervention data are less than
or equal to one-quarter of the pooled standard deviation in the sample, regardless of statistical
significance. However, if differences are greater than 0.05 standard deviations and less than or
equal to one-quarter of the pooled standard deviation in the sample, the analysis must control for
the pre-intervention outcome measure(s) on which the groups differ. If pre-intervention
differences are greater than 0.25 for any of the outcomes in the same domain, the study does not
meet standards. In addition, if there is evidence that comparison groups were drawn from very
different settings (such as rural vs. urban), the lead methodologist may decide that the
environments are too dissimilar to provide an adequate comparison.
