Now that you have watched the video, read the relevant sections of the papers by Hoferichter & Jentsch (2024) and Kisida et al. (2020) refer to the EEF’s guidance notes, and make a judgement about the level of threat posed by missing data in these experiments. Record your judgement in the security rating template, and note down any supporting information.
Open the accordion below to compare your judgement with that of an experienced rater.
The relevant information can be found on pages 2447 and 2449.
The same information that informed us about attrition can be used here to inform us about missing data. The information on page 2447 tells us that attrition was 51% and that it was higher in the control group, among men, and among participants in the high school track of the programme. We might expect to see information about exactly how many people dropped out in the table on page 2449. But only means and standard deviations are reported, not the n for each group at pre and post-test. So, we cannot tell.
Therefore, this study is considered as having a high risk of bias for Missing Data.
The relevant information can be found on page 6.
“Data were successfully collected from all but one of the school groups that agreed to participate […] To maintain internal validity, this school group and its matched control group were dropped from the study.” (Kisida et al. 2020, p.6).
As we have seen from the section on Attrition, total missing data was low. Also, because the researchers removed from the analyses the control school group matched to the treatment school group that did not return the assessments, there is no differential missing data. In addition, the information in Table 1 on page 6 shows similarity between groups analysed, on measured characteristics.
Therefore, this study is considered as having a low risk of bias for Missing Data.
