Oxford University Centre for Educational Assessment

OUCEA undertakes quantitative studies in two main research areas:

1) International assessments

Dr. Therese Hopfenbeck and Dr. Daniel Caro have conducted research with data from international large-scale assessment studies: the Programme for International Student Assessment (PISA), the Trends in International Mathematics and Science Study (TIMSS), and the Progress in International Reading Literacy Study (PIRLS). In recent collaborative work, Dr. Hopfenbeck has applied Item Response Theory (IRT) techniques to investigate the validity of PISA learning strategies scales. Her findings indicate that students who perform poorly in PISA are less likely to provide valid responses to the learning strategies items. Also in collaborative work, Dr. Caro has proposed an analytical framework for the study of educational inequalities with international assessment data. His work shows how to calculate a socioeconomic status (SES) index with international assessment data and how to evaluate critical hypotheses for inequality research with multilevel models. He has also extended the SES framework by applying exploratory structural equation modelling on cultural, social, and economic capital indicators well-grounded in theory.

2) Marking reliability

Professor Jo-Anne Baird has conducted research on marking reliability. In a recent collaborative investigation commissioned by the Office of Qualifications and Examinations Regulation (Ofqual), she has compared the contributions of generalizability theory, Rasch modelling, and multilevel modelling to the exploration of rater effects on examination reliability. In related work, Professor Baird has applied multilevel models to examine drift in rater severity effects over time, rater central tendency effects, and differences in rater effects by previous rating experience. Her findings indicate no drift in rater effects, central tendency to raters’ scoring, and significant variation of severity effects over time.

Further information on quantitative research being conducted at OUCEA can be found here.

Related publications

  • Baird, J., Hayes, M., Johnson, R., Johnson, S. & Lamprianou, I. (2013) Marker effects and examination reliability: A comparative exploration from the perspectives of generalizability theory, Rasch modelling and multilevel modelling. Report commissioned by the Office of Qualifications and Examinations Regulation. Ofqual/13/5261.
  • Caro, D. H., Sandoval-Hernandez, A., & Lüdtke, O. (2013) Cultural, Social and Economic Capital Constructs in International Assessments: An Evaluation Using Exploratory Structural Equation Modeling. School Effectiveness and School Improvement. http://dx.doi.org/10.1080/09243453.2013.812568.
  • Caro, D.H. & Lenkeit, J. (2012) An analytical approach to study educational inequalities: 10 hypothesis tests in PIRLS 2006, International Journal of Research and Method in Education, 35 (1), 3-30.
  • Leckie, G. & Baird, J. (2011) Rater Effects on Essay Scoring: A Multilevel Analysis of Severity Drift, Central Tendency, and Rater Experience. Journal of Educational Measurement, 48 (4), 399-418.
  • Hopfenbeck, T.N. & Maul, A. (2011). Examining Evidence for the Validity of PISA Learning Strategy Scales Based on Student Response Processes. International Journal of Testing, 11(2), 95-121.