Resources on the importance of marking reliability, problems in producing it and ways of making assessment more reliable, including through good mark scheme design.
In this film, we hear from Associate Professor Michelle Meadows (Oxford University), Dr Ed Wolfe (Pearson) and Anne Pinot de Moira (Consultant). They discuss the importance of marking reliability, problems in producing it and ways of making assessment more reliable, including through good mark scheme design.
To cite this video:
Oxford University Centre for Educational Assessment. (2024). Marking and mark schemes [Video]. Vimeo. https://vimeo.com/949058668
A follow-up reading for practitioners:
Tisi J., Whitehouse, G., Maughan S. and Burdett, N. (2013). A Review of Literature on Marking Reliability Research (Report for Ofqual). Slough: NFER.
Some references mentioned in the video:
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association https://www.testingstandards.net/open-access-files.html
- Rhead, S. & Black, B. (2018). Marking consistency metrics – an update. Ofqual report.
https://assets.publishing.service.gov.uk/media/5bfbfd70e5274a0fb775cca3/Marking_consistency_metrics_-_an_update_-_FINAL64492.pdf - Fearnley, A. (2005). An investigation of targeted double marking for GCSE and GCE. AQA paper produced for the National Assessment Agency. https://dera.ioe.ac.uk/id/eprint/9450/1/QCDA104979_an_investigation_of_targeted_double_marking_for_GCSE_and_GCE.pdf
- Wolfe, E. W. & Wendler, C. L. (2020). Why should we care about human raters? Applied
Measurement in Education, 33(3), 189-190. https://doi.org/10.1080/08957347.2020.1750407
