Assessment Materials Online

The Project

Assessment research has some key messages for education practitioners.  In this series, we speak to experts from academia and practice to identify the central issues and provide some top tips on assessment matters. Links to video references and how to cite the videos are given below.  Please feel free to share the links widely:

With thanks to our experts: Prince Agwu, Randy Bennett, Vikki Boliver, Daisy Christodoulou, Alina von Davier, Chris DeLuca, Victoria Elliott, Paula Lehane, Yasmine El Masri, Rebecca Eynon, Art Graesser, Tom Haladyna, Anil Kanjee, Michelle Meadows, Rochelle Michel, Anne Pinot de Moira, Jennifer Randall, Mary Richardson, Leslie Rutkowski, Robert Sternberg, Gordon Stobart, Zeek Sweiry, Natalie Usher, Dylan Wiliam & Ed Wolfe

Presenters were: Jo-Anne Baird, Stuart Cadwallader, Candace Debnam & Michelle Meadows

1)  Understanding Formative Assessment & Giving Good Feedback

In this film we hear from Professor Gordon Stobart (UCL), Associate Professor Victoria Elliott (Oxford University), Daisy Christodoulou (No More Marking), and Natalie Usher (Oxford University). They discuss the importance of formative assessment and how to give students good feedback. (23 mins)

To cite this video:

Oxford University Centre for Educational Assessment. (2024). Formative Assessment and Good Feedback [Video]. Vimeo. https://vimeo.com/943679625

A follow-up reading for practitioners:

Boyd, E., Green, A., Hopfenbeck, T.N. and Stobart, G. (n.d.) Effective Feedback: the key to successful Assessment for Learning. Oxford University Press.

Some references mentioned in the video:

Johnston, P. (2023). Choice words: How our language affects children’s learning. Routledge. https://doi.org/10.4324/9781032680835

Gamlem, S. & Smith, K. (2013) Student perceptions of classroom feedback. Assessment in Education: Principles, Policy and Practice, 20(2), 150-169 https://doi.org/10.1080/0969594x.2012.749212

Boud, D. & Molloy, E. (2013). Feedback in higher and professional education: understanding it and doing it well. Routledge. https://doi.org/10.4324/9780203074336

Winstone, N. E. & Carless, D. (2018). Designing effective feedback processes in higher education: a learning-focused approach. London: Routledge. https://doi.org/10.4324/9781351115940

Carless, D. & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43 (8), 1315-1325 https://doi.org/10.1080/02602938.2018.1463354

 

2)  Marking and mark schemes

Here, we hear from Associate Professor Michelle Meadows (Oxford University), Dr Ed Wolfe (Pearson) and Anne Pinot de Moira (Consultant). They discuss the importance of marking reliability, problems in producing it and ways of making assessment more reliable, including through good mark scheme design. (35 mins)

To cite this video:

Oxford University Centre for Educational Assessment. (2024). Marking and mark schemes [Video]. Vimeo. https://vimeo.com/949058668

A follow-up reading for practitioners:

Tisi J., Whitehouse, G., Maughan S. and Burdett, N. (2013). A Review of Literature on Marking Reliability Research (Report for Ofqual ). Slough: NFER.

Some references mentioned in the video:

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association https://www.testingstandards.net/open-access-files.html

Rhead, S. & Black, B. (2018). Marking consistency metrics – an update. Ofqual report. https://assets.publishing.service.gov.uk/media/5bfbfd70e5274a0fb775cca3/Marking_consistency_metrics_-_an_update_-_FINAL64492.pdf

Fearnley, A. (2005). An investigation of targeted double marking for GCSE and GCE. AQA paper produced for the National Assessment Agency. https://dera.ioe.ac.uk/id/eprint/9450/1/QCDA104979_an_investigation_of_targeted_double_marking_for_GCSE_and_GCE.pdf

Wolfe, E. W. & Wendler, C. L. (2020). Why should we care about human raters? Applied Measurement in Education, 33(3), 189-190. https://doi.org/10.1080/08957347.2020.1750407

 

3)  Validity of assessment

On arguably the most important topic in assessment, Professors Gordon Stobart (UCL), Dylan Wiliam (UCL), Tom Haladyna (Arizona State University) and Jennifer Randall (University of Michigan) cut through the complex literature on the validity of assessment scores and give us their views on what matters. (39 mins)

To cite this video:

Oxford University Centre for Educational Assessment. (2024). Validity of assessment [Video]. Vimeo. https://vimeo.com/954723837

A follow-up reading for practitioners:

Newton, P. E. (2016). Macro- and micro-validation: Beyond the ‘Five sources’ framework for classifying validation evidence and analysis. Practical Assessment, Research & Evaluation, 21, 12. Retrieved from

Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020). Mining Big Data in Education: Affordances and Challenges. Review of Research in Education, 44(1), 130–160. https://doi.org/10.3102/0091732X20903304

Some references mentioned in the video:

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. https://www.testingstandards.net/open-access-files.html

Messick, S. (1989). Validity. In R. Linn (Ed.) Educational Measurement (3rd ed., pp13-104). Macmillan Publishing Group. https://doi.org/10.1111/j.1745-3992.1995.tb00881.x

Mislevy, R. J., & Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educational measurement: issues and practice, 25(4), 6-20. https://doi.org/10.1111/j.1745-3992.2006.00075.x

Almond, R., Steinberg, L., & Mislevy, R. (2002). Enhancing the design and delivery of assessment systems: A four-process architecture. The Journal of Technology, Learning and Assessment, 1(5) https://ejournals.bc.edu/index.php/jtla/article/view/1671/1509

Solano-Flores, G. (2023). How serious are we about fairness in testing and how far we willing to go? A response to Randall and Bennett with reflections about the standards for educational and psychological testing. Educational Assessment 28(2), 105-117. https://doi.org/10.1080/10627197.2023.2226388

Randall, J., Slomp, D., Poe, M., & Oliveri, M. E. (2022). Disrupting white supremacy in assessment: Towards a justice-oriented, antiracist validity framework. Educational Assessment, 27(2), 170-178. https://doi.org/10.1080/10627197.2022.2042682

Combining different assessment types for greater insights. Blog by Daisy Christodoulou on 28 September 2023 (QR code given). https://substack.nomoremarking.com/p/using-innovative-assessments-to-improve.

 

4)  Digital assessment

What are the promises of digital assessment and have they been delivered? Professors Rebecca Eynon (Oxford University), Alina von Davier (Duolingo), and Art Graesser (University of Memphis) address these issues and discuss the future of digital assessment. (30 mins)

To cite this video:

Oxford University Centre for Educational Assessment. (2024). Digital assessment [Video]. Vimeo. https://vimeo.com/976527870.

A follow-up reading:

Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020). Mining Big Data in Education: Affordances and Challenges. Review of Research in Education, 44(1), 130–160. https://doi.org/10.3102/0091732X20903304

Some references mentioned in the video:

Shute, V. J. (2011) Stealth Assessment in Computer-Based Games to Support Learning in Computer Games and Instruction, Information Age Publishing https://myweb.fsu.edu/vshute/pdf/shute%20pres_h.pdf  

Ofqual (2020) Online and on-screen assessment in high stakes, sessional qualifications: a review into the barriers to great adoption and how these might be overcome. Ofqual. https://assets.publishing.service.gov.uk/media/5fd361b7e90e0766326f7f6e/Barriers_to_online_111220.pdf 

 

5)  Assessment literacy

Understanding assessment, or assessment literacy, can be daunting for practitioners. Professors Christopher DeLuca (Queen’s University, Canada), Anil Kanjee (Tshwane University, South Africa), Leslie Rutkowski (Indiana University, USA) and Daisy Christodoulou (No More Marking, England) discuss what teachers and other stakeholders need to know, the common misconceptions and what could be done to remedy them. (36 mins)

To cite this video:

Oxford University Centre for Educational Assessment. (2024). Assessment Literacy [Video]. Vimeo. https://vimeo.com/991529197.

A follow-up reading:

Ramollo, J. K., & Kanjee, A. (2023). Supporting teachers to develop formative assessment knowledge and skills in no-fee schools.?South African Journal of Childhood Education,?13(1), 1–11. https://doi.org/10.4102/sajce.v13i1.1247

Some references mentioned in the video:

DeLuca, C., LaPointe-McEwan, D. & Luhanga, U. (2016) Teacher assessment literacy: a review of international standards and measures.?Educ Asse Eval Acc?28, 251–272. https://doi.org/10.1007/s11092-015-9233-6

Brookhart, S. M. (2011). Educational Assessment Knowledge and Skills for Teachers.?Educational Measurement, Issues and Practice,?30(1), 3–12. https://doi.org/10.1111/j.1745-3992.2010.00195.x 

Chetty, R., Friedman, J. N., & Rockoff, J. E. (2011).?The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood. National Bureau of Economic Research. https://doi.org/10.3386/w17699 

Roth, B., Becker, N., Romeyke, S., Schäfer, S., Domnick, F., & Spinath, F. M. (2015). Intelligence and school grades: A meta-analysis.?Intelligence (Norwood),?53, 118–137. https://doi.org/10.1016/j.intell.2015.09.002 

Rutkowski, D. & Rutkowski, L. (nd) What teachers can’t and can learn from international assessments. Centre for Educational Measurement, University of Oslo. https://www.uv.uio.no/cemo/english/about/news-events-and-publications/publications/documents/working-paper-david-and-leslie-rutkowski-(2)-june-2016.pdf

 

6)  Item writing

Writing good assessments is one of the hurdles to assessment quality. Emeritus Professor Tom Haladyna (Arizona State University, USA), Zeek Sweiry (Ofqual, UK) and Yasmine El Masri (Ofqual, UK) cover topics such as deciding which assessment format to use, how to write good multiple choice questions and pitfalls to look out for and avoiding bias in assessment design. (40 mins)

To cite this video:

Oxford University Centre for Educational Assessment. (2024). Item Writing [Video]. Vimeo. https://vimeo.com/999088094

Some references mentioned in the video:

Rodriguez, M.C. (2005), Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research. Educational Measurement: Issues and Practice, 24: 3-13.?https://doi.org/10.1111/j.1745-3992.2005.00006.x 

The Gibberish Test (PDF): https://www.southampton.ac.uk/~assets/doc/cite-moocs/FutureLearn%20info/FL%20training%20day%203%20social%20learning%2024%20June%202013/4.-MCQ-Exemplars.pdf 

El Masri, Y., Baird, J.-A., & Graesser, A. (2016).?Language effects in international testing: the case of PISA 2006 science items. https://doi.org/10.1080/0969594x.2016.1218323? 

Holland, P. W., & Wainer, H. (1993).?Differential item functioning. Lawrence Erlbaum Associates. https://doi.org/10.1177/014662169301700113? 

International Test Commission website: https://www.intestcom.org/ 

 

7)  Assessment fairness

In the penultimate video in this series, the topic of fairness in assessment is addressed by Professors Mary Richardson (UCL, UK), Jennifer Randall (University of Michigan, US) and Randy Bennett (ETS, US). They address different ways that people have thought about fairness, especially the turn in the literature from standardisation to personalisation to make assessments fairer to different groups. Dr Paula Lehane (Dublin City University, Ireland) discusses how assessments can be made fairer for pupils with special educational needs and Dr Prince Agwu (University of Dundee, UK) outlines some of his work on cheating. These are live issues in assessment systems internationally. (38 mins)

To cite this video:

Oxford University Centre for Educational Assessment. (2024). Assessment Fairness [Video]. Vimeo. https://vimeo.com/1013528047.

Some references mentioned in the video:

Richardson, Mary. Rebuilding Public Confidence in Educational Assessment. UCL Press, 2022. JSTOR, https://doi.org/10.2307/j.ctv23xmr2z. Accessed 5 Oct. 2024.

Nisbet, I., & Shaw, S. (2020). Is assessment fair?. SAGE Publications Ltd, https://doi.org/10.4135/9781529739480

Randall, J. (2021). “ColorNeutral” Is Not a Thing: Redefining Construct Definition and Representation through a JusticeOriented Critical Antiracist Lens. [Wiley]. https://doi.org/10.1111/emip.12429

Randall, J., Slomp, D., Poe, M., & Oliveri, M. E. (2022). Disrupting White Supremacy in Assessment: Toward a Justice-Oriented, Antiracist Validity Framework. Educational Assessment, 27(2), 170–178. https://doi.org/10.1080/10627197.2022.2042682

Baker-Bell, A. (2019). Dismantling anti-black linguistic racism in English language arts classrooms: Toward an anti-racist black language pedagogy. Theory Into Practice, 59(1), 8–21. https://doi.org/10.1080/00405841.2019.1665415

Bennett, R. E. (2023). Toward a Theory of Socioculturally Responsive Assessment. Educational Assessment, 28(2), 83–104. https://doi.org/10.1080/10627197.2023.2202312

Sullivan, A. (2020, March 10) How do we ‘listen to pupils read’ if they are non-verbal? https://www.youtube.com/watch?v=L6f7gnY8BVQ

Agwu, P., Orjiakor, C. T., Odii, A., Onalu, C., Nzeadibe, C., Roy, P., … Okoye, U. (2023). Leadership for ethical conduct of Senior Secondary School Certificate Examination (SSCE) in Nigeria and the challenge of ‘Miracle Examination Centres.’ Oxford Review of Education, 50(3), 349–365. https://doi.org/10.1080/03054985.2023.2229548

Randall, J. (2023): It Ain’t Near ‘Bout Fair: Re-Envisioning the Bias and Sensitivity Review Process from a Justice-Oriented Antiracist Perspective, Educational Assessment, DOI: 10.1080/10627197.2023.2223924

Strand, S. (2021). Ethnic, socio-economic and sex inequalities in educational achievement at age 16. In The Report of the Commission on Race and Ethnic Disparities: Supporting Research. UK Government. https://www.gov.uk/government/publications/the-report-of-the-commission-on-race-and-ethnic-disparities-supporting-research/ethnic-socio-economic-and-sex-inequalities-in-educational-achievement-at-age-16-by-professor-steve-strand

 

8)  University admissions

In the final video in this series, we look at the role played by assessment and other selection devices in university admissions processes, particularly in the US and the UK. We hear from Professors Robert Sternberg (Cornell University, US), Vikki Boliver (University of Durham, UK), Dr Rochelle Michel (Smarter Balanced, US) and Randy Bennett (ETS, US). Professor Sternberg argues that the processes have remained essentially unchanged for many decades. Professor Bolivar and Dr Rochelle discuss the unfairness of the processes for applicants with social disadvantages and how some universities have adapted their processes. Professor Bennett outlines why we need to think more broadly if we are to select the right talent pool in universities. (28 mins)

To cite this video:

Oxford University Centre for Educational Assessment. (2024). University admissions [Video]. Vimeo. https://vimeo.com/1024379613

Some references mentioned in the video:

Marginson, S. (2016). The worldwide trend to high participation higher education: dynamics of social stratification in inclusive systems. Higher Education, 72(4), 413–434. https://doi.org/10.1007/s10734-016-0016-x 

UNESCO (2022) Higher education global data report: A contribution to the World Higher Education Conference https://www.right-to-education.org/resource/higher-education-global-data-report-contribution-world-higher-education-conference 

Sternberg, R. J. (2010). College admissions for the 21st century (1st ed.). Harvard University Press. https://doi.org/10.4159/9780674058590 

https://www.officeforstudents.org.uk/ 

 

********************************************************************

This video series was funded by Oxford University’s Teaching Development and Enhancement fund. Further members of the project team: Ali Ahmed, Warda Arif and Jonathan Tridgell.

********************************************************************

You might also like to see our bitesize podcasts on standard-setting for GCSEs.

Project Details

Start date: 2024
End date: 2024
Theme: Centre for Educational Assessment; Theme 2: Policy, Economy and Society