Two Studies of the Rating Process

Due to the subjective nature of human decision in the context of the marking of constructed response assessment items, those who administer marking projects are interested in identifying best practices with the goal of making the training and marking processes quicker without reducing marking accuracy.

In this presentation, I summarize two studies designed to inform those decisions. In the first study, I conducted an experimental study in which markers were assigned to one of three groups who received different amounts of marking practice during marker training, and I measured the impact of these varying levels of practice on marking accuracy and speed. In the second study, I conducted secondary data analysis in which I regressed textual features of essays (e.g., development, organization) onto measures of the difficulty markers had assigning accurate scores (i.e., a measure of the average distance of marks from an expert consensus score).

ABOUT THE DEPARTMENT OF EDUCATION

In 2019, the University of Oxford’s Department of Education celebrates the 100th year since the passing of a statute creating what was known in 1919 as the University Department for the Training of Teachers. To celebrate our centenary a year-long series of activities will be delivered to address some of the department’s top initiatives for 2019, answer some of the big questions facing education today and to reveal the advancements the department has made to the study of and research in the field of education. Join us as we mark our 100th year and discover more about our anniversary here.

To receive more event details from the Department of Education, join our mailing list.