Department of Education

Viewing archives for Quantitative Methods Hub

Please register ahead of the webinar at this link.

Dr Matt Dickson

We compare estimates of the effects of education on health and health behaviour using two distinct natural experiments in the UK Biobank data. One is based on a widely used policy reform while the other, known as Mendelian randomization (MR), uses genetic variation. The policy reform is the raising of the minimum school leaving age (RoSLA) from 15 to 16 which took place in the UK in 1972.

MR exploits germline genetic variation that associates with educational attainment and is a strategy widely used in epidemiology and clinical sciences. Under the assumption of monotonicity, these approaches identify distinct local average treatment effects (LATEs), with potentially different sets of compliers. The RoSLA affected the amount of education for those at the lower end of the education distribution whereas MR affects individuals across the entire distribution.

We find that estimates using each approach are remarkably congruent for a wide range of health outcomes. Effect sizes of additional years of education thus seem to be similar across the distribution. Our study highlights the usefulness of MR as a source of instrumental variation in education.

Please register ahead of the webinar at this link.

Dr Sandra Mathers

Theory suggests that effective real-time decision-making in classrooms requires teachers to have flexible access to rich and well-organised knowledge of effective teaching practices. Yet prior research on the role and importance of procedural knowledge has been equivocal. This exploratory study used a new video measure of procedural knowledge to examine relationships with observed classroom quality, and establish which opportunities to learn (qualifications, professional development, classroom experience) predict greater knowledge. It focused on preschool teachers’ knowledge of oral language pedagogy, on the basis that early language provides the foundation for children’s later learning. The sample comprised 104 teachers participating in a wider RCT, designed to evaluate a professional development intervention. Teachers were shown two short videos of classroom interactions and asked to identify instances of effective practice. Responses were coded to capture three facets: perceiving (the ability to identify salient language-supporting strategies); naming (the use of specific professional vocabulary to describe interactions); and interpreting (the ability to interpret the interactions observed). The three facets could be empirically distinguished. Explicit and higher-order procedural knowledge (naming, interpreting) most strongly predicted classroom quality. Formal learning opportunities were stronger predictors of procedural knowledge than classroom experience. Intervention effects on classroom quality were mediated by knowledge. Implications for workforce development are discussed.

Please register ahead of the webinar at this link.

Professor Alice Sullivan

This paper examines the relationship between parents’ and children’s language skills for a nationally representative birth cohort born in the United Kingdom-the Millennium Cohort Study (MCS). We investigate both socioeconomic and ethnic differentials in children’s vocabulary scores and the role of differences in parents’ vocabulary scores in accounting for these. We find large vocabulary gaps between highly educated and less educated parents, and between ethnic groups. Nevertheless, socioeconomic and ethnic gaps in vocabulary scores are far wider among the parents than among their children. Parental vocabulary is a powerful mediator of inequalities in offspring’s vocabulary scores at age 14, and also a powerful driver of change in language skills between the ages of five and 14. Once we account for parental vocabulary, no ethnic minority group of young people has a negative “vocabulary gap” compared to whites.

Please register ahead of the webinar at this link.

Dr Jo Blanden

School closures have been one of the most dramatic consequences of the COVID-19 pandemic on society. Concerns about the impact of school closures on children’s learning were raised early on in the pandemic and work continues to mitigate lost learning. There is also widespread concern about the detrimental impact of the pandemic on children’s mental wellbeing, but there are likely to be a number of mechanisms at work here, including parents’ employment situation, anxiety about relatives’ health and social isolation. We specifically examine the role of school closures in England on the emotional and behavioural wellbeing of children aged 5-11, as measured by the Strengths and Difficulties Questionnaire (SDQ) in the UK Household Longitudinal Study.

 

Logos for the Bell Foundation, UNBOUND and the Department of Education

Published today and written by Professor Steve Strand and Dr Ariel Lindorff, Department of Education, the report titled ‘English as an Additional language, Proficiency in English and rate of progression: Pupil, school and LA variation’ can be accessed here:  https://bit.ly/3vasAT6

Over the past year most children have experienced some degree of learning loss, and that loss is significant for pupils who speak English as an Additional Language (EAL) who may have also experienced language learning loss during this period. Therefore, as pupils return to school in England, it is essential that schools and policy-makers apply the key insights on EAL learner attainment from a five-year research programme in order to provide appropriate catch-up support and resources to enable EAL learners to mitigate the language and learning lost during school closures.

The research programme investigated the relationships between English as an Additional Language, Proficiency in English and the educational achievement of EAL learners at school. The fourth and final report¹ by the University of Oxford, funded by The Bell Foundation and Unbound Philanthropy, has been published today. This research series has identified that it takes learners more than six years to progress from the lowest to the highest levels of English language proficiency, it has also shown that it is only at the highest levels of proficiency that learners are able to fully access the curriculum and therefore achieve their academic potential. This evidence demonstrates that as part of the Government’s catch-up plans, it is important to include and focus catch-up support and resources on learners who use EAL as it may take them longer to regain both the language and learning lost during school closures.

As there are almost 1.6 million learners recorded as using ‘EAL’ in England which constitutes just under one-in-five (19.5%) of all pupils aged 5-16, it is likely that many teachers will be working in, or have experience of, teaching multilingual classes. ‘EAL’ is used to refer to any pupil who has been “exposed to a language at home that is known or believed to be other than English. It is not a measure of English language proficiency or a good proxy for recent immigration.” (Department for Education (DfE), 2020).

However, learners using EAL are a diverse and heterogeneous group with country of birth, time of arrival in the school system, first language spoken, previous education and background all contributing to that diversity and their likely educational achievement². For example, an EAL learner could be a second or third generation ethnic minority student who speaks English fluently but has a second language as part of their cultural heritage, alongside a new arrival to the English school system who is new to English.

As the report highlights, the term “EAL is too blunt a tool to understand pupils’ language learning needs”, it does not provide any information on a student’s likelihood to succeed academically or what targeted support they will need. This is what the research series set out to discover so that schools have the information they need in order to target their support in the right place and in the right way.

The four reports in the research programme provide robust evidence on why Proficiency in English is the single most important factor in determining a pupil’s likelihood to fulfil their academic potential and why assessment of Proficiency in English, as well as how that informs the support a learner will need, is so important:

  • Proficiency in English is central to understanding achievement and levels of need among pupils who use EAL. It can explain 22% of the variation in EAL pupils’ achievement, compared with the typical 3-4% that could be statistically explained using gender, free school meal status and ethnicity³.
  • Where Proficiency in English is measured⁴, for example in the devolved nations, it can be seen that EAL pupils who are starting to acquire proficiency score below the national average, those who are at mid-point proficiency are very close to the national average and those with the highest levels of proficiency typically score higher than First Language English (FLE) speakers³. This shows that at high levels of proficiency there are positive associations between speaking more than one language and achievement. What is typically a barrier to achievement is low proficiency in the language of instruction at school. Pupils need to be supported so that they can acquire the proficiency that they need to access the curriculum and successfully demonstrate learning.
  • For two-thirds of pupils who are new to English at the start of Reception it takes more than six years to progress to the highest levels of proficiency⁵. EAL pupils who entered school in later year groups are found to make the same rate of progress in the same amount of time as those joining in Reception¹. This means that pupils entering the English school system as new to English, at whatever age, may need at least six years of support before they achieve the highest levels of Proficiency in English.
  • The judgement of Proficiency in English, and the time to progress across levels, is strongly influenced by the individual making the assessment¹, rather than necessarily reflecting a difference in the pupils themselves. Accurate, consistent assessment is necessary in order to ensure that pupils are given appropriate support to progress in Proficiency in English and gain access to the curriculum.
  • Although schools in England are not required to assess and record Proficiency in English, it is interesting to note that in Wales, where proficiency is measured, it was found that pupils assessed as having the highest level of proficiency in Year 6 were much more likely than others to be assessed as monolingual English/Welsh¹ in Year 7. This suggests that there may be some association of the term “EAL” with “needing language support” rather than “exposure to another language at home” (DfE). This indicates a need for training in how to conduct Proficiency in English assessment and provides further evidence for the need to record Proficiency in English in addition to an EAL flag, as the two are independent dimensions.
  • There is no evidence that being in a school with a high proportion of pupils acquiring Proficiency in English has any negative association with achievement for other pupils in the school, either overall or specifically for pupils with First Language English.

“As schools re-open to all pupils in England, the findings of this research programme are particularly timely, as they help schools to understand why it is essential, as well as how, to support EAL learners to mitigate the language and learning loss they may have experienced during school closures. As the research shows, the term ‘EAL’ is not a sufficient measure of the need for language development support. Schools need to be able to accurately establish a learner’s current level of Proficiency in English, then determine the type of support required to meet an EAL learner’s language and learning needs and set tailored support strategies which will enable learners to regain and/or improve their English language skills. As the research has demonstrated, it is only when learners achieve the highest levels of proficiency that they are able to fulfil their academic potential. This is why it is vital that the Department for Education does not overlook this group of learners in its plans for catch-up programmes, because EAL learners have not only lost learning, they have also lost language learning during school closures and as a result, may take longer to return to their previous attainment levels. The Bell Foundation is asking the Department for Education to introduce a requirement for schools to assess their learners’ Proficiency in English levels for their internal monitoring purposes and we encourage schools to robustly assess Proficiency in English as it is integral to student achievement.” Diana Sutton, The Bell Foundation

Professor Steve Strand, Department of Education, University of Oxford, lead research author goes further, “Our research programme provides evidence on what influences an EAL learner’s likelihood to succeed academically, something that is both important and relevant following a year of disrupted education caused by the pandemic. The empirical evidence shows that ‘English as an Additional Language’ is a poor indicator of pupils’ likely level of educational achievement, instead, it is their Proficiency in English that is central to understanding achievement and levels of need. It also shows that being bilingual can have positive associations with achievement as pupils at the highest levels of English proficiency typically have higher educational achievement than their monolingual peers. The converse of this, is that those at the lowest levels of proficiency will underperform, which, as the evidence shows, will have a detrimental impact on their life chances. This fact is often overlooked due to misleading average exam and assessment scores. As a result of these findings it is clear that there is a need for a universally understood and applied definition of what constitutes the different levels of proficiency. As our evidence shows, on-going, consistent and independently moderated and reliable use of teacher-assessed Proficiency in English would benefit all learners who use EAL as it informs the specific support a learner will need to fulfil their academic potential.”

To conclude, drawing on the commissioned research, as evidenced above, The Bell Foundation has identified three practical strategies that teachers can use to support learners who use English as an Additional Language to succeed. This is particularly important after prolonged school closures, or absence from school, as it will help to mitigate any language and learning loss experienced during those periods:

  1. Use initial and on-going assessment of both language proficiency and cognitive skills to establish the level of need among individual learners. Use evidence-informed tools and resources, for example The Bell Foundation’s award-winning EAL Assessment Framework and digital Tracker, to undertake robust and consistent assessment, moderation and recording of Proficiency in English levels.
  2. Set tailored targets and support strategies for teaching and learning to support learners to progress to higher levels of proficiency. Through achieving academic linguistic proficiency⁶ learners will be able to fully participate in school and access the curriculum and, as a result, to fulfil their academic potential.
  3. Where possible, engage with training and CPD on how to assess Proficiency in English to ensure consistency of measurement and how to support learners at each level of proficiency.

¹ The report analysed nine-years of anonymised Proficiency in English data from the Welsh Pupil Level Annual School Census (which the research team established to be equally relevant in England). Strand, S. & Lindorff, A. (2021) ‘English as an Additional Language, Proficiency in English and rate of progression: Pupil, school and LA variation’, University of Oxford, The Bell Foundation and Unbound Philanthropy https://bit.ly/3vasAT6

² Hutchinson, J. (2018) ‘Educational Outcomes of Children with English as an Additional Language’, Education Policy Institute, The Bell Foundation and Unbound Philanthropy, http://bit.ly/EALoutcomes

³ Strand, S. & Hessel, A. (2018) ‘English as an Additional Language, Proficiency in English and pupils’ educational achievement: An analysis of Local Authority data’, University of Oxford, The Bell Foundation and Unbound Philanthropy, http://bit.ly/EAL-PIE18

⁴ The Department for Education in England introduced a five-point Proficiency in English scale, from A ‘New to English’ to E ‘Fluent’, to assess the English language proficiency of learners using EAL in the 2017 School Census. This meant that schools began to be aware of the importance of proficiency. Although the requirement was removed in 2018, the Department did issue an advisory note in 2019 (Department for Education (June 2019) ‘Attainment of pupils with English as an additional language’) which acknowledged the diversity of this cohort of learners. The Department for Education’s five-point Proficiency in English Scale (now withdrawn) ranged from ‘A’ New to English, through ‘B’ Early Acquisition, ‘C’ Developing Competence, ‘D’ Competent, to ‘E’ Fluent

⁵ Strand, S. & Lindorff, A. (2020) ‘English as an Additional Language: Proficiency in English, educational achievement and rate of progression in English language learning’, University of Oxford, The Bell Foundation and Unbound Philanthropy http://bit.ly/EAL-PIE20

⁶ Academic linguistic proficiency refers to mastery of abstract and formal communication relating to specific subject areas which contributes to educational success. This includes listening, speaking, reading, and writing about subject area content material, as well as skills such as comparing, classifying, synthesizing, evaluating, and inferring. (Cummins, 1981, 2000)

‘International evidence for primary and secondary schools suggests an extended period of remote learning is likely to result in poorer educational outcomes, particularly for early-years children, students from low socioeconomic backgrounds, those with English as a second language, those with special learning needs, and students who are generally less engaged with school, though data is limited and varied.’ Scientific Pandemic Influenza Group on Behaviours (SPI-B) and the Department for Education: Benefits of remaining in education: Evidence and considerations, November 2020: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/935192/spi-b-dfe-benefits-remaining-education-s0861-041120.pdf


How to cite this report:

Strand, S. & Lindorff, A. (2021) ‘English as an Additional Language, Proficiency in English and rate of progression: Pupil, school and LA variation’, University of Oxford, The Bell Foundation and Unbound Philanthropy


 

Please register ahead of the webinar at this link.

University students’ beliefs about writing have been shown to impact writing outcomes. However, writing beliefs have not been explicitly examined in populations of students with dyslexia.

The aim of this survey study was to investigate whether writing beliefs could be measured in the same way across UK university students with and without dyslexia. A newly developed writing beliefs questionnaire (Galbraith and Baaijen, in preparation) was completed by UK university students with and without dyslexia (N=562). Exploratory structural equation modelling (ESEM) was used to test whether the questionnaire had a 5-factor structure, representing different types of writing beliefs. Invariance testing was used to see whether this structure held across the two groups. Follow up comparisons of latent means were conducted to see whether students with and without dyslexia had different strengths of each type of writing belief.

In this session, Sophie will discuss the ESEM framework, and how it was used within the study. She will also discuss the study’s results.

Please register ahead of the webinar at this link.

The classroom is a highly social and emotional setting, and teachers’ emotional experiences in class have profound consequences for teacher well-being and student motivation and achievement. However, it is unclear how teacher emotions emerge during the lesson. The main aim of the In DEPTh-project (Donker, Mainhard, & Van Gog) was to investigate how teachers’ dynamic experiences during real-life lessons were associated with their emotional outcomes. We explored the potential of different ways of analyzing and combining observed interpersonal teacher behavior (in terms of agency and communion; using Continuous Assessment of Interpersonal Dynamics/joystick method) and physiological time-series data (in terms of heart rate; using the VU-AMS) for understanding between-teacher differences in emotions and well-being. During the seminar, Monika will discuss both measurement methods, the analyses, as well as give a short overview of the results of the In DEPTh-project.

Please register ahead of the webinar at this link.

Teachers’ motivational beliefs—i.e., teachers’ self-efficacy and felt responsibility for educational outcomes—can shape their professional decision-making and approach to teaching. However, theorized associations with student outcomes remain elusive. In a multi-level analysis with 96 Swiss vocational teachers and their 1,300 students, we examined the interrelations between teachers’ self-efficacy, responsibility, teacher- and student-reported autonomy-supportive versus psychologically controlling teaching, and student motivation (emotional, behavioral, and cognitive engagement). Teachers’ motivational beliefs predicted their endorsement of autonomy-supportive teaching, which in turn predicted student-reported autonomy support. Student-reported autonomy support was a powerful predictor of student engagement. Teachers’ motivational beliefs did not predict student-rated instructional practices and engagement directly, and indirect effects via teacher- and student-rated autonomy support were small. Teacher- and student-reported controlling practices were not significantly correlated. The degree of (mis)alignment of teacher- and student-reported instructional practices is a key ingredient in understanding the often missing link between teacher motivation and student outcomes.

 

Please register ahead of the webinar at this link.

One potential reason that motivational research around education has never picked up in Africa may be that locally grounded measurement scales hardly exist. This presentation is about the design, implementation, and results of an attempt to develop and validate a motivational self-report instrument for the Rwandan context. Within the first study, motivational drivers of educational engagement of Rwandan students were scoped out through qualitative methods. Results were also used to develop self-report measures. Within the second study, the internal structure of the motivational scales was tested through exploratory factor analysis. An emerging 4-factor solution of self-beliefs, utility, costs, and unimportance fitted the data within both the maths and language domain. Within the third study, confirmatory factor analyses supported the four-factor structure of the scales, and suggested measurement invariance especially across gender, age, and academic domain. Finally, the motivational factor structure is compared especially with the ones from the Western-borne literature.