The Quantitative Methods Hub organises termly seminar programmes that build on the tradition of: (1) addressing substantive educational research questions with quantitative methods, and (2) demonstrating the application and use of a wide range of different quantitative methods. Presentations are given by both academics and higher degree students within friendly lunch-time seminars that are open to all.
Hybrid-presentations will take place on Mondays at 1pm-2.15pm in Seminar Room A (except for 28 October in Seminar Room D). Our speakers will present in person or remotely. It is also possible to take part virtually by signing up to the zoom-meeting.
All Welcome
Convenor in HT 2024, Lars-Erik Malmberg, Professor
How is students’ academic effort associated with mathematics achievement? A multi-informant investigation employing complementary variable- and person-centred analyses
Dr. Robin Nagy, University of New South Wales (Australia)
14 October, 1pm-2.15pm
For in-person attendance: Seminar Room A, 15 Norham Gardens, Oxford
Join Teams for online attendance
Abstract
Students’ effort is considered a major factor in their educational development, but scant research has purposefully focused on it and its links with academic achievement. The aims of this research were to examine how student- and teacher-rated effort uniquely predict achievement gains at student and classroom levels (controlling for student covariates, together with prior effort and achievement). Data were collected from 1,548 secondary school students and 72 teachers in 114 mathematics classes from Years 7-10 in nine Australian independent schools. Analyses included multilevel confirmatory factor analysis, multilevel path analysis, and follow-up latent profile analysis. Findings demonstrated that across the whole sample, teacher-rated effort uniquely predicted students’ academic achievement gains at both individual student and classroom levels, but student-rated effort did not. Follow-up person-centred analysis revealed profiles for whom self-rated effort did predict growth in academic achievement. Thus, employing complementary variable- and person-centred analyses, this investigation was able to highlight specific and unique ways in which effort is associated with students’ achievement gains.
Rethinking belonging: Measurement challenges across educational and professional contexts
Dr. Ellen L. Usher, PhD
Scientific Director, Education Science, Professor of Medical Education, Mayo Clinic College of Medicine and Science
21 October, 1pm-2.15pm
For in-person attendance: Seminar Room A, 15 Norham Gardens, Oxford
Join Teams for online attendance
Abstract
Sense of belonging is a fundamental human need that influences well-being, success, and retention in educational and professional settings, making accurate measurement essential. This talk examines the complexities of measuring belonging through two studies. The first is a randomized controlled trial involving first-year college students from underrepresented groups (N = 1,178), where multi-group structural equation modeling assessed the impact of a social belonging intervention on academic outcomes. While some positive effects were found, results varied, suggesting differential impacts on first-generation and racial/ethnic minority students. The second study is a systematic review (52 studies) assessing the conceptual precision, theoretical grounding, and psychometric rigor of sense of belonging measures used with physicians. Findings revealed significant variability in how belonging is conceptualized and measured. Both studies highlight the importance of attending to item-level details for developing more accurate assessments, offering key insights for research and practice in educational and healthcare settings.
Exploring the dynamics of motivation and anxiety in English speaking: A multivariate longitudinal approach using Latent Growth Class Models (LGMC)
Hao (Chuck) Wu, University of Oxford
28 October, 1pm-2.15pm
For in-person attendance: Seminar Room D, 15 Norham Gardens, Oxford
Join Teams for online attendance
Abstract
For decades, oral presentations have become a common method of assessment in language learning classrooms. Nonetheless, anxiety is a persistent negative feeling pervasive in EFL learners. To tackle this concern, this mixed-methods longitudinal study investigated the growth trajectories of 171 Chinese EFL learners’ L2 motivation and anxiety in four consecutive oral presentations. Results show that: (1) As the number of EFL learners giving oral presentations increased, the L2 motivation levels increased, and the anxiety levels decreased. (2) Those who were initially more anxious about giving oral presentations had higher decrease rates during the four oral presentations. (3) There was co-development but inverse relationships between ideal L2 self and anxiety and between ought-to L2 self and anxiety. These findings suggest that students’ perceptions of L2 motivation interact with anxiety levels over time but in a sophisticated fashion. Finally, pedagogical implications for EFL oral presentation instruction are provided.
Physiological processes as a “component” of achievement emotions? – A psychometric network approach
Miriam Wünsch, LMU Munich, Germany
4 November, 1pm-2.15pm
For in-person attendance: Seminar Room A, 15 Norham Gardens, Oxford
Join Teams for online attendance
Abstract
The definition of achievement emotions poses that psychophysiological processes constitute one component of the emotional experience in situations of learning and assessment. Therefore, research increasingly employs physiological measures in addition so self-report in the assessment of achievement emotions (Pekrun et al., 2023). Despite this increasing use, however, we are still lacking an in-depth understanding of how physiological measures relate to the individual’s subjective experience of achievement emotions and how such information can be used to supplement self-report in research and practice.
The present research therefore explores the relation between psychophysiological measures (electrodermal activity and heart rate) and subjectively experienced, self-reported achievement emotions on both an inter- and an intraindividual level using a psychometric network approach (e.g. Epskamp, 2020). In a laboratory study, we obtained repeated physiological and self-report measures from N = 159 participants during an achievement task. The results underline the importance of a distinction between inter- and intraindividual relations and provide food for thought on how physiological processes constitute a “component” of achievement emotions and their application in research and practice.
Finding the key social-emotional skills for learning and wellbeing: A multi-method synthesis
Dr. Xin Tang, Shanghai Jiao Tong University, China
11 November, 1pm-2.15pm
For in-person attendance: Seminar Room A, 15 Norham Gardens, Oxford
Join Teams for online attendance
Abstract
Social-emotional skills, the skills about managing relationships, emotions, and tasks, have been highlighted in the research and practices. Given its broad scope, multiple frameworks and indicators have been proposed, causing inconsistency in findings. In this talk, I will introduce a series of studies we conducted in order to find out the key social-emotional skills for students’ learning and wellbeing. Throughout those studies, we have used relative weights analysis, dominance analysis, machine learning approaches, cross-cultural/county comparisons, with OECD social-emotional skill survey data. The findings suggested that persistence, curiosity, self-control tends to be the most important skill for achievement, whereas optimism, energy, stress resistance for wellbeing. The implications of the findings (e.g., targeted intervention) and our future directions will be discussed, particularly on utilizing the power of Generative Artificial Intelligence tools.
Investigation of Rater Effects using Graph Theory and Exponential Random Graph Models: examples from high-stakes examination data
Dr. Iasonas Lamprianou, University of Cyprus, Cyprus
18 November, 1pm-2.15pm
For in-person attendance: Seminar Room A, 15 Norham Gardens, Oxford
Join Teams for online attendance
Abstract
This presentation investigates rater effects in high-stakes examinations using Graph Theory (GT) and Exponential Random Graph Models (ERGM). Rater effects, such as biases and inconsistencies, are common challenges in the scoring process, impacting the fairness and validity of assessments. GT and ERGM offer powerful tools to visualize complex rater interactions and reveal hidden patterns that traditional reliability indices, such as Krippendorff’s alpha or Fleiss’s kappa, may not capture.
Using real-world examination data, we demonstrate how graph-based models provide nuanced insights into rater behavior, allowing for the identification of outlier raters and systematic biases. This approach is not intended to replace established methods (e.g., Rasch models) but rather to complement them.
Our empirical analysis shows that GT indices and Rasch model estimates are strongly correlated, highlighting their complementarity. Additionally, certain GT indices can be mathematically linked to Rasch model estimates through the pairwise estimation method, further strengthening their theoretical connection.
Comparative effects of college preparation and career preparation on educational attainment in Massachusetts, USA
Pierre Lucien, University of Oxford
25 November, 1pm-2.15pm
For in-person attendance: Seminar Room A, 15 Norham Gardens, Oxford
Join Teams for online attendance
Abstract
Literature suggests that promoting equitable educational attainment may be one of the effective ways to address the historical inequities in American society, due to its positive association with an array of desirable outcomes. College preparation and career preparation in high school have both been shown to have positive effects on college enrollment, persistence, and completion, respectively. In Fall 2018, Massachusetts launched Early College and Innovation Career Pathways—a college preparation intervention and a career preparation intervention, respectively—to promote educational attainment and career success for all. This study uses generalized propensity score (GPS) inverse probability of treatment weighting (IPTW) to gauge the comparative effects of participating in either program on college readiness, access, and success. Given that an emergent share of the literature and the United States government define college and career readiness collectively, findings and discussions of this paper explore variations in outcomes by intervention and by racial and socio-economic groups to inform policy and practice towards promoting educational attainment for all in Massachusetts high schools.
Using bifactor nonlinear CFA to investigate cognitive differentiation in general and specific cognitive abilities
Dr. Lisa Bardach, Justus-Liebig University Giessen, Germany
2 December, 1pm-2.15pm
For in-person attendance: Seminar Room A, 15 Norham Gardens, Oxford
Join Teams for online attendance
Abstract
Understanding how the structure of cognitive abilities changes depending on age and ability (age differentiation and ability differentiation) has critical implications for cognitive ability assessments and cognitive developmental theories. Most differentiation research has focused on general intelligence; however, as children increasingly invest in specific domains and school-taught subjects, we argue these investments should rather affect their domain-specific ability structures. This study capitalized on a representative longitudinal sample of 17,979 children from the U.S. who were assessed in mathematics, reading, science, working memory, and cognitive flexibility at seven waves (from Kindergarten entry to fifth grade). We applied longitudinal non-linear confirmatory factor analysis with Montecarlo integration to analyze the data. The non-linear model was set up as a bifactor model in that all tests were regressed on the g factor, but each test score also loaded on one of five orthogonal content-specific ability factors. The results revealed that loadings on a general intelligence factor remained similar but domain-specific factor loadings increased on most tasks during the kindergarten–fourth-grade period before dropping in fifth grade. Hence, age and ability differentiation are conceptually distinct, with the former pertaining to specific abilities and the latter to general intelligence. Further, children compensated for lower general intelligence with higher levels of domain-specific abilities. Our findings were robust to the addition of autoregressive paths, which likely contribute to the stability among achievement tests in the early school years. Overall, our study motivates a more nuanced understanding of children’s cognitive development.
