Is my bias
your bias?
We investigated bias in academic outcomes for UK physics degrees (2012/13 to 2019/20)
Is my bias
Download PDF poster
⚠️ Video to be uploaded after PERC; technical difficulties! ⚠️
Is my bias your bias?:
The variability of demographic gaps in the good degree rate for UK physics degree programmes
- Presenter
- Astra Sword (astra[dot]sword[at]open.ac.uk)
- Supervisors
- Sally Jordan and Annika Lohstroh
Motivation
In the United Kingdom (UK), demographic gaps in the rate at which graduates from different backgrounds are awarded the top academic outcomes (i.e., degrees graded 1st or 2:1; a.k.a. the good degree rate) are important measures of equity for the higher education (HE) sector [1].
Research Questions
- Research Question 1
- How similar are outcome gaps across different physics courses?
- Research Question 2
- How do the outcome gaps in physics compare to other subjects?
Data and Methods
We procured a Large, National Degree Outcome dataset (LaNDO) from HESA [2]. This dataset includes all first-degree students in higher education from 2012/13 through to 2019/20. We used the IOP’s degree accreditation registers for the same time period [3] to identify accredited physics degrees programmes, highlighted in green in the results.
For each characteristic, we calculated the risk difference Δ in the good degree rate according to:
Δ = P(G|¬M) - P(G|M)
Where 𝐺 is the event of obtaining a good degree and 𝑀 is the event of belonging to the majority (modal) group. To avoid distortions due to COVID-19, we calculated results for 2019/20 separately, indicated by a *; all other results are for 2012/13 to 2018/19.
Within Physics
We calculated the weighted (by cohort size) mean of risk differences across years for each accredited physics programme. To ensure data protection, programmes with fewer than 23 students were excluded from the analysis [4].
Across Subjects
We calculated risk difference for all students on accredited physics degrees; then grouped by subject code (given in brackets) [5]; science, engineering, technology subject status; and at the sector level. We calculate 95% confidence intervals via the Newcombe score method [6].
Results
Non-Enhanced Degrees
Non-enhanced degrees are degrees that typically take three years to complete and result in a batchelor's level qualification.
- Gender
- In line with the rest of UK HE, women are more likely to gain a good degree in physics, but are disadvantaged on specific programmes.
- Ethnicity (UK-Domicile Only)
- The award gap for non-white students in physics is smaller than in the sector, but several physics programmes have gaps larger than the sector average.
- Disability Status
- In contrast to the sector, physics students with a disability marker are generally much less likely to obtain a good degree.
- Age on Entry
- Award gaps across age categories are highly dispersed in physics, with some much larger programme gaps than the sector average in both directions.
Enhanced Degrees ✨
Enhanced degrees, also known as an integrated or undergraduate masters programmes are undergraduate programmes that award a masters-level qualification. They are typically the same as their non-enhanced equivalent, but with an extra year of masters' level content.
- Gender
- For enhanced degrees, the gender gap is much smaller in physics but still statistically significicant and favouring women.
- Ethnicity (UK-Domicile Only)
- Similarly, for Ethnicity the award gaps for enhanced degrees are generally smaller tha those for non-enhanced degrees; except in physics!
- Disability Status
- For disability status, the pattern across subjects isn't as clear for enhanced degrees as it is for batchelors level programmes; but it still points to a gap disadvantaging students with a disability marker.
- Age on Entry
- Award gaps across age categories reimain highly dispersed and of a similar size for enhanced degrees in physics; but larger across all the other subject areas examined.
Discussion
For each demographic characteristic examined, we find the gaps in degree outcomes to be surprisingly hetergenous, with substantial gaps in both directions when looking at outcomes for specific institutions. Below we indicate two statistical issues that may explain the observed heterogeneity within physics and discrepancy between subjects.
Future Work
Our next steps will be to generalise this analysis to more characteristics and analyse the dispersion of physics degree outcomes. More broadly, we are working towards building an multinomial regression model that takes into account intersectionality, causal inference, and the possibility of over/under-dispersion.
References
- [1] Office for Students. (2021, March 11). Participation performance measures—Office for Students (Worldwide). Office for Students. https://www.officeforstudents.org.uk/about/measures-of-our-success/participation-performance-measures/
- [2] Jisc. (2022). Tailored datasets. Jisc. https://www.jisc.ac.uk/tailored-datasets
- [3] Institute of Physics. (2022). Degree accreditation and recognition. Degree Accreditation and Recognition | Institute of Physics. https://www.iop.org/education/support-work-higher-education/degree-accreditation-recognition
- [4] HESA. (2022). Rounding and suppression to anonymise statistics | HESA. https://www.hesa.ac.uk/about/regulation/data-protection/rounding-and-suppression-anonymise-statistics
- [5] HESA. (2012). JACS 3.0: Principal subject codes. https://www.hesa.ac.uk/support/documentation/jacs/jacs3-principal
- [6] Seabold, S., & Perktold, J. (2010). statsmodels: Econometric and statistical modeling with python. 9th Python in Science Conference. https://www.statsmodels.org/devel/generated/statsmodels.stats.proportion.confint_proportions_2indep.html
- [7] UCAS. (2014, October 13). Mature undergraduate students. UCAS. https://www.ucas.com/undergraduate/applying-university/mature-undergraduate-students
- [8] Dean, C. B., & Lundy, E. R. (2016). Overdispersion. In Wiley StatsRef: Statistics Reference Online (pp. 1–9). American Cancer Society. https://doi.org/10.1002/9781118445112.stat06788.pub2
- [9] Hernán, M. A., Clayton, D., & Keiding, N. (2011). The Simpson’s paradox unraveled. International Journal of Epidemiology, 40(3), 780–785. https://doi.org/10.1093/ije/dyr041