Student Evaluations of Teaching and Academic Career Pathways

University management has been consulting on the development of its Academic Career Pathways for some time. The new promotion pathways came into effect from the start of the 2018-19 academic year, and will be used for promotions for the first time in 2019.

Sheffield UCU has been broadly in favour of changes to academic promotion criteria and procedures that increase transparency around the expectations at each grade, that provide clear pathways for those on teaching-specialist and research-specialist contracts, and that ensure that there are ways for colleagues to move between different pathways.

However, when the detailed expectations for performance at each level were released last summer, we were surprised and disappointed that the University had prioritised student evaluations as the primary (and required) means of evidencing high quality teaching.

There is extensive evidence that student evaluations do not measure teaching quality, and that they are biased on the basis of personal characteristics of the instructor – including gender and race. The feedback we received from members about the inclusion of this metric in the new career pathways was overwhelmingly negative. We therefore presented a briefing paper to members of the University’s senior management, briefly outlining the extensive academic research on these issues, and identifying the implications of that research for the formulation and implementation of student evaluations, and for the use of student evaluations in employment decisions such as promotion. You can access the latest version of our briefing paper, compiled by SUCU committee member Simon Stevens, here: ‘Academic Career Pathways and Student Evaluations of Teaching.’

As a university, we argued, Sheffield should adopt policies that are evidence- and research-based and that reflect international best practice. In assessing teaching quality for the purpose of promotion, the University should not require the use of a metric that is known to be flawed. And the University should adopt policies that promote good teaching, rather than policies – such as prioritising of student ratings in promotion decisions – that have a negative impact on teaching quality.

We are pleased that, as a result of the meeting at which we discussed our briefing paper with representatives of senior management, the University has agreed to remove the primacy of student evaluations from the Academic Career Pathways. In the revised expectations document, ‘consistently excellent student feedback’ is now listed only as one of a number of possible (and not required) forms of evidence that staff could choose to use to demonstrate that they have met the teaching criteria for promotion.

We understand that the University will be reviewing its formulation and implementation of student evaluations later this year, and we hope that that the University will take this opportunity to implement our other recommendations on this issue, especially regarding the kinds of questions in student evaluation questionnaires that invite biased responses.

We have also been concerned to hear from members that this semester the University has implemented a new policy that requires module leads to produce a plan of action if any question in the student evaluations of their module receives a ‘satisfaction score’ of less than 75%. We share the deep concern that many members have expressed about this policy, which creates an additional, punitive burden for staff on the basis of a metric that does not measure teaching quality. This additional workload burden is more likely to fall on those against whom student evaluations are known to be biased, including female instructors and BAME instructors. And, as we argue in our briefing paper, policies that prioritise student evaluation ratings in this way have a negative impact on teaching quality: instructors are incentivised to devote their attention and energy to approaches that correlate with improved student ratings but that are unrelated to (and may even have a negative impact upon) improved student learning.

We are surprised that the University has agreed to alter the academic career pathways criteria in light of the evidence on the limitations and biases of student evaluations of teaching, and yet has simultaneously implemented a burdensome new policy that ignores those limitations and biases. We hope this policy will be quickly abandoned.

We will continue to work on this issue, and will be convening an action group on the University’s use of student evaluations and other metrics on Wednesday 1 May, 1-2pm in Hicks F30: please come along to discuss our branch response to these issues.