Radical proposed changes to the Research Excellence Framework (REF) must be examined closely to allay concerns they will reintroduce game-playing and exclude early career scholars and technicians, say experts who have welcomed a year-long delay to the exercise.
Announcing that the?REF results day will now occur in December 2029, the REF team drew attention to the “complexities” for universities in “using Hesa [Higher Education Statistics Agency] data to determine REF volume measures” and “fully breaking the link between individual staff and institutional submissions”.
Under new proposals, institutions will no longer provide lists of researchers submitting outputs to the next REF but will instead declare their number of research-active staff to Hesa, with the tally used to determine the volume of outputs submitted by each unit of assessment. As in 2021, institutions will need to?submit 2.5 outputs for every full-time equivalent researcher,?although there will be?no minimum or maximum number of outputs per researcher.
The new process of how “volume-contributing staff” should be identified and declared to Hesa is, however, too vague and open to misrepresentation, argued Emiliya Lazarova, head of the University of East Anglia’s School of Economics, who heads the Royal Economic Society’s body for university economics departments.
“Every institution is allowed to define who is a research-contributing staff member – for some institutions, that means staff who spend 10 per cent of their time on research, but for others it’s 40 per cent,” said Professor Lazarova.
“If it’s just 10 per cent, institutions could easily boost their average score and research power by bringing in people on very fractional contracts. Richer universities will be most able to buy research position and power – poorer institutions will be at a real disadvantage.”
Proposed spot checks to ensure that a unit’s output pool was broadly representative of?its research staff would not make any difference if “everyone can define their own numbers” and “play the system as they like”, said Professor Lazarova, adding: “Unless you have strict guidance what are you actually checking?”
The decision to bar graduate students from submitting sole-authored papers, including their PhD thesis, was also misguided as it would prevent many early career researchers from submitting to the REF, she added.
“For economics, it’s a disciplinary norm to have a really good ‘job market paper’ written solely by a PhD student – this is found in other social science subjects, and I don’t see how stopping these entering the REF is going to help make it more inclusive,” said Professor Lazarova.
The lack of precision about which staff should be declared as “volume-contributing” would also tempt some universities to exclude “research enablers” who did not have a stack of research papers to their name, added Amanda Bretman, dean of research quality at the University of Leeds.
“There is a big group of people – such as facilities managers – who don’t have direct responsibility for producing research but help hundreds of people to do their jobs,” said Professor Bretman.
“We want to value their contribution, but we also have to make tough decisions about the best REF submission for the university,” she continued, explaining that higher numbers of volume-contributing staff would also mean having to produce more impact case studies.
“Every institution will be thinking really hard about their output pools,” said Professor Bretman, who said she “welcomed the extra time to reflect on these issues”.
“I’m not sure this is just about ironing out a few issues with Hesa collection – if it was this could probably be done without this delay.”
