Universities in England are under intense pressure to measure what value they are adding. Policy wonks go giddy with excitement when new data become available. Indeed, for a time after its results? in 2016, the??threatened to dominate university policy.
While I was minister for universities and science between 2010 and 2014, I ensured that the underlying data – which set out the earnings of graduates one, three and five years after completing particular courses –?were collected and made available to researchers to analyse for prospective students.?The focus on taxable pay helps assess how much of their fee loans graduates will be able to pay back. But the data are not a good metric for economic outcomes overall because, for example, they undervalue universities in areas where pay is lower. They also penalise part-time work, even if well paid, as they only measure total earnings.
In an article?published in Times Higher Education on 11 April 2019, I warned that we were in danger of putting too much weight on this one dataset. I am now much more optimistic that LEO data will be seen in perspective, as it is understood that this one measure cannot capture a course’s full economic value to graduates. However, there is now a new kid on the block, a new measure of graduate employment outcomes. And I fear we are in danger of repeating the same mistake and rushing to give it too much weight.
The old graduate outcomes measure, the? (DLHE) survey, questioned graduates just six months after they graduated. That time frame made it easier to track outcomes, but it was hardly a measure of long-term careers. And it could be abused – one university reportedly offered its graduates a short work-experience programme five months after they left so that it scored remarkably well on DLHE and could boast about its graduate outcomes in its marketing materials.
After a lot of hard work by the Higher Education Statistics Agency, the new Graduate Outcomes data, centrally administered, now report graduate destinations after 15 months. released in June 2020, cover the 360,000 students who graduated in 2017-18, making it the biggest annual social survey in the UK. Even so, the samples could still be too small to assess an individual course at a particular university, which might have?fewer than?10 graduates, for example. Of recent graduates in work, the data suggest that 82 per cent of those who studied science subjects and 71 per cent with non-science degrees are in high-skill occupations, defined as those in 1-3. (These are then called “graduate occupations”; the remaining categories 4-9, which are for intermediate or low-skill work, are regarded as non-graduate occupations.) Black and ethnic minority graduates have an employment rate 8 percentage points lower than white graduates.
The Office for Students is already using the new data to track “progression to managerial and professional jobs”, as part of measuring quality and standards. But there could be perverse effects. Some graduates stay near their university. If it is located in an area with fewer high-skill occupations, it will appear to do worse compared with universities in the prosperous south east, which undermines the government’s broader levelling-up agenda. Many groups of ethnic minority students unfortunately do less well in the job market for any given level of degree, so universities that recruit more of them will also appear to do worse.? from the Higher Education Policy Institute on the gender pay gap used the new data to show that 75 per cent of full-time employed male graduates were in high-skill jobs, as against 68 per cent of women – potentially creating a perverse incentive to recruit fewer women.
These selection effects are a real danger because the Office for Students last November about minimum standards for university courses, which would include a baseline for the proportions of students who “progress[ed] to managerial and professional employment” (or higher-level study) – without any adjustment for the characteristics of the student body. Other criteria are completion rates and the proportion of students who “continue from their first to second year”.
By adopting such metrics, instead of rewarding value added, the OfS will be rewarding the selection of a certain type of student. The regulator says it will take account of this in its interventions, but any published starting point could still do real damage. education secretary Gavin Williamson urged it to use such measures as part of the minimum standards in its regulatory regime. The proposed standards might be low, but there is still a danger that once established they are ratcheted up, which would represent a threat to the progress?that has been made over the past decade in broadening access to higher education and measuring real educational quality.
The new metric for Graduate Outcomes is getting so much attention because it plays into the highly charged debate about graduates in non-graduate jobs. Are we wasting resources educating young people way beyond what they need for a job? Do they then displace other people, who did not go to university but are adequately qualified to do such jobs? Are these underused graduates unhappy and frustrated – like moody figures from a Dostoevsky novel?
It is by no means easy to assess if a graduate really is in a “non-graduate job”. The Standard Occupational Classification, mentioned above, is actually designed to assess skills. Using it?for the unintended purpose of defining jobs as “graduate” or “non-graduate” has some peculiar results. Some jobs are treated as “non-graduate” even though they may require a degree – veterinary nurses, for example. And there is diversity even within a given category of job. Managing a hotel is counted as a graduate job, but managing a pub is not. Some pub managers are graduates. Are those typical pubs or ones with bigger management challenges? There is a pay advantage for a graduate even in a non-graduate job, which suggests they may be doing rather different work. Moreover, the very process of taking a job title and then converting it into an occupational classification can go wrong. Yet, in future, that could determine if funding for a course is eliminated or even if a university in England loses its degree-awarding powers.
The closer one looks, the trickier it becomes to apply this classification to education policy.
Behind this there is the classic English assumption that the task of education and training is to end up with a round peg in a round hole. This is why selective admissions will be rewarded in the new quality measures. But it is a static model?that does not recognise distance travelled nor allow people to advance or jobs to change. A metric assessing whether you are in a supposedly graduate-level job just 15 months after leaving university is taking a very limited, short-term view. It favours jobs?that get you on a brisk ascent to a plateau and penalises careers where development is generally slower. As a result, jobs with technical educational requirements do well. Entrepreneurship does badly. So do careers in the creative industries.
When a graduate is pronounced to be stuck in a non-graduate job, there are two different interpretations of what this means for education and skills policy – and both are unpalatable. It might be that they do not have the necessary skills for “graduate work” at all. Andreas Schleicher of the OECD is quoted in John Kampner’s 2020 book Why the Germans Do it Better: Notes from a Grown-Up Country as arguing that the UK’s problem is that “only 5 per cent of the labour force have higher skills than what’s required for their present job. That is a huge threat to productivity”. That would be a serious challenge to the quality of our educational system but would not suggest we have a workforce?that is wasting its skills.
The alternative critique is that graduates are “over-educated”, with real skills?that are not being deployed, leaving them unhappy and frustrated. It takes us back to the world of Jude the Obscure and the Marquess of Salisbury who, according to his biographer Andrew Roberts, “never believed in over-educating the working classes”. He wrote that it was hard to induce the working man to send his children to school to learn because, although it meant that occasionally a child might get a clerkship, nonetheless “in ninety-nine cases out of one hundred the boys must fail and return sullen and discontented men to the plough tail”.
I find the concept of over-education repellent and was disappointed the Office for National Statistics used the word in the title of My starting point is that we are all under-educated. There is always more to learn and more to try to understand. The value of education goes beyond economic returns – though there are legitimate questions about the best use of public money. Moreover, it matters whether graduates and indeed non-graduates are unhappy in their work, something?that touches on deeper issues of human fulfilment and flourishing.
The new Graduate Outcomes data also provide evidence on this: 86 per cent of graduates assess their current activity as meaningful (88 per cent for sciences, 84 per cent for non-sciences – and 93 per cent for the subset of non-scientists who studied education). In addition, 88 per cent say that their current activity fits with their future plans and 72 per cent that they are utilising what they studied (75 per cent for science versus 69 per cent for non-science). These are not bad scores. And graduates have quite high levels of well-being. This shows that there are other important benefits from higher education beyond getting a “graduate job”. Indeed, for Universities UK found that only 34 per cent decided to go to university to get a higher salary.
In in the Oxford Review of Economic Policy, labour economists Francis Green and Golo Henseke also showed that, even when graduates fail to get high-skill jobs and are unhappy about that, they still enjoy “significant alternative benefit from higher education…including better self-reported health and greater external benefits for the rest of society”. These apparently over-qualified graduates are less satisfied with their work than non-graduates doing the same job but also enjoy the non-financial benefits of higher education. That really is a tricky trade-off.
The new data on Graduate Outcomes can illuminate these deep and important issues. It would be a great pity if, instead, a crude interpretation of them is used to judge the success or failure of a university. To drive up standards, we need to focus on value added by the university and the journey travelled by the student, not on selection of students likely to do well anyway.?
David Willetts is president of the Resolution Foundation and chancellor of the University of Leicester. He was minister for universities and science from 2010 to 2014. His book, A University Education, was published by Oxford University Press in 2017.
