É«ºÐÖ±²¥

We are proud of our reputation for teaching quality," says David Mackintosh, deputy vice-chancellor of Kingston University.

"This is our primary focus as an education institution, so we are committed to recognising and rewarding excellent teaching, as well as research."

But in doing so, he believes the university faces a challenge: finding fair and equitable criteria with which to assess top-quality teaching.

One of the most obvious sources of information is student feedback from questionnaires, but this is not always an accurate guide.

"Some subjects are less popular, and this can sometimes generate a context in which good teaching per se can risk being unfairly assessed," Mackintosh says.

"Looking at pass rates or the improvement in student grades is another way to assess teaching, but the difficulty is that, in most cases, students have been taught by several lecturers so it isn't always possible to credit student success to one individual."

Where there is strong evidence, the university's promotions committees do take this into consideration, he says, but Kingston would like to "go further" and take greater account of teaching at selection panels.

The difficulty of assessing teaching is an issue that is raised time and again when academics bemoan the lack of promotion prospects associated with the area, in contrast with those associated with research. But with government ministers complaining that excellent university teaching is "not universal" and that too little is done to reward it, and with students demanding more for their money, it looks increasingly like a hurdle that universities will have to overcome.

The idea that it is easy to assess research performance and difficult to assess teaching is a myth, according to Paul Ramsden, the former head of the UK's Higher Education Academy and originator of Australia's national student survey.

The reality, he says, is that both are difficult to assess - but that is no reason not to make the attempt.

"My personal experience is that it is very hard to assess research," Ramsden says.

"Subjective factors about impact have an enormous influence. Many of us will have sat on appointments and promotions panels in academic environments where there has been quite lively debate about the extent to which the citation analysis, or the evidence of impact, or the number of papers actually reveals that someone is in the category deserving promotion.

"Having said that, I think the way to deal with the assessment of teaching is to make it less subjective and to incorporate elements that can be tested externally. For example: 'Has your idea about a new way of teaching Jane Austen been taken up in other universities? What's the evidence of that? Show me it has happened.'"

The use of teaching criteria in university promotions policies varies widely.

Reward and Recognition in Higher Education: Institutional Policies and their Implementation, a recent study by the HEA and the "Genie" Centre for Excellence in Teaching and Learning at the University of Leicester, examined the promotion policies of 104 British institutions (see boxes below).

In many cases, teaching criteria were not mentioned at all - and, in the 73 cases where they were, only 45 policies included specific criteria for assessing it.

The study also sought to examine how these policies were enacted. However, very few universities were able to provide any data, which the researchers say suggests a lack of transparency.

The information that was available indicated that strength in teaching was rarely a key reason for promotion in the research-intensive universities, particularly at higher levels.

This helps to explain the findings from the first stage of the HEA-Genie project, which surveyed more than 2,700 academics and found that most respondents felt that teaching is not rewarded in promotions to the degree that it should be.

But some institutions - even those focused heavily on research - have been making considerable efforts to address the issue.

In one example, a research team from Queen Mary, University of London visited universities Australia, Sweden and elsewhere in the UK to examine different approaches to encouraging, assessing and rewarding excellent teaching in research-intensive institutions, including the ways in which the measurement of teaching performance informed human resources policies and procedures.

The aim was to identify a series of indicators of teaching performance appropriate for Queen Mary, including ones that could be used to underpin an individual's case for teaching excellence.

"The measurement of teaching performance does not appear to all staff to have had the same robustness that is broadly accepted as governing the measurement of research performance," the Queen Mary researchers say in their 2007 report.

They agree with Ramsden that measuring both research and teaching in higher education are complex processes, but that for research, the methodology has been extensively debated and developed within the academic community.

"A consensus appears to have emerged that generates, for example, broad confidence in the integrity of judgements made as part of the research assessment exercise process," they observe.

"The conviction underlying this project was that a similar consensus over the measurement of teaching excellence would make a significant contribution to the further enhancement of teaching and learning across the college."

They too found many different policies and procedures in operation, and also points of contention.

At the University of Sydney, for example, academics had to demonstrate high levels of achievement in both teaching and research to gain promotion, but there was scepticism from some staff about the rigour with which cases for "outstanding" contributions to teaching were scrutinised during the promotions process.

Other universities created a separate category of "professorial teaching Fellow" that did not require evidence of international-calibre research output.

Some universities used teaching portfolios as part of the promotions process; others did not. There were worries about whether promotions panels had the expertise to evaluate the teaching portfolios brought before them, and about the fact that the claims made in those portfolios could not always be evidenced.

References from heads of department held considerable weight in some institutions. In others, staff expressed concern at their institution's focus on using student-satisfaction data to assess academics' contributions.

Despite these issues, the Queen Mary team succeeded in developing a set of indicators that are now set to become part of the university's promotions criteria.

Morag Shiach, vice-principal (teaching and learning) at Queen Mary, who led the project, says: "We have tried to mirror the kind of evidence base that was used in relation to research, so it is about national and international profile and marks of esteem - prizes, awards and fellowships for teaching. We also highlighted getting external-grant income to support projects to develop teaching.

"From 2008, we made the indicators available for colleagues to use to structure their cases for promotion. Now we have developed a new strategic plan and, as part of that, we have committed to embedding these indicators in all probationary, mentoring and promotion processes."

Philip Plowden, dean of Northumbria University's School of Law, believes that his university's promotions policy shows that it is perfectly possible to have a meaningful assessment of achievement in teaching.

Northumbria's policy examines how what happens in the classroom is "taken to an institutional or national context - through leadership, project management, research - in a variety of different methods", he says.

"My own view is that we can expect excellent learning and teaching from all staff, and what the position of fellowship or chair should acknowledge is the way in which personal practice is then developed to effect change across the university or (across the) sector."

Earlier this year, Sandra Wills, executive director of learning and teaching at the University of Wollongong in Australia, gave a keynote lecture at a UK National Teaching Fellows conference, titled "Teachers DO Get Promoted".

Wollongong has a highly developed system that allows staff to base their case for promotion on excellent teaching, but Wills says she still encountered cynicism from staff.

"No matter how many times you tell people that they do get promoted on the basis of their teaching, the mythology persists that they don't," she says.

So, while serving on the university's promotions committee, Wills began collecting data about its decisions.

The numbers, gathered over four years, told a positive story. Staff who based their case predominantly on teaching had an 80 per cent chance of promotion at both the senior lecturer and associate professor level at Wollongong.

This success is thanks to the university's clear guidance on submitting evidence about teaching, its training for both applicants and promotions-committee members, and a mentoring process based on ensuring that applications are ready before they reach the committee, regardless of whether they are based on research or teaching, says Wills.

The emphasis is on staff having a mix of evidence about their teaching: from students, peers, supervisors and also via self-reflection.

As part of this process, Wollongong has even developed a system of formal peer review for teaching.

"Because research is the big focus for promotion and peer review of research is standard practice, I wanted to make sure we had an equivalent for teaching. Unless teaching is peer reviewed, it won't be held in as much esteem," Wills claims.

"There is a view that peer review of research is the bee's knees, but in fact it is fraught as a system. Peer review of teaching is just as flawed in one sense, but it is only one indicator and is not the only way we judge people."

Informal peer review, where colleagues offer one another constructive suggestions on how to develop as teachers, is encouraged at Wollongong.

But staff also have the option of formal peer review, carried out under formal university processes, for academics who wish to collect evidence for a probation or promotion application. It is not as intimidating as it sounds, says Wills. The outcome reaches the promotions committee only if the academic being reviewed wants it to, and the process is entirely confidential.

For formal peer observation of teaching, the first step is for the academic involved to choose the reviewers from a list of accredited staff - all of whom are scholars who have been nominated by faculty deans as esteemed teachers and who have been trained as peer reviewers.

The reviewer and reviewee meet to discuss the observation before it happens, including the "teaching context", such as the class to be observed and the aims of the exercise.

After the observation, the reviewer writes an honest appraisal of the reviewee's strengths and weaknesses, following a standard format, and offers to discuss the results.

The reviewee then decides whether he or she would like the reviewer to prepare a formal "evaluation summary".

If they press ahead, the reviewee receives a copy, and he or she may then decide whether or not to use the report as evidence in an application for promotion. To be considered for promotion, applicants need two reviewers, one from their faculty and one who is external to it. The process is still developing, extending the formal peer-review process to curriculum development and educational practice.

"People always think that peer review of teaching is peer observation of performance, but we believe teaching isn't just about your performance in the classroom," Wills notes. "It covers many activities and we're interested in the full range."

The system's 80 per cent success rate is not indicative of low standards, she adds. It reflects the fact that the university does all it can to ensure that applications appear before the committee only when they are ready. "A lot of work is done behind the scenes," she says.

Wills is now examining the success rate for promotion based on an academic's research record. In the first year of monitoring, a success rate of 95 per cent was identified, suggesting that the chances of promotion for scholars focusing on research still seem to be slightly better than for those focusing on teaching. Wills hopes to see promotion prospects in the two areas equalise.

Back in the UK, Kingston's Mackintosh believes that what universities really require is a good national model to help them develop their policies and procedures.

Ideally, he says, there would be a fair and appropriate national reference point that all universities could sign up to.

"We feel this is something that higher education is wrestling with and we would welcome a cross-sector approach to identifying such a model."

Help is at hand. Drawing on her analysis of UK universities' policies and procedures, Annette Cashmore, who led the Genie projects for the HEA, is developing a set of criteria she hopes could form a flexible national framework.

The idea is to help institutions define achievement and excellence in teaching and learning, while taking into account their diverse missions (see bottom box, page 37).

She is also compiling dozens of career-path case studies (see box left), based on interviews with academics about their experiences, and will be collaborating with Wills in Australia to collect examples of good practice and to monitor promotions data in a number of universities.

But Cashmore does not think this will be enough. What is also needed, she says, is a culture change in universities at all levels, including among senior managers and promotions panels.

And universities need firm incentives to reward good teaching, to match the strong financial reasons for institutions to prioritise research.

Cashmore would also like to see all institutions record statistics that demonstrate the use of teaching-related criteria.

"One solution would be to make transparency and consideration of promotion criteria and their implementation an essential component of institutional quality reviews. The role of the Quality Assurance Agency could be vital here," she suggests.

PROPOSALS FOR É«ºÐÖ±²¥ EVALUATION OF LEARNING AND TEACHING

Based on her research, Annette Cashmore is developing a set of criteria for the evaluation of learning and teaching to form a national framework.

The criteria are meant to be flexible and not exhaustive and to be suggested to institutions to help shape their own policies. They will offer detail about what type evidence could be used.

LECTURER/EQUIVALENT

- Input to delivering or leading teaching

- Organisation of courses/modules

- Student feedback

- Peer observation

- Peer feedback/review

- Evidence of evaluation of teaching approaches

SENIOR LECTURER/EQUIVALENT

- Evidence of scholarship of teaching and learning, such as awareness of relevant literature, teaching informed by research (own and others'), and writing and contributions to textbooks

- Institutional awards

- Own research in teaching and learning

- Input into institutional policies

CHAIR

- Presentations and publications

- National awards

- Evidence of national/international impact

- Input into national/international policy and strategy

CAREER-PATH CASE STUDIES

TEACHING FELLOW at a Russell Group university

Following a period of postdoctoral research in the US, Clive decided to return to Britain. He was appointed to a five-year temporary lectureship in a pre-1992 university, replacing the teaching of a senior member of staff who had been awarded a fellowship.

Clive had a fairly high teaching load and struggled to get his microbiology research started. He became passionate about teaching and really enjoyed interacting with undergraduate and postgraduate students. After three years, he began to look for a permanent post and applied for several at the university but was unsuccessful, mainly due to his lack of grant funding and publications.

Over the next couple of years, he published several microbiology papers. In 2000, he was appointed to another five-year temporary lectureship, this time at a Russell Group university. Again he relished the teaching, but felt under pressure to obtain grant funding for his microbiology research.

Despite this, he became more involved in teaching initiatives, and contributed to the successful bid for a Centre for Excellence in Teaching and Learning at the university. In 2005, he made the decision to move away from laboratory-based research and transferred to the CETL as a teaching Fellow. This post allowed him to innovate and begin more serious educational research.

But funding for the national CETL initiative is drawing to an end, Clive's current contract expires this summer and his future is uncertain. His institution has discussed the possibility of a "teaching-only" contract, but it has a ceiling of how far he can go in terms of promotion. Clive is 44 and has a growing family.

LECTURER at a 1994 Group university

After graduating with a PhD, Simon spent several years teaching in a challenging inner-city comprehensive school, but became disenchanted with it. He managed to get a postdoctoral research assistant job in the department where he had studied for his PhD and quickly re-established himself in the lab.

Three years later, he was appointed as a lecturer in the same department. With his strong background in teaching, he was interested in developing new approaches and gradually moved away from lab-based research. This was not a problem, as it was a large department with many "research stars", although he did feel some pressure, particularly when the department made its submission to the research assessment exercise.

He has been told by his head of department that it is unlikely that he will progress beyond lecturer without a good subject-based research profile. Simon feels that he has missed the boat and has concentrated on his teaching. He has been at the top of the lecturer scale for several years, and despite applying for promotion based on his teaching record, he has been unsuccessful twice.

He received a university teaching award a couple of years ago and is developing a case for a National Teaching Fellowship, which he hopes will strengthen his promotion chances.

LECTURER in biosciences at a Russell Group university

Richard has always been keen to teach. As a postdoctoral researcher, he took the opportunity to supervise students when he could, and he gave tutorials for first and second years. He did not get paid any extra for this, but did not mind because he enjoyed it.

In 2008, he secured a lectureship at a Russell Group university. He felt that it was a combination of his physiology research and his teaching experience that got him the job.

Now he is completing a postgraduate certificate in higher education and hopes that his future career will allow him to combine teaching and research in his field.

His mentor advised him not to get too involved "in that pedagogical research business" because it would slow down his lab research. But Richard is determined to be involved in teaching and learning as much as he can.

His current teaching contact time is about 150 hours per year - not including time spent supervising project students.

His mentor and now his head of department are concerned and want him to reduce the time he spends teaching. However, they are not pressuring him. Richard remains confident in his own abilities and the opportunities that may come up for him at his institution.

SCIENTIST at a Russell Group university

Jane is a scientist who has stayed at the same institution for the whole of her academic career. From 1983 to 1987 she worked on a fixed-term contract as a postdoctoral worker, playing a significant role in directing the research of the lab.

Throughout this time she enjoyed interacting with students in the lab, and also started doing some lecturing on an undergraduate course. She found the work challenging, but enjoyable.

In 1987, Jane was successful in getting a temporary lectureship, and after a couple of years it was made permanent. Her portfolio of teaching grew and she became interested in taking a wider view of teaching and learning. However, she was told to "cut back on this teaching stuff". She did so and focused instead on her lab research, still doing some teaching, but only the minimum.

Jane was promoted to senior lecturer in 1993. This had been her aim; she now decided that she would ignore the previous advice given to her and begin to do more teaching, course and module development. She also maintained her lab work. She spent five years as head of department and became involved in institutional policymaking and strategic development.

In 2008, she was awarded a personal chair. Jane hopes that this will help her to pave the way for others to have an academic career that can involve both teaching and research.

PRINCIPAL LECTURER at a post-1992 university

Sarah became a senior lecturer in modern languages in a post-1992 university in 1996. Part of her remit was to set up two modules for the first-year undergraduate programme.

She really enjoyed this initiative and her teaching, and worked to establish a new system of pastoral care for students. Her involvement in quality assurance committee work for the university also grew and as a result of all this, in 2005 she became a principal lecturer within the institution.

Sarah has high hopes of achieving further promotion and may aim for a chair position in the future. She is carrying out research into teaching approaches in modern languages, but does not feel encouraged. She is not given specific time for the research and now has a high workload of both teaching and administrative duties. She has also tried, unsuccessfully, to look up the criteria for promotion at her university.

Sarah has seen people promoted to a chair on the basis of their teaching and learning activity. Usually it is clear that they are doing a lot of teaching-related research and are involved in committees. However, clear direction is not available as to the sort of things that are needed to gain promotion.

Source: Interviews by Genie CETL, University of Leicester. Identities have been protected.