New grant evaluation procedures in the Netherlands that ask panellists to consider an applicant’s academic citizenship and contribution to open science have turned funding decisions into a “lottery”, it has been claimed.
At the start of last year, the Dutch Research Council (NWO) began a new two-stage assessment of applications to its main competitive research funding scheme, known as Veni, Vidi, Vici, which invites each applicant to submit an “evidence-based CV” along with the outline of a research idea for review by a “broad scientific committee”.
Reviewers on the Vidi stream for early career researchers, who can apply for up to 800,000 (?695,000) over five years, are asked to assess applicants’ “key outputs”, but also to examine whether a “researcher's qualities clearly exceed what is customary within the international peer group”, including the strength of their “contributions to open science and academic citizenship”.
However, this filtering process – which takes place before successful applicants submit full research proposals – is leading to some “truly exceptional” postdocs being rejected at the first round while notably less stellar candidates have progressed, said Matilde Galli, group leader of the Galli cell cycle regulation lab at the Hubrecht Institute, based in Utrecht.
“If you’re not an expert in a certain discipline, it’s quite hard to judge the strength of a proposal, but you can assess more general things like contributions to open science,” said Dr Galli.
“I’ve nothing against open science – and scientists should be recognised for it – but that should come from an employer. A research funding agency should examine whether a grant application is good enough,” she added.
The narrative format of applications – which has also been introduced in Switzerland and the UK – also encouraged applicants to make boastful claims about their achievements, continued the Italian-Dutch scientist. “People are making claims that can’t be checked, and it favours those who like to show off about how fantastic they are."
The Dutch reforms are part of wider changes linked to the government’s Reward & Recognition (R&R) programme, designed to eliminate bias in decision-making, which have forbidden the mention of journal impact factors or h-indexes, or the inclusion of publication lists or prizes won.
The switch to narrative CVs and the consideration of CVs by non-specialists has turned funding decisions into a “lottery”, said Dr Galli, who noted that her “near-identical” application rejected by the Vidi talent scheme was later awarded a European Research Council starting grant in 2021.
“I’m really worried by this trend, and I’m not sure I?will stay in the Netherlands if it becomes such a struggle to prove that you are producing good work that should be funded,” said Dr Galli.
Willem Mulder, professor of precision medicine at Radboud University and Eindhoven University of Technology, said he was “concerned about the way language is being deployed to enforce top-down, one-size-fits-all-but-suits-nobody policies”.
“The ‘narrative CV’ that has been stripped of all evidence is referred to as the ‘evidence-based CV’. Obfuscating policy with misleading language is concerning and anti-scientific,” Professor Mulder said. “Fair and merit-based allocation of capital is the way to advance science.
“As environment and supervision are critical early on, I agree that early-stage investigators should not be judged on their track records as much as experienced principal investigators. However, since the best predictor of future success is?past action, experienced PIs must have a demonstrated track record of excellent productivity.”
Robbert Hoogstraat, senior project leader at R&R, told Times Higher Education that open science and academic citizenship were “sub-criteria” used to assess CVs but they did not influence consideration of research proposals, where excellence remained “central” to decision-making.
“The amount of emphasis on open science or academic citizenship…is decided by the panel members. We do take these signals seriously and are working on…rubrics to help panel members weigh all these different sub-criteria consistently,” said Mr Hoogstraat.
He added that a report by an independent advisory council to the Dutch parliament had examined concerns about how research was being assessed and supported the current approach, which was also backed by the Dutch science minister, Robbert Dijkgraaf, a professor of theoretical physics at the University of Amsterdam.
