On 24 September 2018, UNIFR joined a fast-growing number of institutions in signing the San Francisco Declaration on Research Assessment (DORA). Our University thereby commits to evaluating researchers, for hiring, promotion and funding decisions, without using “publication metrics”, basing its decisions only on other ways of assessing research quality.

Metrics galore

‘’Publication metrics’’ are quantitative indicators describing a researcher’s record in terms of papers published in scientific journals, such as the journal’s impact factor, the h-factor, total citation counts, etc. Such indicators have been increasingly used in the past two decades in research assessment, because they are easy to obtain and provide objective criteria, to the extent of becoming crucial for the career of many researchers.

They played an important role around the turn of the Millenium to allow the most productive and impactful researchers to obtain funding and tenured positions even when they were not supported by influential professors. Publication metrics thus made an important contribution to loosening the influence of “patronage” that often played an excessive role in researchers’ careers in the past.

But now many share the opinion that these metrics have outlived their usefulness. Researchers have integrated the key role played by these metrics, and developed ways of getting around the system: publish large numbers of less important papers, publish tentative or controversial results to generate citations, or join networks of collective authorships to increase publication counts with minor contributions. For these reasons, while publication metrics may provide useful indications about the importance and impact of a researcher’s work, they may be misleading and even discourage more innovative or risky lines of research.

The DORA declaration

With DORA, institutions commit not to use publication metrics, and base their decisions on expert evaluations of the actual scientific content of publications, also taking into account non-publication related research products such as data sets, impacts on policy and practice, museum exhibitions, etc. The DORA declaration also includes commitments to transparency regarding the decisions taken by academic institutions. Here are two key passages of DORA:

“ Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions. “

“ Be explicit about the criteria used to reach hiring, tenure, and promotion decisions, clearly highlighting, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published. “

The Swiss National Science Foundation (SNSF) signed the Declaration in 2014, and has since modified its funding attribution process accordingly in order to focus on scientific quality rather than publication metrics.

However, implementing DORA is not a solve-all solution. Its application requires vigilance, to break free of the tyranny of publication metrics without falling back into the arbitrariness against which the metrics were adopted in the first place. This implies a constant effort regarding quality and fairness in research evaluation. UNIFR has now taken one more step in this direction.

 Links:

- San Francisco Declaration on Research Assessment (DORA)

- DORA at SNSF