The San Francisco Declaration on Research Assessment (DORA) suggests a catalogue of measures and recommendations that are more apt in providing a fair and transparent assessment of research quality. For funding agencies in particular, DORA recommends the following list of practices in addressing research assessment (from a catalogue that also includes institutions, publishers, evaluating organizations, and individual researchers):
General Recommendation:
Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.
For Funding Agencies:
Be explicit about the criteria used in evaluating the scientific productivity of grant applicants and clearly highlight, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published.
For the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.
When evaluating research proposals HFSP places a strong emphasis on the quality of the applicants’ research and not on impact factor or journal in which the article was published. The suggested practices pertaining to HFSP are already in place in our application guidelines to applicants and review committee instructions. Therefore accepting the DORA standard does not add new procedures to our existing work flow and HFSP will add explicit statements in all application guidelines for the grant, fellowship, and CDA competitions as of 2015.
The full text of the San Francisco Declaration on Research Assessment (DORA) is available here.