As of June 2024 we are official members of CoARA (Coalition for Advancing Research Assessment). CoARA is a global coalition of research organizations, funders, national and regional assessment agencies, among other types of organizations, mobilized to reform the way research products, researchers, and their institutions are assessed.
The Agreement on Research Assessment Reform (ARRA)
In 2022, an initial CoARA group, originally focused on European organizations, published the ARRA (The Agreement on Reforming Research Assessment). This agreement is based on the understanding that the evaluation of research products, researchers and/or research institutions needs to be rethought, mainly by avoiding metrics based on publications. With this, the negative impacts of pursuing publication in journals with a high impact factor, which include, for example, reduction of methodological rigor and the high cost for publications, can be avoided.
The 4 core commitments of the agreement are summarized in the box below. The need to diversify scientific contributions that are considered in evaluations and a more relevant role for qualitative evaluations to the detriment of the use of quantitative metrics are highlighted.
Core commitments of ARRA
(Agreement on Reforming Research Assessment)
Recognize the diversity of contributions to, and careers in, research, in accordance with the needs and nature of the research
Contributions to science are not just papers; other research products, research practices and related activities (supervision, training, etc.) must also be recognized. Likewise, careers in science other than the role of researcher must be recognized and valued.
Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators
Abandon inappropriate uses in research assessment of journal- and publication-based metrics, in particular inappropriate uses of Journal Impact Factor (JIF) and h-index
Avoid the use of rankings of research organizations in research assessment
Six supporting commitments are also included in the agreement, focusing on enabling change to new assessment systems and enabling continuous learning and use of evidence in decision-making.
Activities of the Brazilian Reproducibility Network
By becoming a member of CoARA, we are also signatories to ARRA. With this, we indicate that the BrRN shares the vision of need for reforms and adopts the commitments of the agreement for its activities. As we are not an organization that evaluates researchers, we can work on supporting our members to implement reform measures and on seeking evidence on the best ways to do so.
Our first activity in this direction has already been announced: in May we presented a list of recommendations for CAPES indicating that reforms should be made in the evaluation model for Brazilian postgraduate courses. The next steps should include recommendations aimed at the coordinators of these programs, with concrete actions that can already be implemented independently and suggestions of relevant tools and platforms for each area of knowledge.
Additionally, all BrRN members can contribute to CoARA working groups. Currently discussed topics include: open infrastructures for responsible evaluations, multilingualism and biases in research evaluation, experimentation in research evaluation, responsible metrics and indicators, recognition of high-quality peer reviews, among others. The complete list and more information about each group can be seen here. If you would like to contribute, please contact us to organize our contributions.
Other ideas for how BrRN can act are emerging and as we organize an action plan (over the next year) we are open to listening to our members and broader interested community. Contribute by writing to us via email (redereprodutibilidade@gmail.com) or participating in Zulip (www.reprodutibilidade.zulipchat.com).
Comments