Brazilian Reproducibility Initiative publishes paper on lessons learned from the project

New preprint describes the challenges faced by the project and points to ways to strengthen reproducibility in lab biology

The Brazilian Reproducibility Initiative (BRI), a project that contributed to the creation of the Brazilian Reproducibility Network, has published a new preprint describing the main challenges faced over its seven years of work.

The project involved 56 labs and brought together more than 200 researchers to carry out 143 replications of experiments published over the past two decades by scientists affiliated with Brazilian institutions. Because of its national scope, the BRI offered a close-up view of how academic labs carry out replications using a confirmatory approach.

One of the main challenges identified was the lack of a common language for describing methods in lab biology. Even with detailed, preregistered protocols, different labs interpreted the same instructions in different ways. This variation reflects how technical knowledge in biology is passed on — mostly between peers or absorbed tacitly — and underscores the importance of developing and adopting shared conventions.

Another important lesson concerned validating whether replications were methodologically adequate to test the original result. The attempt to establish predefined validity criteria was not successful due to the difficulties labs had both in drawing up these criteria and in adhering to them later on. The solution was to create an independent validation committee that assessed procedures and protocol deviations without access to the results — a strategy that seems more feasible for individual labs to implement.

The paper also reflects on the limits of preregistration in experimental biology. For most methods in the field, researchers' main degrees of freedom lie in decisions made during data collection, not in data analysis. Since many of these decisions depend on intermediate observations made during the experiment itself, protocols are hard to preregister in advance, and the benefit of doing so appears smaller than in fields where most of the complexity lies in analysis. For this reason, the concept of preregistration needs to be adapted to reflect the reality of each research area.

The broadest lesson, however, is structural. Contingencies such as logistical problems and a shortage of human and financial resources — common and challenging, especially in contexts like Brazil's — end up having a major impact on labs' day-to-day routines. While these challenges were intensified by the COVID-19 pandemic, they are also compounded by the fact that academic research groups tend to be small, highly dependent on individual people, and not very robust to setbacks because of their own structure.

Beyond making labs less robust, this fragmented structure also hinders their ability to collaborate. A large-scale, multicenter project like the BRI requires a degree of hierarchical coordination that neither the participating labs nor the organizing committee were used to. The difficulties in implementing large-scale projects point to the importance of specializing and professionalizing both technical support teams at universities and project management structures.

The Initiative's first results

In early 2025, the BRI released its main results in a preprint, which was later revised in 2026. Starting from a sample of biomedical experiments published by Brazilian research groups, the project planned 180 replications, of which 143 were carried out and 90 were deemed valid by the validation committee created for the project, with between 20% and 44% of experiments successfully replicated. The results suggest that part of the low replicability reflects characteristics of the published experiments, but that technical variation between labs also plays an important role.

Einstein Foundation Award

In November 2025, the BRI received the Einstein Foundation Award, an international prize awarded by the Einstein Foundation Berlin in partnership with the BIH QUEST Center for Responsible Research. The Initiative was recognized in the institutional category. The award acknowledged the project's results, its impact on strengthening research practices, and the legacy it leaves for organizations such as the Brazilian Reproducibility Network itself and BRISA.

Read more: https://osf.io/preprints/metaarxiv/8y3tg_v1