Navigation und Service

Critical appraisal tool (CAT) for the evaluation of evidence from animal observational epidemiological studies and Test of a risk modelling tool (Rrisk) in the context of EFSA - SA13 (EFSA FPA SA 013)

11/2022-07/2024

Funding programme / funding institution: Europäische Behörde für Lebensmittelsicherheit (EFSA) - Italien

Grant number: GP/EFSA/AMU/2020/02-SA13

Project homepage: -

Project description:

Title: Critical appraisal tool (CAT) for the evaluation of evidence from animal observational epidemiological studies and Test of a risk modelling tool (Rrisk) in the context of EFSA

Tasks:

Evidence appraisal is a fundamental step in the EFSA process for evidence use since it allows weighing the evidence when it has to be assessed/integrated. Along the years, systematic evidence appraisal has been performed at EFSA in several scientific assessments, with the use of Critical Appraisal Tools (CATs) as a support. The latter consist of a pre-defined list of the possible sources of bias for the studies selected for the assessment, guidance for judging the risk of bias arising through each of them and a narrative or quantified expression of the judgement. In the ‘Draft for internal testing Scientific Committee guidance on appraising and integrating evidence from epidemiological studies for use in EFSA’s scientific assessments’ one of the preliminary recommendations is: ‘Risk of bias tools have a long history of use for randomised controlled trials in humans. There is room for further development of these tools to capture the differences of different observational designs and use for other populations (e.g. animals). It is recommended that EFSA collaborates at the European and international level with relevant organisations and initiatives to harmonise developments in this area’.

The working group EvaRisk (evaluation methods for epidemiologic evidence in risk assessment) of the BfR Committee for Evidence-Based Methods in Risk Assessment is aiming at the development of a tool for rapid assessment of risk of bias (raRoB) for evaluating the evidence from individual human observational epidemiological studies in the risk assessment context. EvaRisk is currently developing a draft prototype tool for quick and transparent assessment;ng from this first draft of the tool, the specific agreement 05 aims at supporting the further development, testing and finalisation of a validated tool for evaluating the evidence from individual human observational epidemiological studies. This tool is expected to be available for all EFSA risk assessments for which such epidemiological evidence exists, particularly in the areas of pesticides, contaminants, nutrition, food additives etc.

A need for testing the applicability of the main tool to the evaluation of observational studies on animal populations in animal health and welfare assessments has been identified; one of the objectives of this specific agreement will be to assess such applicability and to  test and adapt the tool for this purpose, if necessary This rapid, standardised, and fit-for-purpose tool would allow the evaluation of evidence from individual animal observational epidemiological studies more efficiently and in a harmonised way, leading to more consistent and transparent appraisals of epidemiological studies for use also in EFSA’s risk assessments.

Dissemination of the tool to EFSA staff and experts will be achieved through presentations at Panel/Unit meetings. Other dissemination activities may also be implemented.

In many risk assessments the collection of evidence will not stop with the evaluation of a single study. Instead, information from several studies will be combined; conceptual and quantitative models play an important role in such an integration. With the Shiny Rrisk software the BfR developed another tool for the definition, implementation, and full transparent documentation of probabilistic risk models. The tool can quantitatively integrate the knowledge and uncertainties about elements of the model. It allows the modeller to specify a model from scratch and thereby fills a gap where standard models do not exist. The Shiny Rrisk solution is user-friendly and allows the implementation of models via its Shiny interface more easily; this broadens the circle of experts carrying out these risk assessments. Therefore, the second objective of this specific agreement will be to test the existing and developed Shiny Rrisk tool for use also with view of the requirements from EFSA and to validate as well as to propose/implement necessary adaptations. For this (at least three) case studies will be prepared, envisaging the re-implementation of models (or parts of models) from existing EFSA opinions or other sources as agreed upon. Additional user interviews should explore the user-friendliness from an EFSA viewpoint, including the reporting needs of EFSA. Finally, the possibility to host the tool on the R4EU platform will be explored.

Up