Parma, Italy
June 2012

EFSA Journal 2012;10(6):2755 [92 pp.]. doi:10.2903/j.efsa.2012.2755

Type: Guidance of the Scientific Committee/Scientific Panel
On request from: EFSA
Question number: EFSA-Q-2010-01343
Adopted: 23 May 2012
Published: 13 June 2012
Affiliation: European Food Safety Authority (EFSA), Parma, Italy

Abstract

The European Food Safety Authority (EFSA) requested the Panel on Plant Health (PLH Panel) to provide guidance for the evaluation of the effectiveness of the options for plants and plant products for reducing the risk of introduction and spread of harmful organisms in the European Union territory. Two operational tools are presented: a checklist for evaluating a proposed risk reduction option (RRO) and a database of references corresponding to published guidance documents or experimental assessments of RROs. The checklist can be used by the Panel or the dossier-submitting parties to verify whether all required information is provided in support of a RRO, to quickly describe information supplied to EFSA and to identify major gaps in the data. Four types of RRO assessments are distinguished in the proposed checklist according to their purposes and characteristics: experimental assessment of the effectiveness of the option to reduce pest infestation in plant material/products under laboratory/controlled conditions; experimental assessment of the effectiveness of the option to reduce pest infestation in plant material/products under operational conditions; analysis of the applicability of the RRO; and assessment of the effectiveness of the option to reduce the risk of pest entry from an infested area to a pest-free area. The database of references is intended to assist the Panel in (i) identifying potential RROs for a given pest and plant material, and (ii) quickly retrieving relevant experimental data and guidance documents for assessing a proposed RRO. In addition, the current document provides recommendations for assessing RROs, specifically: on experimental design; on the use of statistical methods including approaches for studying uncertainty; on the use of quantitative pathway analysis and spread models describing their advantages and limitations; and on recommendations for general surveillance and specific surveys

Summary

The European Food Safety Authority (EFSA) asked the Panel on Plant Health to deliver guidance on the methodology for evaluation of the effectiveness of options for reducing the risk of introduction and spread of organisms harmful to plant health in the European Union territory.

This guidance document was prepared by the Panel to address mainly the quantitative evaluation of the effectiveness of risk reduction options (RROs). When data and/or information are available, the quantitative methods described in this document can be applied. When only limited or no data and/or information are available, the Panel performs qualitative evaluations that are briefly described in this guidance document. The Panel developed this guidance document to be used for the assessment of RROs together with the guidance on a harmonised framework for risk assessment (EFSA Panel on Plant Health (PLH), 2010a) and the guidance on the evaluation of pest risk assessments and risk management options prepared to justify requests for phytosanitary measures under Council Directive 2000/29/EC (EFSA Panel on Plant Health (PLH), 2009). The guidance provided in this document complements and does not replace the two above-mentioned documents when responding to requests for scientific advice on issues related to the evaluation of the effectiveness of options for reducing the phytosanitary risks within the EU in order to support the decision-making process under Council Directive 2000/29/EC.

Two operational tools are presented in this guidance document:

• a checklist for evaluating a proposed RRO
• a database of references of scientific documents presenting recommendations on how to assess RROs and/or describing experimental assessments of RROs.

The two tools have different purposes. The checklist includes a series of items that can be used by the Panel to check whether all required information is provided to support a RRO. Four types of RRO assessments are distinguished in the proposed checklist according to their purposes and characteristics:

i.experimental assessment of the effectiveness of the option for reducing pest infestation in plant material/products under laboratory/controlled conditions
ii.experimental assessment of the effectiveness of the option for reducing pest infestation in plant material/products under operational conditions
iii.analysis of the applicability of the RRO
iv.assessment of the effectiveness of the option for reducing the risk of pest entry from an infested area to a pest-free area.

The checklist can be used by experts to make a preliminary assessment of documents and data submitted to EFSA to support a RRO (e.g. a temperature treatment of plant material) and, more specifically:

• to quickly describe the information provided to EFSA (i.e. report and experimental results) to support a proposed RRO
• to identify major gaps in data submitted to EFSA
• to organise the work of the Panel when evaluating a dossier.

This checklist could also be used by the author of the submitted dossier or by the author of a pest risk analysis to verify whether all the requested data have been provided.
The second tool is a database of references corresponding to published guidance documents or experimental assessments of RROs.

The content of these documents is summarised in a table presented in Appendix B. This database of references can be used by the Panel to find some specific experimental results on the effectiveness of a given RRO, or to find guidance documents for designing RROs. Although this database does not intend to include all existing references on RRO assessment, it may help the Panel experts to quickly retrieve the relevant experimental data and guidance documents for assessing a proposed RRO, or for assessing a range of options in a pest risk analysis. It can also be used to identify potential RROs for a given pest and/or plant material.

Finally, based on the literature review described in this guidance document and on its own experience, the Panel is able to formulate several recommendations on the use of quantitative methods for assessing the effectiveness of RROs.

Recommendations on surveillance (as defined in ISPM No 5, Glossary of phytosanitary terms)

• General surveillance should evaluate the possible occurrence of a pest in an area, using all relevant (quantitative and qualitative) information on the current pest distribution in and near the area, the ecological conditions of the area, the presence of host plants and other potential pest niches, and the import and trade rates of host plant products in the area. The conclusion of general surveillance and a discussion of the level of uncertainty should be presented along with all information used to reach the conclusion.
• Specific surveys should be conducted to test an explicitly formulated hypothesis on the occurrence of a pest in an area and/or on its incidence. They should be performed on a statistical basis, using relevant quantitative and qualitative information on the area, the pest, the host plants and other potential pest niches. They should provide a conclusion on pest occurrence and the uncertainty of the conclusion, expressed as the confidence level to detect the pest above the threshold prevalence of the survey.
• Methodology to integrate the results from general surveillance and specific surveys should be implemented in cases in which a conclusion on pest occurrence is difficult to reach.

Recommendations on the design of experiments

• The checklist provided herewith should be used prior to, and during, the experimentation.
• The information requested in the checklist and pertaining to the plant and to the pest should be first as complete and precise as possible.
• The objectives (e.g. mortality rates, maximal pest density acceptable) and confidence levels of the tests should be clearly stated and, when relevant, compared with the current standards.
• A complete description of the experimental or observational design should be provided, including: variables used to measure effectiveness; factors influencing effectiveness that were or were not taken into account in the experiments; description of facilities and equipment; description of treatments; methodology followed for monitoring critical parameters; description of experimental design; presentation of the data; and description of the statistical analysis.

The complete datasets produced by the experiment and/or the observations and used in the analyses should be kept available with a full definition of all the variables.

Recommendations on the use of statistical methods for assessing the effectiveness of options for reducing pest infestation

• Uncertainty about the effectiveness of RROs should be studied by computing confidence intervals with classic statistical methods or credibility intervals with Bayesian methods.
• The probit 9 threshold for mortality rate should not be systematically used as the reference threshold for assessing the effectiveness of RROs. Instead of using a specific threshold for mortality rate, it is recommended that the risks of pest entry and establishment associated with the RRO under consideration be analysed.
• Although not frequently used in plant pathology, equivalence tests and, more specifically, non-inferiority tests are useful tools for comparing two RROs and testing whether a proposed RRO is at least as good as a currently implemented RRO.
• Depending on the nature of the available experimental results, different types of generalised linear models can be fitted to data to study the relationship between the dose of a treatment and its effectiveness. Such models are commonly used in chemical risk assessment, but are also applicable in treatment effect assessment.

Recommendations on the use of quantitative pathway analysis and spread models

Quantitative pathway analysis and spread models have several advantages compared with experimental and/or observational studies:

• They allow risk assessors to quantify the effects of RROs, singly or in combination, on several variables such as probabilities of entry, establishment and spread, or magnitude of impact. They do not restrict the assessment of RROs to their capabilities for reducing pest infestation.
• Quantitative pathway analysis and spread models can address uncertainties and can be used to study the effect of different sources of uncertainty on the risk of entry, establishment, spread and impact.
• They enable to perform a sensitivity analysis to identify the most influential parameters in a model that define the most effective RRO.

These advantages make these quantitative tools attractive for assessing the effectiveness of different RROs. However, their application can be difficult in practice owing to the amount of data required to develop such models. In the case of missing data, the uncertainty associated with the model outputs could be high and decrease the ability of the model to discriminate between different RROs, thus diminishing the usefulness and value of the models.
 

Website: http://www.efsa.europa.eu

Published: June 19, 2012