Key principles
This text is a summary from the Supporting Opinion of the Uncertainty Analysis Guidance SO 7 and the Guidance on Uncertainty Analysis Ch 2.
Read more about different types of assessment.
Key principles when conducting case-specific assessments
A case-specific assessment is needed when there is no standardised procedure for the type of assessment in hand, and when parts of the assessment use standardised procedure but other parts are case-specific or deviate from the standardised procedure (e.g. for refinement or urgency), and for calibrating standardised procedures when they are first established or revised.
The uncertainty analysis should start at a level that is appropriate to the assessment in hand. For assessments where data to quantify uncertainty is available and/or where suitable quantitative methods are already established, this may be included in the initial assessment. In other assessments, it may be best to start with a simple approach, unless it is evident at the outset that more complex approaches are needed.
Uncertainty analysis should be refined as far as is needed to inform decision-making. – This point is reached either when there is sufficient certainty about the question or quantity of interest for the decision-makers to make a decision with the level of certainty they require, or – if it becomes apparent that achieving the desired level of uncertainty is unfeasible or too costly and the decision-makers decide instead to manage the uncertainty without further refinement of the analysis.
Refinements of the uncertainty analysis using more complex or resource-intensive methods and options should be targeted on those sources of uncertainty where they will contribute most efficiently to improving the characterisation of uncertainty, taking account of their influence on the assessment conclusion and the cost and feasibility of the refinement.
The characterisation of overall uncertainty must integrate the contributions of identified sources of uncertainties that have been expressed in different ways.
Key principles when using standardised procedures
Standardised assessment procedures with accepted provision for uncertainty are common in many areas of EFSA’s work and are subject to periodic review.
Most standardised procedures involve deterministic calculations using a combination of standard study data, default assessment factors and default values.
Using a standardised procedure can greatly simplify uncertainty analysis in routine assessments.
The documentation or guidance for a standardised procedure should specify
the question or quantity of interest,
the standardised elements of the procedure,
the type and quality of case-specific data to be provided and
the generic sources of uncertainty considered when calibrating the level of conservatism.
It is the responsibility of assessors to check the applicability of all these elements to each new assessment and check for any non-standard sources of uncertainty relevant to the question under assessment.
Any deviations that would increase the uncertainties considered in the calibration or introduce additional sources of uncertainty, will mean that it cannot be assumed that the calibrated level of conservatism and certainty will be achieved for that assessment.
Therefore, assessors should check for non-standard uncertainties in every assessment using a standardised procedure.
In assessments where none are identified, it is sufficient to record that a check was made and none were found. When non-standard uncertainties are present, a simple evaluation of their impact may be sufficient for decision-making, depending on how much scope was left for non-standard uncertainties when calibrating the standardised procedure.
Where the non-standard uncertainties are substantial or the standardised assessment procedure is not applicable, the assessors may need to carry out a case-specific assessment and uncertainty analysis.
Key principles for development or review of a standardised procedure
The level of conservatism provided by each standardised procedure should be assessed by an appropriate uncertainty analysis.
- This is to ensure they provide an appropriate degree of coverage for the sources of uncertainty that are generally associated with the class of assessments to which they apply.
The uncertainty analysis for the development or review of a standardised procedure
is conducted as for a case-specific assessment to evaluate the probability of meeting the defined requirements, but where the procedure is adjusted if necessary to achieve an appropriate level of probability.
This requires defining the management objective for the procedure, and how often and/or to what extent that objective should be achieved in the future standardised assessments where the procedure will be used.
The uncertainty analysis should consider all relevant uncertainties, including uncertainties about how the standard study designs used to generate data, and any default factors, assumptions, scenarios and calculations used in the assessment, relate to conditions and processes in the real world.
Consultation with decision-makers will be required to confirm that the level of conservatism is appropriate.
Key principles for urgent assessments
A scientific assessment requested to be completed within an unusually short period of time.
In some situations, e.g. emergencies, EFSA may be required to provide an urgent assessment in very limited time and the approach taken must be adapted accordingly.
Uncertainty is generally increased in such situations, and may be a major driver for decision-making.
Characterisation of uncertainty is therefore still necessary, despite the urgency of the assessment.
The approach to providing it must be scaled to fit within the time and resources available.
Every uncertainty analysis should express in quantitative terms the combined effect of as many as possible of the identified sources of uncertainty affecting each assessment.
When time is severely limited, this may have to be done by a streamlined expert judgement procedure in which the contributions of all identified sources of uncertainty are evaluated and combined collectively, without dividing the uncertainty analysis into parts.
This initial assessment may need to be followed by more refined assessment and uncertainty analysis, including more detailed consideration of the most important sources of uncertainty, after the initial assessment has been delivered to decision-makers.