Composite Risk Index (CRI) Methodology Update

Background

In 2019 the PRC published (in PRR2018) its preliminary concept of the Composite Risk Index (CRI) and the corresponding methodology that could be used to measure the performance of the European ATM system, either as a whole or of its individual entities (service providers, Member States).

Since then, the PRC has refined its CRI methodology to take account of local specific operating conditions and has published the updated and improved methodology in a Technical Note (Commission 2020).

This CRI methodology will be of interest to ATM key decision makers, policymakers and operations staff as well as to Safety specialists, because the PRC’s methodology by nature is scalable and therefore can provide high level information for decision makers as well as more in-depth knowledge about risk exposure to safety experts.

Why a Composite Risk Index?

Safety in aviation is paramount. Although safety procedures and policies have become well established, implemented and enforced, there is always a risk, however remote, of complacency creeping in. To ensure that this never happens, a CRI methodology can assist the key parties concerned to ensure that all aspects of safety management systems, for example policy, safety oversight, safety data reporting, pro-active actions to prevent safety threats, etc., are given top priority at all times.

A standard safety report with typical incident or accident trends may highlight some aspects. However, such safety reports do not always give an insight and overall impression of how safety is performing over time or enable comparisons with different States or stakeholders. It is from this perspective that the PRC has developed a CRI Methodology. It is not designed to replace reporting on specific aspects of safety. Its purpose is to assist key decision makers, policymakers, and operations staff as well as Safety specialists by providing an easy overview to monitor and possibly compare the safety risks, either for one provider or for a State to make an overview of annual developments. Ultimately, based on data availability, it can be used as a kind of benchmark between ANSPs, States or regions at one moment in time or over several years.

Problem: How can the total safety risk be measured?

Risk is the potential for mishaps or other adverse variation in the cost, schedule, or safety performance of the ATM system. Safety risk therefore can be explained as the potential for mishaps that could result in injury, fatality, equipment or system damage or total loss.

Conceptually, all safety programmes desire accurate safety risk quantification in order to provide a meaningful expression of risk. As there are typically multiple safety risks associated with a system or event, the quantification of total safety risk is a major challenge.

One possible way to define the total safety risk of any system is using the concept of a composite risk estimate (Burns 2016). Current methods of obtaining this composite risk estimate use summing techniques to add the individual risks and produce a single number (Burns 2016),(Nardo 2005). This method seems natural, however, it is often difficult to determine particular occurrence probabilities (e.g. when historical information is of limited time series) or to quantify their severity (e.g. when information in safety databases is missing). That makes the additive computation of risk difficult or impossible.

Moreover, although risk in general can be quantified, as it represents a combination of probability and severity of specific occurrence happening, the human perception of risk often influences how risk is addressed (Slovic 2016). For example, on the level of decision makers the risk perception does not necessarily map directly to probability and severity in a linear fashion. That makes computation of total risk additionally difficult and subjective.

What is the PRC’s proposal to measure total safety risk?

In PRR 2018, the PRC presented the preliminary concept of the CRI and a corresponding methodology that could be used to measure the performance of the European ATM system as a whole or also its individual entities (service providers or Member States).

This initial calculation of the CRI was mainly based on reported safety occurrences. More specifically, the CRI was represented as a cumulative risk value calculated aggregating all reported, assessed and severity classified key safety‐related incidents to form an index.

The measure of risk exposure was based on probability and severity that considers the human perception of equivalent risk. The overall idea behind the CRI was that the performance of the safety system can be analysed in three important broad categories: (1) the quality of the reporting system with reporting entity, (2) measured risks within the system, and (3) the human perception of risk.

Click here for an explanation of this preliminary CRI methodology including a working example

At the same time, the PRC also has highlighted that there could be possibilities to further improve the CRI by considering specific local operating conditions, airspace size, capacity and/or complexity. In order to improve the initial CRI methodology, further consideration of local operating conditions, airspace size and complexity of the airspace were taken into consideration. As a result, three additional components, besides the original one (Safety Data), were considered in the new methodology. The new CRI methodology includes hence four distinct components, or Intermediate Composite Indexes (ICIs):

  1. Safety data (with the following parameters: number and type of safety occurrences, severity classification, and human perception of risk),
  2. Traffic / exposure data (consisting of the following parameters: flight hours and airport movements),
  3. Complexity (namely adjusted density, structural index and Complexity index), and
  4. Reporting practices (described through parameters of reporting rate and reporting culture).

Figure 1 shows all components of the proposed CRI index.

CRI Methodology Components

Figure 1: CRI Methodology Components

Further Information

For more detailed information, please refer also to the PRC Technical Note explaining the further development of the methodology (Commission 2020). The technical note explains in detail several statistical methods used to model index weights, overall computation, logic behind it, its use and limitations, and lastly areas of further improvement and expansion.

References

Burns, M. 2016. Logistics and Transportation Security, a Strategic, Tactical, and Operational Guide to Resilience. CRC Press, Taylor and Francis Group.
Commission, Performance Review. 2020. “Updated Composite Risk Index (CRI) - Methodology.” EUROCONTROL. https://ansperformance.eu/library/prc_tn_2021_01_Composite_Risk_Index_Methodology.pdf.
Nardo, Saisana, M. 2005. “Handbook on Constructing Composite Indicators: Methodology and User Guide.” Organisation for Economic Co-operation and Development; OECD Statistics Working Papers.
Slovic, P. 2016. “Perception of Risk.” https://www.researchgate.net/publication/325954197_The_perception_of_risk.