Risk scoring has been, historically, a significant part of many risk management activities. It is a method for evaluating which risks are the most critical to control. Successive scoring of risk controls allows teams to see how the level of risk on the evaluated design, process, function, operation, etc. may improve over time.
Within risk exercises such as failure modes effects analysis (FMEA), risk scoring is a multiplicative process involving three factors: severity of harm (S), probability of occurrence (O), and detectability (D). These will yield a risk priority number (RPN) that, depending on the guidelines a business has in place, will determine whether a risk is acceptable or requires risk control measures. Because these scores are often expressed on scales between 1 and 10 (least to greatest), the RPN for FMEAs has a potential scoring range between 1 and 1,000.
To the layperson, the RPN and its range may seem like an ideal method of risk evaluation: because the scoring range is so large, it must be simple to delineate that high-scoring risks are more dangerous than lower-scoring ones. This is not necessarily true. The mathematics of the RPN are more complicated than they appear at first glance, and it is important to recognize the drawbacks before implementing the RPN into risk exercises like the FMEA.
Ordinal Scales, Intervals, and Limited Scores
Within the RPN exercise, the scoring scale is ordinal: each factor’s scale typically ranks between the integers of 1 and 10. These integers give numerical value and weight to the categories within each factor. The category “Could Result in Moderate Harm” under severity might be assigned a 5 or 6 value.
One of the major problems these numerical values pose within the RPN is interval-scale data. Both the direction of the numbers and their distance must be clear when scoring. It is understood that the distance between 1 and 2 is one integer; however, this does not describe the change in categorical value that happens within each factor. If S = 1 means the severity of harm is minimal and S = 2 means the severity of harm is minor, does the interval scale appropriately describe that categorical relationship? What are the defining characteristics of “minimal harm” versus “minor harm” that can be quantified in an ordinal scale? Determining the answers to these questions is a difficult exercise.
What compounds this problem is the effect business requirements have on defining the categories within each factor of the RPN. Severity, occurrence, and detectability are all features of the FMEA, but how to describe, define, and rank the categories of each is an interpretive exercise. Each organization has its own process for defining their RPNs; as a result, determining the interval scale is an issue that risk management teams have to reconcile.
Even if the interval scale is worked out and teams think that numerical values have been properly assigned to the appropriate category within each factor, there are mathematical issues that can complicate scoring and subsequent prioritizing. Despite the fact that the scoring range for the RPN is between 1 and 1,000, because it is a multiplicative process many of those potential scores are impossible to achieve. In reality, only 120 values exist within that scoring range that arrive as the result of the RPN. For example, prime numbers like 13 can never be scored. The theoretical range is limited in reality.
There are 1,000 possible scores, but they all become sequestered into the 120 possible values. This means that a score can be achieved more than once, confusing the prioritization process. For example, an RPN of 80 could be achieved by any of the following calculations:
S = 4 x O = 10 x D = 2
S = 8 x O = 1 x D = 10
S = 5 x O = 4 x D = 4
S = 2 x O = 5 x D = 8
Because all of these have equal weight in accordance with how an RPN is set up, how do you prioritize which risk to mitigate first? Score 1 is fairly likely to have a severe outcome, highly likely to occur, and hard to detect, whereas Score 2 is very likely to have a severe outcome, but hardly likely to occur and very easy to detect. Identifying which one of those risks becomes a priority based on its final RPN score becomes a subjective exercise even after an objective mathematical process; the very purpose of the RPN is subjugated by its own scoring limitations.
The RPN and Usability Engineering
The other problem the RPN encounters concerns usability determination. Design and process risks can be controlled during the product development process. Once a device reaches the end-user, however, risk evaluation becomes a much more complicated process.
Because users are human, they may operate or interact with the device in unintended ways, potentially exposing operators or patients to harm. Depending on who is using the device, in what environment, and what the intended use is, different hazards, hazardous situations, and harms can arise. These factors all complicate the RPN. For example, teams may identify that a risk with a syringe is the needle breaking under the skin. The severity ranking will likely go unchanged, but its probability of occurrence changes with the user. If a skilled nurse is handling the syringe, there is low probability of that harm occurring. However, if the nurse’s capabilities are impaired by exhaustion or they are operating the syringe in an abnormal use environment, that probability could rise. These factors could also exacerbate the harm, affecting its severity level.
Detection scoring is also impacted by these factors. Detecting a defect in manufacturing that could cause harm is often straightforward; detecting harm resulting from use error is not. Users will not be able to anticipate how their use leads to a harm until it is too late. As well, they could put themselves in hazardous situations that don’t necessarily result in harm. Detection therefore becomes an arbitrary mechanism that depends on too many other things to be a reliable scoring factor.
Drawing Meaningful Data from the FMEA
The RPN is not necessarily an invalid tool for the FMEA process. It gives risk management teams insight as to how severe a risk can be. However, it suffers from its own limitations and has the very real potential to draw attention to scored risks that are not necessarily as critical as others. Many standards and regulatory bodies now advocate for a primary focus on severity of harm within FMEA exercises. Analyzing how severe a harm will be can help teams better prioritize their risk control processes. The RPN can get teams to that point, but it is up to them to draw meaningful conclusions from the data that the RPN presents.
Learn More About Cognition's Risk Management Offerings
Download our white paper on the Cognition PHA Templates for product development.