Discussion Topic: Risk Analysis methods
Depending on the project, a number of different risk analysis methods are available, but like anything, none of them are perfect. If one of them was, the product that used that method would never have issues but reading the headlines proves that has yet to happen.
For example, many risk analysis methods, like FMECA, sometimes use a numbered scale to categorize the risk. Some will use a scale of 1 to 5 and others will use a scale of 1 to 3. At the end of the day, the problem with this is that the numbers are arbitrary and not quantifiable in any direct fashion. There's subjectivity to them.
What are some other drawbacks of the risk analysis methods you have come across? Read about a method and describe its pros and cons.
Spiral Medical Development
Preliminary Hazard Analysis is a tool of analysis based on applying prior experience or knowledge of a hazard or failure to identify future hazards, hazardous situations and events that might cause harm, as well as to estimate their probability of occurrence for a given activity, facility, product, or system. The tool consists of: (1) the identification of the possibilities that the risk event happens, (2) the qualitative evaluation of the extent of possible injury or damage to health that could result, (3) a relative ranking of the hazard using a combination of severity and likelihood of occurrence, and (4) the identification of possible remedial measures.
•Helps ensure that the system is safe
•Modifications are less expensive and easier to implement in the earlier stages of design
•Decreases design time by reducing the number of surprises
•Hazards must be foreseen by the analysts
•The effects of interactions between hazards are not easily recognized
As a nuclear engineer, I have experience working at nuclear plants in probabilistic risk assessment. At one particular nuclear plant, they use bayesian updating to calculate the risk of a component failing in the nuclear plant. They basically collect data by starting the component up and every time the component starts up initially, that's a pass and every time it stalls, that's a fail. They use this data along with bayesian updating to calculate the risk of failure of the component and assess when the component needs to be replaced. The pros of this method is that is uses actual data from the component, and when properly done, it is at least somewhat accurate. The problem I have with this particular nuclear plant using this method is that it didn't appear to be standarized. They used an excel spreadsheet to do the calculations and I would constantly find errors in the spreadsheet. They had nothing in place to find these errors. They could have resolved it by having multiple people reviewing the calculations and redoing the calculations to ensure that they all came to the same answer. Instead, they had a manager who knew little to nothing about the actual calculations sign off on the calcs. As an actual nuclear engineer, I understand that a nuclear plant is designed to have failures of components and still function properly. There is no risk to the public or the actual plant due to this but it is just one step closer to being a problem to the public or actual plant due to them not reviewing their work.
A risk analysis is simply a careful examination of what, in your work, could cause harm to people, so that you can weigh up whether you have taken enough precautions or should do more to prevent harm. Workers and others have a right to be protected from harm caused by a failure to take reasonable control
measures. Accidents and ill health can ruin lives and affect your business too if output is lost, machinery is damaged, insurance costs increase or you have to go to court. You are legally required to assess the risks in your workplace so that you put in place a plan to control the risks.
Identify the hazards
Decide who might be harmed and how
Evaluate the risks and decide on precautions
Record your findings and implement them
Review your assessment and update if necessary
Fault Tree Analysis(FTA); which is used to give a graphical representation of the reliability of a system in order to predict the performance of its elements and identify any key poor performers that need further designing.And maintains the overall life cycle cost of keeping a system available.
1. Allows a clear representation of the process.
2. Pin points any failings within the system that could cause complete failure without additional barriers in place.
3.Very important in industries where a failure will have catastrophic results.
4.Its main advantage is in the use of simple equations of probability for drawing the comparison.
1. FTA is of limited value on non-critical systems and systems with multiple states of failure.
2. Not effective where the repair rates vary with the transient nature of failure.
Currently, I have been working on updating a pFMEA for Maquet products and they follow a similar 1 to 5 scale to numerically quantify a risk. A pFMEA is a process failure mode effects analysis which breaks down each process for each major product line. Within each process their are set protocols or procedures along with controls. Each process for each product will identify and evaluate any potential failures of that process. The scale for determining a risk has a set outline but it is very subjective. A scale of 1 can be "not that harmful with a low possibility of occurring." But that is based on the person identifying the risk. Each risk has to have statistical support for its claim. Meaning, for each numerical scale their is a 95% or 99% confidence interval which it is analyzed against. Keeping a pFMEA up to date is a project in itself because it requires that all protocols be updated along with controls. The changes in these protocols and controls may also affect the risk score in analyzing the process.
A collection of some drawbacks experienced by most risk analysis methods:
1. Slight variations between similar projects/medical devices. Difficulty estimating variations and their magnitude and impact.
2. Synergistic effects of several, combined risks. 1 + 1 = 3.
3. Information gaps. Many of these methods were developed decades ago.
4. Telling whether a risk is major or minor.
5. For experiments, there are no universal standards (lb. vs. kg).
6. Are often complex and need a trained profession to perform and understand.
Different risk analysis methods like FMECA,FMEA includes (FMEA - PFMEA, Design FMEA - DFMEA), Root Cause Analysis (RCA),RCM (Reliability Centered Maintenance), and Risk and control self assessment (RCSA).
For example: FMEA Failure mode and effects analysis.
- FMEA is a procedure for determining where processes, products or designs are most likely to fail and why.
FMEA is a highly effective way to evaluate products. FMEA enables early identification of single failure
points and system interface problems that can impact safety.
- Identification of critical areas of the system (Root Cause Analysis).
- Basis for ISO 26262
- Using the FMEA method, an RPN (Risk Priority Number) is calculated by multiplying the three categories of Severity, Occurrence and Detection. Each one assigned a value between 1 and 10, where 10 is high.
- Failure and reliability rates are particularly difficult to estimate when human performance is involved; which could be considred as the major disadvantage.
- Severity and occurrence ratings are often difficult for individuals or teams to estimate. Much time can be spent in discussion.
For successful risk management, FMEA should be regularly updated as new potential failure modes are identified and corresponding control plans are developed.
Root cause analysis (RCA) is a method of problem solving used for identifying the root causes of faults or problems. RCA has ‘general’ principals associated with it throughout many articles.
1. Define and describe properly the event or problem
2. Determine the causal relationships
3. Identify effective solutions
4. Implement and track solutions
Detailed methodology as to how to does this type of analysis. There are solutions which are to be tracked throughout the problem begin fixed which helps mitigate the risk of failure.
From the definition above it sounds as though a problem occurred already. Then based on the problem how do we fix it. The issue with this analysis is that you take a risk in having to wait for a problem to occur in order to fix it. There is high risk associated with this analysis because this works more for post-problems. This isn’t a pre-risk type of analysis unless it’s used for previous problems and set in place before a new project starts.
The Monte Carlo Simulation can be used to model uncertainty in inputs or any problem having a probabilistic interpretation. It can be used to create prices for assets based on an average of hundreds to thousands of hypothetical scenarios.
Because this analysis is ultimately a statistical test, many of its drawbacks come from the assumptions that must be made to do it. One assumption is that these simulations are built around a specific type of statistical distribution. Using the incorrect distribution leads to meaningless results. Another example is that assumptions must be made about the inputs for this analysis. Selection of the inputs related to the probability of the success of what is being analyzed is a key issue that determines the usefulness of the simulation.
Risk identification can be a difficult task because it is often highly subjective, and no unerring procedures available that may be used to identify risk events and scenarios other than relaying heavily on the experience and insight of key project personnel.
Project managers should be prepared to perform different types of risk analysis. For many projects, the quicker qualitative risk assessment is all you need. But there are occasions when you will benefit from a quantitative risk analysis.
Quantitative risk analysis is a numeric estimate of the overall effect of risk on the project objectives such as cost and schedule objectives. The results provide insight into the likelihood of project success and is used to develop contingency reserves. Projects that require a Contingency Reserve for the schedule and budget. Large, complex projects that require Go/No Go decisions (the Go/No Go decision may occur multiple times in a project). Projects where upper management wants more detail about the probability of completing the project on schedule and within budget.
Just to elaborate more on the Fault Tree Analysis, which has been discussed above and which is commonly used in the medical Device Industry. The FTA is a qualitative model that can be assessed quantitatively. It can be applied to new products before their design details are available and to already existing products. This is an effective top-down approach. The team starts with the undesired consequence or top event and identifies the initiating and contributing events that must occur to produce it. These events are combined using logic gates. A logic gate is the point at which two or more independent events are combined to produce a higher-level event. The logic gate determines whether the sub event probabilities or frequencies should be multiplied, for an and gate, or added, for an or gate. If all events under a gate are necessary for the higher event to occur, an and gate is used. If each of the events is sufficient to produce the higher event on its own, an or gate is used. Both mechanical failures and human errors can readily be included in a fault tree. If failure rates for each event on a fault tree are available or can be estimated from generic data, the top-event frequency can be calculated and compared to a company's internal risk-acceptability criteria. A fault tree is a powerful risk-analysis tool, but its greatest limitation is the availability of relevant failure data. Therefore, fault trees are generally best used to compare risks of various alternatives. The greatest benefit of a fault tree is that events that contribute most frequently to the top event can readily be identified, and mitigating measures can be focused on reducing the frequency of these events.
Reference: Basic Principles of Risk Management:www.wipro.com
The drawbacks of risk analysis are the rating scales that can be added or changed based on the company's preference. This is the same for risk impact and risk likelihood, which makes the ratings seem irrelevant. However, they do have benefits. The pros of qualitative analysis are its simple assessment methods, lack of need to determine the frequency, and no need to quantify the impact on cost and schedule since qualitative risk analysis doesn't quantify the risk impact on project costs, and schedule is saved during the analysis. The cons are that evaluations are subjective, lack differentiation within risk categories, and vague results.