If an AI system identifies a potential quality issue but a QA professional disagrees with it, who should make the final decision and why ?
The QA professional should make the final decision. AI is a useful tool that speeds up analysis, but it is not fully reliable. AI in this scenario can lack data and context needed to reach a conclusion, and it can produce false positives, misread inputs, or reflect bias from its training data. The QA professional, by contrast, holds accountability for the outcome and understands the business, user, and regulatory context. For these reasons, AI should advise rather than decide. At the same time, the QA should also document why he disagrees with the AI, so the decision can be reviewed and the process improved over time.
I believe that the AI is a decent tool to use while performing QA, but I don't feel that it should override Human judgement from actual experts. QA experts have experience in the real world and can properly and accurately translate and understand all of the parameters needed for the proper assurance and instructions for the designated objectives.For example if a method has to aligned with a companies desired standards an AI model may have conflicts or be unaware of how to handle the entire span of the systems while a QA member who has experience within the system can make safer and reliable judgments as they have real experience with all of the tangible parts of the organization. Overall I feel AI is a great tool to support and maybe support viewing other perspectives but still is not a valuable source of QA that would override an expert.
I think the QA professional should make the final decision, but they should not just ignore what the AI system identified. AI can be very helpful for finding patterns or problems that a person might not notice right away, but I don't think it should completely replace human judgment, especially when patient safety could be involved. If the AI identifies a quality issue and the QA professional disagrees, I think the best approach would be to investigate why they are getting different results. The QA professional could review the data, check the requirements, and possibly involve other team members before making the final decision. This way, the AI is being used as another tool to help with quality rather than being the only one making the decision. Personally, I would feel more comfortable having a qualified person responsible for the final decision. If something goes wrong with a medical device, there needs to be someone who can explain why the decision was made. I think AI can support QA professionals and make their work more efficient, but the final responsibility should still stay with the people who understand the product and the situation.
If an AI system identifies a potential quality issue but a QA professional disagrees with it, who should make the final decision and why ?
In this context, I believe the QA professional should make the final decision. This is because the QA professional often understands additional context that the AI does not. For instance, the QA personnel would have a deeper understanding of the application of the device and what quality issues affect that application. They likely have insights from design, regulatory, and medical professionals that the AI system does not have.
In this case, the final decision should heavily depend on the level of risk involved instead of just choosing an AI system. If the AI flags a quality issue and a QA professional disagrees, then the disagreement is actually a bigger red flag issue that needs another level of review. For a low risk issue, then an QA may be able to evaluate the evidence and close it out themselves. However, if it is a case where it can affect patient safety and product performance, then it should be escalated to another reviewer or have an overarching team to analyze the situation before making a final decision. Furthermore, companies should make a procedure ahead of time on how the disagreements should be documented and handled. If there are real life situations similar to these, then people may trust the AI system when the conclusion matches what they believe. Reviewing how often an AI and a QA professional disagree can also help determine if the AI system needs to be improved or if there are trends that the employees have not noticed but the AI has. Since AI has become more involved with quality systems, in this case do you believe companies should make risk levels that require a second review when an AI and QA professional disagree with one another?
@aca In my opinion, having a tiered second review has merit and is consistent with how regulators view software in quality systems. The greater the impact of the AI tool output on patient safety or product quality, the more scrutiny and oversight should be given. Hence tying the need for a second review with the risk makes sense, I think.
There is one more point I’d like to add: you said that people tend to believe AI when it confirms their views, which is a problem by itself. Consensus may conceal biases, so companies should also analyze the instances of disagreement on the part of AI and quality assurance, especially when it comes to critical issues. It may be a good idea to provide the second reviewer with the evidence prior to giving them any information about the AI conclusion or the first review.
Monitoring the disagreements is a proper thing to do as it is consistent with the companies’ activities of dealing with trends and fixing repetitive problems. If there are disagreements between AI and quality assurance that have similar nature, it may indicate that the model needs to be fixed. This may refer to the model drift, and any update to the model should go through change control and be revalidated before it's used again.
I agree that the QA professional should probably have the final say, but I also think the disagreement itself shouldn't just be ignored after the decision is made. If the AI keeps identifying something that the QA person doesn't see, or the opposite keeps happening, I would want to know why.
Working in manufacturing has made me look at things like this more from the investigation side. If the same type of disagreement keeps happening, maybe the bigger question isn't really who is right each time. It could be something with the data going into the system, how the AI was trained, or even something in the process that people are interpreting differently.
I think AI could be useful not only for catching quality issues, but also for showing patterns that might otherwise be missed over time. If the AI and QA professionals repeatedly disagree on the same type of issue, at what point do you think that should become an investigation into the system itself?
The QA professional should make the final decision especially when it comes to quality issues. On the analytical side, the AI would be able to spot inconsistencies or small issues that may have been overlooked by accident. In terms of quality issues though, I believe that human input holds more value but it would take into account more nuance and reasons why things turned out as they did. This is not to knock on how much AI can greatly impact the quality assurance process with how quick it can sort through aspects of the project. Human judgement can not go understated though, a less analytical thought process can ask the extra question as well as see more value in things being left more ambiguous.
Like most other posts, I believe that the final decision should be made by the QA professional. From what I have learned, AI tools are increasingly being used in medical device companies, specifically in quality related activities but mostly as support rather than making the final decisions. It could be useful in risk analysis or documentation support.
The important thing to note is that the FDA is aware of software used in production and quality management systems and how it can involve automation. The FDA most likely has a guidance on how establishing confidence in this type of software through a risk based approach. I also think there is a risk of becoming too dependent on AI. Even if an AI system identifies a potential quality issue, a QA professional still needs to understand why the system reached that conclusion and determine whether it actually meets the appropriate requirements.