Forum

Notifications
Clear all

Test Method Validation: What are they, When are they used

8 Posts
8 Users
0 Likes
761 Views
(@mmd55)
Posts: 80
Trusted Member
Topic starter
 

Where I work, there is always a discussion about if a test method being used in R&D of a product should be validated. Test method validation is used to identify the variation present within a test and determine if that variation is below an acceptable limit for the test to be used for official submissions to regulatory bodies.

I have also been told that if a test includes animal tissue, you should not try to have that test method validated. The reasoning is that tissue possesses inherent variability and that it does not play nicely with a process that is meant to identify sources of variability. 

Does anyone have any experience working with or conducting test method validations and understand how it relates to the process of design controls? Or have any further explanation I may have missed about why certain test methods are purposefully not validated?

 

Thanks,

Matt

 
Posted : 26/10/2021 5:18 pm
(@cruzdonato)
Posts: 30
Trusted Member
 

During my internship, I recently worked with a couple engineers in reevaluating test methods for a device they're currently developing. It's a mechanical-based device, so some of the tests are done via an Instron. For some of the tests, we needed to perform a Gage R&R (Repeatability and Reproducibility) and a Measurement System Analysis. The Gage R&R evaluates if the tool used to acquire the data, in this case the Instron, is outputting consistent data between samples which usually involves some statistical analysis to observe variability and repeatability. The Measurement System Analysis (MSA) is much broader and evaluates the tool used, the operators using them, and the rationale behind the passing/failing criteria of the test. Some test methods do not require a Gage R&R or an MSA if the data is more qualitative than quantitative, or in the example that you shared, if there is already inherent variability that can't be eliminated. 

It was important to run these analyses to validate the test methods and provide accurate, insightful data. Often times, the engineers would have to go back and change something in the protocol or create a whole other protocol for a particular feature/function. It's quite easy to get stuck in a "validation hole" and lose yourself in the data from what I've seen. 

 
Posted : 27/10/2021 10:06 am
(@jaf22)
Posts: 83
Trusted Member
 

I actually have a large amount of experience with test method validation as I recently completed my labs validation for our non-compendial testing. Test method validation is not needed for standardized testing, but is needed for all testing that is custom to an implant, instrument, or overall system. Its important to understand what the test is doing, whether that is quantitative or qualitative analysis, to determine the MSA run (Attribute or Variable). There are other ways to validate a test method but an MSA using a Gage R&R is the most typical way of validating test methods as there are usually guidelines set by the company within a production environment as these studies are run often to validate machining. One area to focus upon is the tolerancing set within your statistical software and making sure that your quality counterpart is also able to understand your logic as TMV has come up during audits as an area of focus. I would reinforce the fact that TMV is becoming more popular in industry and can be a large focus of a test lab's projects. 

 
Posted : 27/10/2021 9:40 pm
(@anthonynjit)
Posts: 78
Trusted Member
 

I do not have first hand experience with this, but from previous classes I have learned just about what you have stated. Due to complex nature of testing on animal tissues, you're study is susceptible to countless variables that can not all be tracked. This is in large part why the selective breeding of animals used for science is so important. Mouse are bred with or without specific strains of DNA such as C57BL/6 or BALB/C to test under certain conditions. It is assumed that the breeding and test conditions to raise these animals are correct and exact so that down the line when further studies are being completed there should be little to no question of their reliability.

Do you believe there should be validation of animal tissue studies? Is there an alternative way to validate the tissue "enough" to be considered correct?

 
Posted : 31/10/2021 6:46 pm
(@jadebowale)
Posts: 25
Eminent Member
 

Animal models to predict human conditions in biomedical research relies on the notion that basic processes are sufficiently similar across species to allow extrapolation. Animal model validity has similarities between the model and the human condition it is intended to model, but no formal validation of models is applied. The reason why is because devising a validation system for animal models is the lack of a clear gold standard with which to compare results. While a complete adoption of the validation approach for alternative methods is probably inappropriate for research animal models, key features, such as making data available for external validation and defining a strategy to run experiments in a way that permits meaningful retrospective analysis. Criteria that may need to be consider is location where sample sources, minimizing genetic mutations in specimen, and time span in which these resources are used after harvesting, standardized assay analysis specification, etc.

Varga, O. E., Hansen, A. K., Sandøe, P., & Olsson, I. A. (2010). Validating animal models for preclinical research: a scientific and ethical discussion. Alternatives to laboratory animals : ATLA38(3), 245–248. https://doi.org/10.1177/026119291003800309

 
Posted : 09/04/2022 3:27 pm
(@ag2265)
Posts: 70
Trusted Member
 

I agree with @jadebowale and how animal models are used in biomedical research to help predict reactions that would occur in the human body. An important factor to consider is that whenever animal testing is being conducted, it is not the first of its kind. What I mean by that is a clinical trial will always be based off of a previous study or studies depending on the nature of the research. Protocols have been passed down and there are specific criteria that is followed in every clinical trial to ensure efficacy and safety. Furthermore, certain animal models have been deemed the most similar to the human species in regards to certain parts of their bodies. For example, chinchillas have been used heavily in auditory testing as their ears have a similar structure and function to humans. The stringent process of designing a protocol for animal studies based upon previous work and literature somewhat acts as validation for the study. As you said, one cannot be 100% certain with the reactions that researchers might witness when conducting the research so it cannot be validated due to the high variability. However, with the guidelines in place for how clinical studies should be drafted and conducted the validation step is taken care of during that phase, compared to the testing phase.

 
Posted : 13/11/2022 3:26 pm
(@jbdoddo1)
Posts: 39
Eminent Member
 

@jadebowale 's response was very well phrased. I agree with her assessment that peer review and retrospective external benchmarking both are meaningful and substantial methods to validate an in vivo model. I believe the same principle can be applied in vitro as well. As you get into more into specifications regarding how the device behaves without consideration of the biological interface (mechanical testing, electrical reliability, etc.), then looking into appropriate standards by the national biomedical governing bodies (FDA, GMP, etc.), international bodies (ISO, etc.), or trade organizations (ASME, IEEE, etc.) become good playbooks and partners to validate your test.

 
Posted : 13/11/2022 4:24 pm
(@es446)
Posts: 78
Trusted Member
 

To answer on how test method validation relates to design controls, at my company we use a document called a Master Validation Plan (MVP). Basically, the MVP contains a table listing out all the different tests performed throughout the course of the manufacturing process, and provides the reference to its validation protocol. Certain tests do not require validation though, and in those cases, the rationale as to why validation is not required for that particular test is provided instead. For instance, we do not need to perform validation on a test which follows a compendial method from the USP (United States Pharmacopeia), because these USP tests are already recognized as reputable standards by regulatory bodies. Utilizing an MVP is a great way to ensure all the necessary validation activities have been completed, and if they haven't been completed, the reason why can be documented in a clear easy-to-find place within the DHR.

 
Posted : 13/11/2022 6:38 pm
Share: