When a product fails verification, the immediate reaction is often to assume a fundamental flaw in the design or materials. However, in many cases, the testing conditions themselves may need reevaluation. In the case of the bottle label development project, the failure occurred when labels detached during a 10-minute water submersion test. While this test is useful for ensuring durability, it raises the question: is this condition representative of how the product will actually be used? For example, if these bottles are stored in a standard laboratory setting or only exposed to occasional splashes, a prolonged submersion test may not accurately reflect real-world usage. Instead of outright failing the label, a more nuanced approach could involve tiered testing—submerging bottles for 2, 5, and 10 minutes to understand the failure threshold. This method allows for a more detailed analysis of how quickly labels begin to degrade, helping determine whether slight modifications (such as a stronger adhesive or a protective coating) would be enough to meet the required performance criteria.
Another factor to consider is the test setup itself. The verification test included a pre-cleaning step where bottles were wiped with 70% ethanol and air-dried. This step may unintentionally affect label adhesion by altering the surface properties of the glass. If the bottles used in real-world conditions are not cleaned with ethanol prior to labeling, this discrepancy could create an unrealistic test failure. Additionally, the water bath was heated to 37°C, which may accelerate adhesive degradation more than would occur at room temperature. A more practical approach might involve testing under multiple conditions—room temperature submersion, refrigerated storage, and intermittent wet exposure—to create a comprehensive performance profile. If a label fails under all reasonable conditions, then material changes should be explored. However, if it only fails under extreme conditions that are unlikely in practice, refining the test procedure rather than the product may be the best path forward. What do you think—should testing always mimic worst-case scenarios, or should it prioritize real-world usage conditions?
Keeping all these details in mind, I think a balance of both perspectives can be valid depending on the product and the function of that product. Mimicking worst case scenarios helps combat Murphy's law as much as possible. It is true that "anything that can go wrong, will go wrong." Especially with certain products that are produced in the tens of thousands and require users to handle them. There could be a number of operator errors that can be encountered. Although the lessons of human factors can be used to lessen the risk of operator errors, they could possibly still happen. That is a huge plus of mimicking the worse-case scenarios. However, I think in some other products like for the label example in this weeks simulation, that mimicking worst-case scenarios are not mandatory. It is extremely unlikely the user is going to give their glass bottle a bath in hot water. All normal label adhesives are prone to adhesive degradation in such a hot water bath. That being said, I think balance is the key. Do you want to make a reliable and professional product? One hundred percent. After all, that is the goal of all products whether its from labels to pacemakers. However, some tests may not represent operator error and other rare scenarios that will involve the product.
As has already been discussed, I agree that it's important to test a variety of conditions. This way, you can mimic more common/ "real world" settings and also test the limits of the product. With that being said, in the context of medical devices, it's critical to have proper labeling of products so that your consumer knows what you have assumed the "real world conditions" to be. For instance, in the product you have mentioned above, you are assuming the product will be used at room temperature and so there is no problem with the label failing at warmer temperatures. The key is to include this information on the label so that your consumers will not try and use it in warmer conditions. Some improper medical device usage can be extremely dangerous. Doing research on your target consumer ahead of time will also help determine what conditions you should test your product in. They will be able to confirm what the most important features of the product are.
Testing should strike a balance between worst-case scenarios and real-world usage conditions. While extreme conditions help define the absolute limits of a product’s durability, they may not always reflect how the product will actually be used. If a test is too harsh, it could lead to unnecessary design changes that increase costs without adding real value. On the other hand, if testing is too lenient, the product might fail in actual use.
In the case of the bottle labels, a prolonged submersion test may not be representative of typical use, especially if the bottles are mostly exposed to occasional splashes or short periods of moisture. A tiered testing approach—submerging bottles for 2, 5, and 10 minutes—would provide more useful data on when and why failure occurs. Additionally, adjusting the test setup to better match real-world conditions, such as skipping the ethanol pre-cleaning or using room temperature water, could lead to more accurate results.
Ultimately, testing should aim to simulate real-life conditions as closely as possible while still accounting for reasonable variations in use. If a product consistently fails under expected conditions, then design improvements are necessary.
In order to create a robust final product, it is important to understand the worst-case scenario outcomes. If a product undergoes rigorous testing of worst-case scenarios, it will be assumed to last under less severe and more realistic conditions. In my opinion, it is important to mimic the most severe conditions that a product may be exposed to, even if the likelihood of that scenario happening in real life is very low. If a product is not able to consistently pass testing, then severity can be reduced on a case-by-case basis according to risk level of failure. In the example of the bottle label, risk severity is relatively low even in the worst-case scenario, so it may be appropriate to perform less intense testing. While labels are unarguably important, the worst-case scenario here has low likelihood to cause a lot of harm to the user of the product. In a case where a device or drug that is going to come into direct contact with a patient is being tested, it may be more important to test every scenario because there is higher severity potential risk involved if that worst-case scenario were to occur.
I totally agree that testing conditions play a huge role in verification failures, and sometimes, the issue isn’t the design itself but how the test is set up. The temperature factor is a big one—37°C is definitely going to affect adhesive performance differently than room temperature or occasional splashes. If the bottles aren’t actually experiencing that level of heat in real-world use, the test might be harsher than necessary.
I also love the idea of tiered testing instead of a strict pass/fail at 10 minutes. Testing at 2, 5, and 10 minutes could give way more insight into when and how the labels start degrading. The ethanol pre-cleaning is another important detail—if that’s not standard practice for real-world use, it might be skewing results.
Instead of automatically assuming a design flaw, refining the test conditions seems like a smarter first step before considering costly material changes!
I personally believe that testing in extremes is important. For example, say the labels were only tested for what was deemed "practical application" and tests were never conducted on bottles wet for the time period. The labels pass without issue and are sent out. Then, during transport the bottles get wet, which could happen for any number of reasons. A bottle could break and leak or the shipping box could get wet in the rain during transit, leading to the labels all peeling off and all of the bottles having to be discarded. Or the labels could become wet inside of a lab. They could be placed near a sink and become wet, a bottle could break in storage, or even during use there could be a drip or spill onto the bottle. Since the bottle's labels were never tested to be wet for this duration, the labels would fall off. This could lead to an unknown number of mistakes or issues within the lab using the product, or at the very least the bottle would be discarded. A perfectly good product would be thrown out and both the company and the customer who purchased the product would be at a loss. Testing only for what is deemed "realistic" is a massive bias and will lead to product failures because you may not always know what is realistic for every single application and use a customer has for a product. To build upon that, accidents do happen and it is important to try and account for these accidents to ensure the product remains usable.
That being said, there are extremes that don't need to be tested, such as dropping the bottle off of a three-story building. This will likely never happen, and we know the bottle will break at heights long before that point is reached. However, drop tests and mechanical testing were absolutely done at some point in the bottle's development to know these outcomes and likely involved some sort of relative extreme.
No one wants a product that only works sometimes, under the right conditions. Everyone wants a product that they know they can always use and will not fail on them. Reliability is a massive part of a successful product and part of this is at the very least understanding how it functions under extreme conditions.
Verification testing is one of the most critical steps in the medical device development process. It is supposed to ensure that the product meets the design requirements, is compliant, and would work as expected. Unfortunately, many issues tend to arise that lead to verification failures. If these failures are not addressed properly, they can lead to delays, increased costs, and issues with regulations.
Insufficient or unrealistic test conditions is one of the most prominent reasons that lead to these failures. For example, if a medical device is to be used continuously in an operating room, it may be tested under optimal laboratory conditions which is not the case in reality. Stress factors like human interaction, operating times, and temperature changes are not accommodated for when the device is used in a clinical setting which leads to failure.
A well designed verification process reduces late stage costly changes and increases the chance of obtaining regulatory approval. The challenge emerges, however, in devising how much leniency will be deemed sufficient in meeting the altered requirements – excessive strictness can cause undue failures while overly accommodating testing conditions may mask important danger levels. What do you think companies should do regarding borderline verification failure? Is it better to set stricter criteria for testing to enhance the reliability of the device, or is it better to set a more lenient approach to conditions?
You bring up a really good point that I think is highly debated in this high stakes industry. Medical device developers will try to increase the scope and range the devices are usable to maximize usability and profitability. For example, in the case of Simulation 1, if the labels on the bottles are able to withstand higher levels of heat and longer periods of submersion, the product might be better marketable. Often times, expanding the range is not about the product itself. For example, the bottle might mostly be placed in a controlled laboratory environment, but the conditions might not be as controlled when it comes to shipping the product. In this case, testing worst case scenarios is important to understand what the upper and lower limits of the conditions the product can withstand are. Expanding the range of usability really matters when it comes to shelf life of certain medical products. Expanding shelf lifes helps make the product more marketable and valuable to the users. However, whatever claim you make on the product must be 100% true. For example, even if the product fails tests 1% of the time when put under certain conditions, you cannot claim that the product is fine to use in those conditions. In summation, expanding the range and scope of the product (such as testing worst case scenarios) has its benefits but should only be done when it is 100% true.
When Verification tests fail, I think that the best approach is to first evaluate whether the model still fills the need of the customer. The testing parameters could've been set incorrectly because they excluded a large amount of customers. In the case of the labels falling off, the temperature parameters need to align with various temperatures that costumers are storing them in, not just at high temperatures around 37C. In another case where a company is claiming the device to have a specific spec, then you can't change the test parameters if verification fails. This is because the device can't move further in the design process if it fails to meet what the company claims it to be.
One thing to consider with the testing of worst-case scenarios is the resources that get used within this verification process. Worst-case scenarios can involve a multitude of possibilities without specification, and in turn need a large amount of equipment and materials to test. This increases the time-line and money needed for verification tests, acting against the benefits testing can bring to a product. With this in mind, testing of a worst-case scenario of a real-world usage condition can be useful. Specifying usage conditions mitigate a large amount of the issues related to resources, timeline extensions, and costs. Testing worst case scenarios help predict possible outcomes that might arise from incorrect usage, transport issues, etc. They can also aid in identifying potential problems that exist in instructions and current product. However, in addition to this it is important to test the product to the specific situation it is designed for and instructed in the materials arriving to consumers with the product. If instructions are properly followed, pre-made criteria should still be met. Extremes, or worst-case scenarios can still exempt important factors that should be considered in determining if a product is successful. A great example of this is temperature, especially with how some materials can only be maintained within a specific temperature range, degrading outside of it.
