Medical Device mechanical testing & sample size determination – Part one
When developing a Medical Device, pre-clinical testing is critical to quantify & qualify the performance of your product. BAAT Medical promotes to start testing early and often in the design process through medical device prototyping before going into verification testing. Additional benefits of creating and testing a physical device are early insights into manufacturing challenges and quick identification of the attention points in the design or isolated features.
In this article, part one, we will give you a general overview of sampling in relation to mechanical testing. In part two, we will provide a calculation example of the sample size determination for a static mechanical verification test.
The ultimate goal of early evaluation is that verification testing required for Medical Device Submissions can be met without surprises and will be accepted by authorities without questions or the need for costly re-testing. For a successful evaluation one needs to identify where to test, when to test, what to test, how to test and finally how many to test. This article will focus on the last point (i.e. sample sizes). The ability to withstand any forces acting on your product is a typical performance aspect that requires verification and benefits greatly from early evaluation. For this reason, sample sizes in relation to Mechanical Evaluation both in early development as well as testing for submission (i.e. verification testing) will be used as an example.
Mechanical Evaluation: Feasibility
The objective of early design testing, often referred to as feasibility testing, is to get to know your device. Sample sizes should be chosen by a medical device engineer such that the project team can assess and confirm their design choices. Sample size rationales should reflect that approach. Statistics can be used, but a justification could be as simple as explaining that a single prototype will be used to assess a working principle and why that is sufficient at this stage of the design process. If a test is destructive, consider only testing the part(s) of interest instead of the entire prototype or start with a Finite element analysis (FEA) to get a feeling for the performance and confirm the outcome in a test with 1 or 2 samples.
Although it may be tempting to use as little samples as possible while designing, consider that quantifying variability can be a goal in and of itself. An example of this are 3D printed products that will be created in a batch on a build plate. The product located in the centre is not identical to those at the edges or those in between. The composition and age of the input material will also have an effect in addition to many other factors in the printing process. As such it makes sense to collect a few samples from multiple batches or from multiple locations on the build plate to quantify just how much that variation is and if it significant to the performance aspect at hand.
A second objective of early testing (once you get closer to the final design) is that the more data you obtain regarding your result distribution, the easier it will be to make a statistically sound sample size rationale when it comes to verification testing.
If it turns out that batch manufacturing is a necessity and performance cannot be met by all samples, then the design or feature being evaluated should be re-assessed and this is something you want to know as early as possible.
A second benefit to using multiple samples (once you get closer to the final design) is that the more data you have obtained regarding your result distribution, the easier it will be to make a statistically sound sample size rationale when it comes to verification testing.
Mechanical Evaluation: Verification
The objective of verification testing is to gather documented evidence that the performance requirements for your intended use are met. A sample size rationale should therefore justify why the chosen sample size is appropriate for this objective. Ideally testing standards (ASTM, ISO) include a sample size rationale that you can refer to, otherwise statistical techniques should be used.
In general, when using a statistical technique to establish a sample size, the calculation is based on predicted performance and accuracy requirements (such as significance and confidence). Given this input, the minimum amount of test samples needed to reach a criterion within the given requirements can be established. The outcome of this calculation is confirmed after the test. To avoid additional testing, you therefore want your input information (i.e. results from the feasibility assessment) to be as accurate as possible.
Medical Device mechanical evaluation, prototyping and partnering with BAAT
When planning the mechanical evaluation for your Medical Device, inevitably someone will ask the dreaded question: How many samples will we need? Depending on who you ask, the answer will generally be variations of “what do we have laying around?”, “it depends” or “3, because we always use 3”.
Frustrating as it may be, “it depends” is the correct answer when setting up your medical device project. In fact, authorities (such as Notified Bodies and the FDA) will expect you to justify the selected sample size number in a rationale when it comes to verification testing.
BAAT Medical knows what the verification efforts are for Medical Devices and has successfully addressed concerns raised by authorities regarding sample sizes for submissions made by our clients in the past. We have a common-sense approach to testing and will advise to test early and often with medical device prototyping to avoid any surprises during verification testing. The goal is always to design a good device, but also to demonstrate without question that the design is safe and effective so that you, the patient, the medical specialists, and the authorities can be confident in its performance.
An in-house Test Facility, Subject Matter Experts for all relevant test fields and a wide network of certified test labs employing validated test methods ensures a first time-right approach when BAAT Medical is involved in your product design and verification.