147 / 274
Estimating Reproducibility from Proficiency Test Data
Are there ways to implement proficiency models into an experiment to estimate reproducibility
of a method? In order to experimentally estimate reproducibility, there must be an experiment
which is operating under reproducibility conditions. Reproducibility conditions are defined by
ISO as “conditions where test results are obtained with the
same method
on
identical test items
in
different laboratories
with
different operators
using
different equipment.”
The AOAC Committee on Statistics has agreed to the following strategy for accepting
proficiency test data as part of method validation.
1.
Preferred Approach: A collaborative study (CS) is designed to follow AOAC guidelines
with a study protocol developed in conjunction with AOAC Statistics Committee advisors.
2.
In the event a collaborative study cannot be organized, Proficiency Test (PT) data may be
used in partial substitution if the following important considerations are met.
a.
Acceptance criteria should be agreed on by the Expert Review Panel before the study
is performed, or before the data are analyzed and reported.
b.
Laboratories in a PT study or check sample program can be used only if they follow
the AOAC Method strictly, without modification.
c.
The minimum number of laboratories should be 8.
d.
Given the estimate for s(r), a 95% CI, and the degrees of freedom from replication,
report s(R), a 95% CI, and the degrees of freedom from reproducibility.
e.
Primary decision criterion for Final Action should be based on the upper limit of the
95% CI of s(R) and s(r).
3.
The statistical estimation techniques described in ISO 13528 are optimized for proficiency
statistics. For validation statistics, AOAC INTERNATIONAL requires the use of the
statistical procedures of ISO 5725, with the provision that no outliers be removed without an
assigned cause. Recommendations on estimation techniques are as follows.
a.
Collect PT data from check sample programs or other PT data available.
b.
Use only results from labs that admit to running the AOAC Method. Results from
other methods shall be removed from the data set.
c.
Do not trim, drop or exclude outliers except for justifiable cause such as an admission
from the laboratory that the method was modified.
d.
Use standard AOAC/ISO5725 ANOVA model to estimate reproducibility standard
deviation.
95