148 / 274
Additional Considerations:
The Official Methods Board should be aware that under section 2 above, repeatability may not
be estimated, as within-laboratory replication in a PT study may not meet the ISO definition of
repeatability conditions. We recommend calculating within-laboratory variance as an
intermediate step to estimating reproducibility standard deviation, but the within-laboratory
variance term should not be called “repeatability.”
The above recommendations do not supersede the requirements set out in OMA Appendix D or
ISO5725 regarding number of matrices, number of levels, or levels per matrix.
Statistical estimates derived from PT studies should be used cautiously – any attempt to
extrapolate reproducibility estimates to different concentration levels or to different matrices
must be discouraged.
Proficiency data is used to certify laboratories to ISO 17025 standards. ISO 13528 is the
standard used for analyzing and reporting proficiency data for laboratory accreditation
purposes. The advantage of using PT data when possible is that it will allow estimation
of reproducibility on existing data sets. In addition, laboratories may be motivated to
participate in proficiency studies in order to satisfy the needs of accreditation bodies and
to give external feedback on laboratory performance. This is something that CS do not
normally provide to participating collaborators. Unfortunately, many laboratories around
the world have been forced to look to cut costs and volunteering to help validate a
consensus method may be seen as an unnecessary cost to a laboratory. In effect this
proposal will allow laboratories to contribute to validation efforts and be rewarded for
their contributions by obtaining a z-score for the proficiency study.
The main problem with using proficiency data to estimate reproducibility is the problem of
method control in proficiency test regimes. In the past, Proficiency Test (PT) data was all about
accrediting the laboratory, and not as much emphasis was put on the method. If PT data is to be
used, it can only come from laboratories that confirm they are using the candidate method
without modification. Other laboratories in the proficiency study may use other methods, but
only data from compliant labs will be used to calculate validation statistics for the candidate
method.
Another common issue when using PT data is that matrix and level decisions are generally not
made for PT studies the way they would be made for randomized, controlled Collaborative
Studies (CS). In a CS situation, the study director will meet with a group of advisors and plan
out the number of matrices, concentration levels, and design the experiment to cover a planned
design space. Proficiency systems very often use convenience samples commonly obtained at
96