Statistics Meeting Book (June 20, 2018)

STATISTICS There are several statistics which might be used to measure the degree of agreement or disagreement of the two methods. One might be a measure of total agreement, such as (a + d) / n, or a measure of total disagreement, such as (b + c) / n. Others might include a measure of association, such as ϕ. There are many others. We will concentrate here on the difference in proportions of positive results for the two methods: dPOD(A,B) = POD(A) – POD(B) (1) For Table 1, this difference is estimated (maximum likelihood, unbiased) by dPOD(A,B) = (a + b) / n - (a + c) / n (2a) = (b – c) / n (2b) Continuity-corrected versions of eq.(2) may be obtained by adding a constant ε to each of a, b, c and d in Table 1. Two such values of ε are commonly used: ε = ½ and ε = 1. For ε = ½, the estimate in eq.(2) becomes dPOD(A,B) = (b – c) / (n + 1) (3) For ε = 1, the estimate becomes dPOD(A,B) = (b – c) / (n + 2) (4) In order to derive large-sample (n >> 1) confidence intervals, some measure of the standard error of dPOD is needed. The simple Wald estimate is SE(dPOD) = √{ [ (b + c) / n – ((b – c) / n) 2 ] / n } (5) The simple Wald confidence interval is dPOD + z SE(dPOD) (6) where ‘z’ is the corresponding normal quantile, e.g., z =1.9600 for 95% confidence. The interval in eq.(6) is typically liberal (too narrow) and under-achieves its nominal confidence level. One ‘improvement’ that may be applied is to correct eq.(5) for bias and to allow for limited degrees of freedom. This improves results for small n. The new SE(dPOD) is given by SE(dPOD) = √{ [ (b + c) / n – ((b – c) / n) 2 ] / (n – 1) } (7)

3