30 / 56
28
JCPSLP
Volume 15, Number 1 2013
Journal of Clinical Practice in Speech-Language Pathology
Results
Test-retest reliability
SLP Time 1 FOCUS
©
scores ranged from 68 to 283
(
M
= 165.09,
SD
= 67.53). SLP Time 2 FOCUS
©
scores
ranged from 74 to 287 (
M
= 165.86,
SD
= 65.54). There
was a significant, positive and very high correlation between
scores for SLP Time 1 and SLP Time 2 FOCUS
©
scores,
r
= .96,
p
< .001; 95% CI .90–.98, N = 22. See Figure 1 for
SLP Time 1 and SLP Time 2 scatterplot. This analysis
demonstrated that there was consistency and similarity of
FOCUS
©
scoring for the two time-points.
Inter-rater reliability
SLP 1 FOCUS
©
scores ranged from 65 to 336 (
M
= 240.83,
SD
= 58.51). SLP 2 FOCUS
©
scores ranged from 84 to 328
(
M
= 238.48,
SD
= 65.10). There was a significant, positive,
and high correlation between SLP 1 and SLP 2 FOCUS
©
scores,
r
= .90, p < .001; 95% CI .83–.94, N = 48. See
Figure 2 for SLP 1 and SLP 2 scatterplot. This analysis
demonstrated consistency and similarity of FOCUS
©
scoring for the two SLPs.
Inter-rater reliability
To establish inter-rater reliability, two SLPs independently
administered the FOCUS
©
for the same child. The FOCUS
©
was completed using Format-1 (i.e., one month apart) or
Format-2 (i.e., during an initial assessment lasting for one or
two sessions), based on the availability of SLPs at that site.
In both formats, one primary/lead SLP conducted the initial
assessments. Ninety-six inter-rater comparisons were
completed for inter-rater reliability procedures. For 84 of
these comparisons, two SLPs completed the FOCUS
©
within an 8-day period (using Format-2) and for the
remaining eight comparisons, two SLPs completed the
FOCUS
©
within a 1-month interval using Format-1.
For each of the 96 comparisons, the two SLPs
interacted with and observed the child separately. They
also had an opportunity to ask parents how the child was
communicating in everyday settings (e.g., “please describe
your child’s ability to participate at home/school/other”).
This was facilitated face-to-face for all 96 comparisons.
Both SLPs also had the opportunity to review the clinical
data collected during the assessment and contact the
parents via telephone for any additional questions that
followed from the parent interview.
Data analysis
FOCUS
©
test-retest and inter-rater reliability were
determined using bivariate Pearson correlations with
one-tailed significance tests. This analysis examines the
linear relationship between variables (e.g., scores) that are
normally distributed (Portney & Watkins, 2009). In this
study, the purpose of the Pearson correlations was to
evaluate the linear associations between SLP FOCUS
©
scores. These scores were normally distributed, with no
outliers. FOCUS
©
raw scores were calculated and entered
in the correlational analysis using the Statistical Program for
the Social Sciences (SPSS; PASW, 2009). For test-retest
reliability, scores were coded “SLP Time 1” and “SLP Time
2”. For inter-rater reliability, scores were coded “SLP 1” and
“SLP 2”. To classify the size of observed correlational
relationships, guidelines from Hinkle, Wiersma, and Jurs
(2003) were used where .00–.30 = little if any correlation,
.30–.50 = low correlation, .50–.70 = moderate correlation,
.70–.90 = high correlation, and .90–1.00 = very high
correlation. There was no study attrition or missing data.
300
250
200
150
100
50
Time 1
50 100 150 200 250 300
Time 2
Figure 1. Scatterplot illustrated for test-retest reliability (same
SLP at two different time points), r = .96, p<.001, r
2
= .92. Seven
SLPs and 22 children were included in the test-retest reliability
procedures.
350
300
250
200
150
100
50
SLP 1
50 100 150 200 250 300 350
SLP 2
Figure 2. Scatterplot illustrated for inter-rater reliability (two
different SLPs for the same child), r = .90, p<.001, r
2
= .81. Six
SLPs and 48 children were included in the inter-rater reliability
procedures.
Discussion
Clinical evaluation of communicative participation is not
usually completed during SLP assessments of children. The
dearth of measures available to capture these skills is a
primary reason for this gap in our understanding of the
real-world effectiveness of SLP interventions for children.
Measuring communicative participation is, however, very
much needed to evaluate the holistic impact of speech-
language therapy on children’s lives (McLeod & Threats,
2008; Washington et al., 2012).
To address the need for clinical measures of
communicative participation, the FOCUS
©
was developed
by a team of Canadian researchers. The FOCUS
©
was
designed to be used by SLPs and parents. Consequently,
the reliability for its use by SLPs and parents needed to be
established. There is prior evidence of the validity of the
FOCUS
©
as well as reliability for parental administration
(Thomas-Stonell et al., 2010). However, further evidence
supporting the reliability of the FOCUS
©
for SLP
administration was needed. This study provides data that
describe the reliability of SLPs’ ratings of young children’s
communicative participation using the FOCUS
©
.