JCPSLP VOL 15 No 1 March 2013

Results Test-retest reliability

Inter-rater reliability To establish inter-rater reliability, two SLPs independently administered the FOCUS © for the same child. The FOCUS © was completed using Format-1 (i.e., one month apart) or Format-2 (i.e., during an initial assessment lasting for one or two sessions), based on the availability of SLPs at that site. In both formats, one primary/lead SLP conducted the initial assessments. Ninety-six inter-rater comparisons were completed for inter-rater reliability procedures. For 84 of these comparisons, two SLPs completed the FOCUS © within an 8-day period (using Format-2) and for the remaining eight comparisons, two SLPs completed the FOCUS © within a 1-month interval using Format-1. For each of the 96 comparisons, the two SLPs interacted with and observed the child separately. They also had an opportunity to ask parents how the child was communicating in everyday settings (e.g., “please describe your child’s ability to participate at home/school/other”). This was facilitated face-to-face for all 96 comparisons. Both SLPs also had the opportunity to review the clinical data collected during the assessment and contact the parents via telephone for any additional questions that followed from the parent interview. Data analysis FOCUS © test-retest and inter-rater reliability were determined using bivariate Pearson correlations with one-tailed significance tests. This analysis examines the linear relationship between variables (e.g., scores) that are normally distributed (Portney & Watkins, 2009). In this study, the purpose of the Pearson correlations was to evaluate the linear associations between SLP FOCUS © scores. These scores were normally distributed, with no outliers. FOCUS © raw scores were calculated and entered in the correlational analysis using the Statistical Program for the Social Sciences (SPSS; PASW, 2009). For test-retest reliability, scores were coded “SLP Time 1” and “SLP Time 2”. For inter-rater reliability, scores were coded “SLP 1” and “SLP 2”. To classify the size of observed correlational relationships, guidelines from Hinkle, Wiersma, and Jurs (2003) were used where .00–.30 = little if any correlation, .30–.50 = low correlation, .50–.70 = moderate correlation, .70–.90 = high correlation, and .90–1.00 = very high correlation. There was no study attrition or missing data.

SLP Time 1 FOCUS © scores ranged from 68 to 283 ( M = 165.09, SD = 67.53). SLP Time 2 FOCUS © scores ranged from 74 to 287 ( M = 165.86, SD = 65.54). There was a significant, positive and very high correlation between scores for SLP Time 1 and SLP Time 2 FOCUS © scores, r = .96, p < .001; 95% CI .90–.98, N = 22. See Figure 1 for SLP Time 1 and SLP Time 2 scatterplot. This analysis demonstrated that there was consistency and similarity of FOCUS © scoring for the two time-points. Inter-rater reliability SLP 1 FOCUS © scores ranged from 65 to 336 ( M = 240.83, SD = 58.51). SLP 2 FOCUS © scores ranged from 84 to 328 ( M = 238.48, SD = 65.10). There was a significant, positive, and high correlation between SLP 1 and SLP 2 FOCUS © scores, r = .90, p < .001; 95% CI .83–.94, N = 48. See Figure 2 for SLP 1 and SLP 2 scatterplot. This analysis demonstrated consistency and similarity of FOCUS © scoring for the two SLPs.

350

300

250

200

SLP 1

150

100

50

50 100 150 200 250 300 350 SLP 2

Figure 2. Scatterplot illustrated for inter-rater reliability (two different SLPs for the same child), r = .90, p<.001, r 2 = .81. Six SLPs and 48 children were included in the inter-rater reliability procedures. Discussion Clinical evaluation of communicative participation is not usually completed during SLP assessments of children. The dearth of measures available to capture these skills is a primary reason for this gap in our understanding of the real-world effectiveness of SLP interventions for children. Measuring communicative participation is, however, very much needed to evaluate the holistic impact of speech- language therapy on children’s lives (McLeod & Threats, 2008; Washington et al., 2012). To address the need for clinical measures of communicative participation, the FOCUS © was developed by a team of Canadian researchers. The FOCUS © was designed to be used by SLPs and parents. Consequently, the reliability for its use by SLPs and parents needed to be established. There is prior evidence of the validity of the FOCUS © as well as reliability for parental administration (Thomas-Stonell et al., 2010). However, further evidence supporting the reliability of the FOCUS © for SLP administration was needed. This study provides data that describe the reliability of SLPs’ ratings of young children’s communicative participation using the FOCUS © .

300

250

200

150 Time 1

100

50

50 100 150 200 250 300 Time 2

Figure 1. Scatterplot illustrated for test-retest reliability (same SLP at two different time points), r = .96, p<.001, r 2 = .92. Seven SLPs and 22 children were included in the test-retest reliability procedures.

28

JCPSLP Volume 15, Number 1 2013

Journal of Clinical Practice in Speech-Language Pathology