JCPSLP Vol 17 No 3 2015

procedures, whereas competence measures refer to the clinician’s level of skill in delivering the intervention to a client. Compliance measures recommended for interventions supporting children’s communication skills include the use of a treatment manual, specific information regarding dosage, and checklists to support fidelity to procedures (Kaderavek & Justice, 2010). Monitoring competence is more challenging, as it requires an evaluation of the manner in which a treatment is delivered. Direct fidelity measures that involve observing a clinician implementing an intervention, or reviewing video footage of the implementation at a later date, are considered the “gold standard”. Indirect fidelity measures such as self-report checklists, logs, and practitioner surveys may be easier to implement but may not be as reliable (Kaderavek & Justice, 2010). Three examples of manualised programs which are recommended interventions for New Zealand SLPs supporting children with speech, language, and communication difficulties (Ministry of Education, 2013) include information about treatment fidelity procedures. These are: the Lidcombe Program for Early Stuttering (Onslow, Packman & Harrison, 2003), the Hanen Parent Program – More than Words (Sussman, Honeyman, Lowry & Drake, 2013), and the Picture Exchange Communication System (PECS; Frost & Bondy, 2002). The Lidcombe Program (Onslow et al., 2003) is used to treat stuttering in young children by teaching parents to use behavioural strategies during interactions with their children. A study of SLPs implementation of this program in Australian community clinics indicated successful outcomes and high levels of treatment fidelity (O’Brian et al., 2013). These researchers concluded that this program was at a phase 4 level on Robey’s (2004) framework for evaluating evidence. The framework ranges from emerging levels of evidence with phase 1 and 2 interventions indicating potential therapeutic effects, phase 3 demonstrating evidence from clinical trials which have controlled for internal validity, and phase 4 interventions have demonstrated effectiveness for specific populations in “real-world” conditions (Robey, 2004). The Hanen Parent Program – More than Words (Sussman et al. 2013) supports parents of children with autism spectrum disorders (ASD) to facilitate communication development through their everyday parent–child interactions. A randomised control trial indicated some significant changes in the way children in the treatment group interacted with their parents (Carter et al., 2011) which may indicate this program has a phase 4 level of evidence. The Picture Exchange Communication System (Frost & Bondy, 2002) focuses on promoting the initiation of communication using symbols and pictures. A meta-analysis of PECS highlights its effectiveness in real-world contexts (Flippin, Reszka & Watson, 2010) which implies that this may also be a phase 4 intervention. To ensure context fidelity, these programs require clinicians to complete 2–3-day training workshops in which they are provided with information to help them select appropriate client groups, recommendations for the size of client groups (Sussman et al., 2013), and recommended dosage for achieving successful outcomes (Onslow et al., 2003; Sussman et al., 2013). This information enables clinicians to match their approaches to client selection and service provision to those documented in research studies.

Compliance fidelity measures include the use of a treatment manual. The More than Words program (Sussman et al., 2013) offers a checklist to support reflection on a core part of the program: home-video coaching visits. Parent evaluation forms are also part of the Hanen More than Words program (Sussman et al., 2013) and feedback on these forms may help with monitoring compliance fidelity. To deliver Hanen parent programs, SLPs must be certified by the Hanen centre, and have a contractual agreement to deliver the programs in the specified manner and to keep up to date with program developments by participating in professional learning activities provided by the centre. However, other types of compliance measures are lacking in these programs including the direct observation of practitioners delivering the programs, self-report surveys of SLPs’ practice, or checklists to support practice. Although no competency measures are reported for any of the programs outlined above, SLPs have extensive training on delivering focused interventions and so may need less intensive training to achieve fidelity (Kaderavek & Justice, 2010). Treatment fidelity in practice The availability of manualised programs and training should make it easier for SLPs to deliver treatments with high fidelity but in reality the situation appears to be more complex. To illustrate, the results of a survey of SLPs in the United Kingdom (Roulstone, Wren, Bakopoulou, & Lindsay, 2012) indicated that they routinely used combinations and adaptations of intervention approaches for children with speech, language, and communication difficulties. In the Roulstone et al. study, SLPs reported that fidelity to the original intervention was difficult in practice and their predominant procedures for evaluating outcomes were teacher and parent feedback and data collected for individual clients. The SLPs indicated minimal reporting of intervention fidelity or outcomes at an organisational level. Roulstone et al. highlighted the uncertainties for client outcomes that are created when treatments are adapted to fit a particular client need or gap, and the importance of measuring treatment fidelity. Joffe and Pring (2008) also found that SLPs frequently combine interventions, in direct contrast to how they were evaluated. The limited attention to treatment fidelity measures may seem surprising given the range of options for monitoring treatment fidelity and the specific recommendations in commonly used manualised programs. One explanation may be that SLPs experience difficulty when there is divergence among the three components of the evidence- based model. For example, in situations when SLPs’ professional judgements related to meeting the child and family preferences might compete with the requirement to implement programs as prescribed by the developers. Another key challenge is the applicability of treatment efficacy studies, which evaluate the causal relationship between the specific intervention and the clinical outcomes in tightly controlled settings (Kaderavek & Justice, 2010). Funded efficacy studies in research settings may involve delivery of a higher dosage of treatment than is feasible in community-based organisations (Hoffman et al., 2013). In contrast, effectiveness research investigates interventions in real-world settings (Kaderavek & Justice, 2010). Roulstone (2015) acknowledged the advantages of manualised interventions, but also identified a need to provide more specific and consistent descriptions of all SLP interventions in order to highlight their explicit impact.

141

JCPSLP Volume 17, Number 3 2015

www.speechpathologyaustralia.org.au