JCPSLP Vol 18 No 3 2016

27 / 156

JCPSLP Volume 18, Number 3 2016 129
of scoring, transcription, recordings and analysis will be
included in the manual to instruct for examiners in both
research and clinical use.
Stage 2. Operationalisation of the
Vietnamese Speech Assessment
Operationalisation is the evaluation and validation process
of an assessment to ensure its validity and reliability (Frytak,
2000). The VSA is currently undergoing operationalisation.
Eventually the VSA will contain consistent assessment
materials, administration and scoring protocols. Once
normative data have been collected and analysed, they will
be added to the manual. Eventually the manual will also
include information about validity (content, construct,
predictive, concurrent) and reliability (internal consistency,
test–retest reliability, intra- and inter-rater reliability).
To date, the VSA authors have considered the content
validity for the VSA. Content validity refers to “the degree
to which the items in the measure cover the domain of
interest” (Frytak, 2000, p. 22). Content validity of the VSA
was conducted first by a systematic examination of relevant
literature and previously designed speech sampling tools
to specify the initial test content. Second, professional
judgement was used to define the test areas and to
evaluate the relevance and representativeness of the test
items with the target construct. The VSA was piloted by the
design team on five adults and one child who were bilingual
Vietnamese–English speakers to examine the relevance
of the word list and scoresheet and to estimate length of
the time required to complete the test. Adults completed
the task in approximately 8–10 minutes, the child in about
20 minutes. After the initial pilot testing, some changes
were made with to stimulus items (e.g., changing images
for the word ‘pin ’ from torch to battery) and prompts (e.g.,
changing the cues for the word ‘tết’ [Tet holiday]). Other
psychometic properties (including internal consistency,
test–retest reliability, inter- and intrarater reliability, criterion
validity, construct validity, item analysis, sensitivity and
specificity, standardisation) will be established in further
steps to operationalise the VSA.
Conclusion
The development of a speech sampling tool requires two
stages: conceptualisation and operationalisation. The
conceptualisation of the VSA has been completed and
work on the operationalisation is continuing. This paper
provides an example of how to begin to undertake test
development in a majority-world country.
References
Cameron, N., & Watt, C. (2006). Vietnamese articulation
test (VAT: Version I-II-III-IV) . Flinders University, Adelaide,
Australia: Author.
Cao, X. H. (2006). Tiếng Việt mấy vấn đề ngữ âm-ngữ
pháp-ngữ nghĩa [Vietnamese: Some issues in phonology-
syntax-semantics]. Thành phố Hồ Chí Minh, Việt Nam:
Khoa học x ã hộ i.
Cheng, L. L. (1991). Assessing Asian language
performance . Oceanside, CA: Academic Communication
Associates.
The Children’s Hospital No.1. (2013). Bộ Kiểm Tra Từ
Đơn Bằng Hình Ảnh [One–Picture Word List]. Thành phố
H ồ Chí Minh, Việ t Nam: Author.
Đoàn, T. T. (2003). Ngữ âm tiếng Việt [Vietnamese
phonetics]. Hà Nội, Việt Nam: Đại họ c Quốc gia Hà Nội.
provided in a manual to ensure that examiners follow the
same testing procedures.
Cueing hierarchy
Examiners are required to give instructions following a
four-step prompt hierarchy to elicit each target word: (a)
open-ended question, (b) gap fill or content-related prompt,
(c) binary choice (with the target word produced first),
and (d) delayed imitation (e.g., “Heart. Repeat, please”
(Tim. Con nhắc lại)). Children are encouraged to respond
spontaneously by naming the picture at the first step as
much as possible. The open-ended question used in step
1 for each target picture is “What’s this?” Some target
pictures will be asked differently to elicit the targeted
response. For example, with the target picture of elephant
(voi ), the examiner asks the child “Đây là con…” (This is a
[animacy word]…) so that the child can fill the target word
after the animacy word produced by the examiner. If the
examiner does not say the animacy/ inanimacy word, the
child might say a compound word (animacy/inanimacy
+ target word, e.g., con voi) instead of saying the target
word only (e.g., voi) because animacy/ inanimacy words
commonly precede a noun in the Vietnamese language
and are acquired early in young children (Pham & Kohnert,
2009; Tran, 2011). Before testing begins, it is useful to train
the child to not include the classifier using common objects
in the environment. If the child cannot label the picture,
additional cues or content-related questions are provided
to elicit the expected response. For example, a content-
related question “What has a long trunk?” is asked to elicit
a target word voi (elephant). If this step fails, the examiner
will give a binary choice by presenting the target word first
to participant, for example, “voi hay chân” (elephant or leg ).
If the participant does not respond to the binary choice, the
examiner will provide the target word for imitation. A list of
prompts and cues for each target word was also created to
support the testing protocol.
Scoring, transcription, recording and analysis
A score form was created which includes columns for the
word in orthography; the adult target in International
Phonetic Alphabet for Standard Vietnamese, Northern,
Central and Southern dialects; the child’s production; and
columns for scoring each phonological pattern.
Children are assessed individually. Examiners are
required to transcribe children’s responses online by
using the International Phonetic Alphabet symbols. It
is recommended that the transcriptions be based on
the children’s first attempt if possible. If children’s first
productions are not clear, then they are asked to repeat the
words. The score form also requires examiners to mark the
prompt or cueing level used for each response.
It is recommended that the children’s responses be
audio-recorded and/or video-recorded with the permission
of children’s caregivers and the school. A microphone
should be placed close to the children’s mouths (within
15 cm) and the video camera should be set up to record
the children’s faces. The recordings can be used to check
reliability between examiners, and to record change in the
children’s speech over time. Video recordings can be used
to check the children’s productions of consonants and
vowels that can be seen on the recording (e.g., bilabials).
The VSA score form provides a relational analysis
(including percentages of consonants, semivowels, vowels
and tones that are produced correctly) and an analysis
of phonological processes/ patterns. Guidance in terms