There is emerging evidence that the performance of risk assessment instruments is weaker when used for clinical decision‐making than for research purposes. For instance, research has found lower agreement between evaluators when the risk assessments are conducted during routine practice. We examined the field interrater reliability of the Short‐Term Assessment of Risk and Treatability: Adolescent Version (START:AV). Clinicians in a Dutch secure youth care facility completed START:AV assessments as part of the treatment routine. Consistent with previous literature, interrater reliability of the items and total scores was lower than previously reported in non‐field studies. Nevertheless, moderate to good interrater reliability was found for final risk judgments on most adverse outcomes. Field studies provide insights into the actual performance of structured risk assessment in real‐world settings, exposing factors that affect reliability. This information is relevant for those who wish to implement structured risk assessment with a level of reliability that is defensible considering the high stakes.
Poster presentatie op conferentie Background: Assessments of functional communication skills of children with cerebral palsy (CP), classified with the Communication Function Classification System (CFCS), often differ between the child's school teacher and the speech language therapist (SLT). Assessment by the SLT is usually based on observations in a clinical setting, which may not be representative of the functional communication skills in daily life. This study evaluated the inter-rater agreement of the CFCS assessed by the school teacher and SLT before and after observation of a communicative situation in the classroom. Methods: Functional communication of 35 children with CP (4 to 18 years; 26 with Alternative and Augmentative Communication, AAC) was classified by the own SLT and teacher using the CFCS. SLT's performed two assessments: the first without additional instructions and the second after observation of the child during a communicative situation in the classroom. For both assessments of the SLT inter-rater agreement on CFCS-level between SLT and teacher was determined using Cohen's weighted kappa statistics. Results: For the whole group, inter-rater reliability was 0.6 before observation in de classroom and 0.7 after observation. In the group without AAC weighted K was 0.67 for both assessments. In the group with AAC weighted K increased from 0.2 to 0.61. Interpretation The increased inter-rater agreement of CFCS classification between teacher and SLT after observation in the classroom, especially for children with AAC, emphasizes the need for professionals to base their CFCS assessment on observation of functional communication in everyday situations.
MULTIFILE
Caregivers of persons with profound intellectual and multiple disabilities (PIMD) often describe the quality of the daily movements of these persons in terms of flexibility or stiffness. Objective outcome measures for flexibility and stiffness are muscle tone or level of spasticity. Two instruments used to grade muscle tone and spasticity are the Modified Ashworth Scale (MAS) and the Modified Tardieu Scale (MTS). To date, however, no research has been performed to determine the psychometric properties of the MAS and MTS in persons with PIMD. Therefore, the purpose of this study was to determine the feasibility, test-retest reliability, and interrater reliability of the MAS and MTS in persons with PIMD. We assessed 35 participants on the MAS and MTS twice, first for the test and second a week later for the retest. Two observers performed the measurements. Feasibility was assessed based on the percentage of successful measurements. Test-retest and interrater reliability were determined by using the Wilcoxon signed rank test, intraclass correlation coefficients (ICC), Spearman's correlation, and either limits of agreement (LOA) or quadratically weighted kappa. The feasibility of the measurements was good, because an acceptable percentage of successful measurements were performed. MAS measurements had substantial to almost perfect quadratically weighted kappa (>0.8) and an acceptable ICC (>0.8) for both inter- and intrarater reliability. However, MTS measurements had insufficient ICCs, Spearman's correlations, and LOAs for both inter- and interrater reliability. Our data indicated that the feasibility of the MAS and MTS for measuring muscle tone in persons with PIMD was good. The MAS had sufficient test-retest and interrater reliability; however, the MTS had an insufficient test-retest and interrater reliability in persons with PIMD. Thus, the MAS may be a good method for evaluating the quality of daily movements in persons with PIMD. Providing test administrators with training and clear instructions will improve test reliability.
