Development and Psychometric Evaluation of the Assessment of Core CBT Skills (ACCS): An Observation-Based Tool for Assessing Cognitive Behavioral Therapy Competence

Abstract

This article outlines the development and psychometric evaluation of the Assessment of Core CBT Skills (ACCS) rating scale. The ACCS aims to provide a novel assessment framework to deliver formative and summative feedback regarding therapists’ performance within observed cognitive–behavioral treatment sessions, and for therapists to rate and reflect on their own performance. Findings from 3 studies are outlined: (a) a feedback study (n = 66) examining content validity, face validity and usability; (b) a focus group (n = 9) evaluating usability and utility; and (c) an evaluation of the psychometric properties of the ACCS in real world cognitive behavioral therapy (CBT) training and routine clinical practice contexts. Results suggest that the ACCS has good face validity, content validity, and usability and provides a user-friendly tool that is useful for promoting self-reflection and providing formative feedback. Scores on both the self and assessor-rated versions of the ACCS demonstrate good internal consistency, interrater reliability, and discriminant validity. In addition, ACCS scores were found to be correlated with, but distinct from, the Revised Cognitive Therapy Scale (CTS-R) and were comparable to CTS-R scores in terms of internal consistency and discriminant validity. In addition, the ACCS may have advantages over the CTS-R in terms of interrater reliability of scores. The studies also provided insight into areas for refinement and a number of modifications were undertaken to improve the scale. In summary, the ACCS is an appropriate and useful measure of CBT competence that can be used to promote self-reflection and provide therapists with formative and summative feedback.