Assessments in school psychology: comparability and diagnostic utility of tests measuring cognitive abilities

In the context of school psychology, students’ cognitive abilities such as intelligence, attention, and executive functions are often assessed for diagnostic classification purposes (e.g., intellectual disability or giftedness; the presence of an attention-deficit/hyperactivity disorder [ADHD]). There are an increasing number of tests measuring these abilities, raising the question of whether different tests aimed at measuring the same psychological attribute yield comparable results. Most previous studies have addressed this question using grouplevel analyses (i.e., correlations among tests). Yet, research investigating whether different tests yield comparable scores for individuals, that is, scores that lead to the same classification, is sparse. This is surprising given that individual-level comparability is essential, as diagnostic classification is most often based on cut-off scores. Thus, the main goal of the present dissertation was to shed light on such individual-level comparability with findings from three studies. Studies I and II revealed that different intelligence test scores were generally highly correlated on the group level, but individual-level comparability was not satisfactory. Specifically, in Study I, the 95% confidence interval (CI) of two test scores obtained from different tests overlapped in only about 60% of all cases. In Study II, the 95% CI of intelligence scores obtained from the same test (Full-Scale IQs vs. Screening IQs) did overlap in 74–99% of all cases. In both studies, comparability decreased toward the tails of the IQ distribution, the very ranges in which diagnostic questions most often arise in practice. Study III revealed that scores of attention and executive functions obtained in parent questionnaires did not correlate substantially with those obtained in performance-based measures. Moreover, only scores from parent questionnaires and not those from performancebased measures were associated with an ADHD diagnosis. Possible approaches for dealing with and enhancing individual-level comparability are discussed. The focus thereby lies on two aspects that were identified to be the most prominent sources of incomparability: different theoretical groundings of tests and measurement error.