Structural and Convergent Validity of Intelligence Composites: Integrating Evidence From Three Analysis Levels

Despite extensive evidence of the reliability and validity of general intelligence (g) composites, current theoretical intelligence models—and with them also recent intelligence tests—de-emphasize g and instead focus more on broad abilities, such as fluid reasoning and processing speed. This although broad ability composites have been shown to be much less reliable and valid compared to g composites. In practice, both g and broad ability composites are interpreted for individuals and used to inform high-stakes decisions. Therefore, it is important to further clarify the validity of their interpretation for current intelligence tests not only at the group level, but also at the individual level. This dissertation thus aims to determine to what extent structural and convergent validity evidence provided at different analysis levels (i.e., the total sample, subgroup, and individual level) supports the interpretation of (a) g composites and (b) broad ability composites.
Structural validity evidence provided by Studies 1, 2, and 3 supports a strong and predominant g factor and weak broad ability factors for two concurrent intelligence tests at the total sample level as well as—for one of these tests—at the level of subgroups differing in sex and age (Study 2). Most of the postulated broad abilities were confirmed for these tests, but Visual Processing and Fluid Reasoning collapsed to one factor in all three studies. Of the confirmed broad ability composites, however, only two were (sometimes) reliable enough to justify their interpretation. Convergent validity evidence provided by Studies 4 and 5 reveals high correlations and small mean differences in g composites of multiple tests at the total sample level, but the g and broad ability composites from different intelligence tests (Study 4) and different g composites from the same tests (Study 5) sometimes showed large score differences at the individual level. These were predicted by IQ level and age, suggesting systematic differences across subgroups that differ in these characteristics. Even after taking measurement error into account by investigating the overlap of confidence intervals (CIs), there was still considerable incomparability. In Study 5, we thus examined if using more accurate reliability coefficients for CIs could increase comparability. Indeed, comparability was substantially improved if test–retest reliabilities or age- and IQ-level-specific internal consistencies were used for 95% CIs instead of one overall internal consistency. Finally, results from Study 5 suggested that the number, g factor loadings, and content of subtests might also influence the comparability of g composites.
The studies of this dissertation provide further support for the validity of the interpretation of g composites—but only if 95% CIs based on accurate reliability estimates are used—and against the validity of the interpretation of most broad ability composites from concurrent intelligence tests. Consequently, score interpretation should focus primarily, if not exclusively, on the g composite, which should consist of a sufficient number of subtests of heterogeneous content and with high g factor loadings. Moreover, especially for high-stakes decisions, at least two tests should be administered that are selected and interpreted with respect to testee characteristics and test content. Explanations and further implications of the findings of this dissertation as well as future directions for intelligence assessment are discussed in light of the goals pursued with intelligence assessments.