Are Intelligence Tests' Contents Trivial?

Critics argue that intelligence test items seem too simple and disconnected from real life to measure something as complex as intelligence. Picture a parent watching their child undergo assessment by a school psychologist. The tasks appear like mere games: counting backward from twenty, defining words like "scorch," explaining how wood and coal are similar. After an hour of such activities, the psychologist recommends special education placement based on below-average scores. The parent remains doubtful—how can such trivial exercises evaluate genuine intelligence?

These concerns have merit at a surface level. Psychologists aren't actually interested in whether someone can complete digit span tasks or define vocabulary words. Rather, these activities serve as indicators of underlying intelligence, much like oral temperature indicates overall health. As one expert explained, scientists care about what these measures reveal regarding individual differences, not the specific tasks themselves. Performance on intelligence subtests represents symptoms of a person's intelligence level. By systematically evaluating these symptoms collectively, psychologists can draw conclusions about an examinee's cognitive abilities.

Strong evidence confirms that these seemingly simple tasks genuinely assess intelligence. Factor analysis consistently demonstrates that intelligence test items measure a general mental ability. Any collection of cognitive tasks will produce a general intelligence factor, and researchers have never identified a cognitive task with zero correlation to general intelligence.

However, factor analysis alone proves insufficient. Tests must also correlate with external variables theoretically related to intelligence—what researchers call criteria. From the field's beginning, test developers understood this necessity. Francis Galton sought connections between his intelligence measures and education level or social class. When he found none, he discarded his methods. Alfred Binet used school performance as his criterion, and when his test scores correlated with academic struggles, he correctly interpreted this as validation.

Modern evidence overwhelmingly supports this approach. Intelligence scores correlate positively with grade-point averages, standardized test performance, educational attainment, adult socioeconomic status, and gifted program placement. These correlations allow meaningful predictions about educational outcomes, regardless of whether one accepts general intelligence theory.

Determining what a test measures requires more than casual inspection of its questions. As one intelligence researcher noted, critics make unwarranted inferences by examining superficial aspects of test content. Most individually administered tests don't require paper and pencil, aren't timed, and use only elementary concepts. Over a century ago, researchers recognized that classifying tests through mere inspection "may form an interesting pastime, but it can hardly be taken seriously as a contribution to science."

Data from factor analysis and criterion correlations must guide our understanding. The evidence overwhelmingly confirms that these tests measure intelligence effectively. Though items may appear unimportant, they function as the scientific "yardstick" for measuring intelligence—just as a thermometer measures temperature despite only displaying mercury expansion. What matters are the cognitive processes people engage when answering test items, and every cognitive task requires demonstrating intelligence to some degree.



From Chapter 8 of "In the Know: Debunking 35 Myths About Human Intelligence" by Dr. Russell Warne (2020)