Is Intelligence Whatever Collection of Tasks a Psychologist Puts on a Test?

Debunking myths on human intelligence

Dr. Russell T. WarneChief Scientist

Is Intelligence Whatever Collection of Tasks a Psychologist Puts on a Test?

. . . many psychologists simply accept an operational definition of intelligence by spelling out the procedures they use to measure it ... Thus, by selecting items for an intelligence test, a psychologist is saying in a direct way, “This is what I mean by intelligence.” A test that measures memory, reasoning, and verbal fluency offers a very different definition of intelligence than one that measures strength of grip, shoe size, hunting skills, or the person’s best Candy Crush mobile game score. (Coon & Mitterer, 2016, p. 290)

Many psychology textbooks suggest that intelligence is operationally defined by whatever procedures psychologists use to measure it. This common misconception holds that intelligence is merely an arbitrary collection of abilities—what Gottfredson (2009) called viewing "intelligence as a marble collection." According to this flawed reasoning, intelligence only seems to exist because psychologists bundled various abilities together to produce an IQ score, and different test creators could theoretically measure completely unrelated abilities.

This reasoning fails for several important reasons. First, the general ability factor (g) is not simply a sum of mental abilities. Factor analysis identifies the overlapping variance across different cognitive tasks while eliminating each task's unique components. As Jensen explained, g is "a distillate, representing the single factor that all different manifestations of cognition have in common" rather than a mixture or average of diverse tests. Because factor analysis extracts this common element and removes unique portions, the specific collection of tasks matters little, provided the test includes several types of cognitive items.

Charles Spearman (1927) discovered this principle and called it the "indifference of the indicator." The term "indicator" referred to a test's surface content—whether vocabulary, matrices, digit span, or spatial reasoning. "Indifference" meant that this surface content is ultimately unimportant; all cognitive tasks measure intelligence to some degree, and g remains consistent regardless of item format. Though radical when proposed, Spearman's claim has received substantial research support.

This doesn't mean all cognitive tasks measure g equally well. A task's effectiveness at measuring intelligence is called its g loading, ranging from 0 to 1. Vocabulary and matrix reasoning typically show very high g loadings (up to .80), while short-term recall and reaction time tasks show lower loadings. More complex tasks generally have higher g loadings than simpler ones. Tests built with high g-loading tasks can be shorter while producing superior intelligence estimates.

The strongest evidence against the "arbitrary collection" view comes from studies administering multiple intelligence tests to the same individuals. If intelligence were merely arbitrary task combinations, different tests would correlate poorly. Instead, research consistently finds near-perfect correlations. Stauffer, Ree, and Carretta (1996) found g factors from traditional subtests and computerized tasks correlated between r = .950 and .994. Keith et al. (2001) found the Woodcock-Johnson III and Cognitive Assessment System g factors correlated at r = 0.98, despite the CAS not being designed to measure g. Floyd et al. (2013) examined six test combinations and found g factor correlations averaging r = .95. Johnson's research with larger samples found g correlations of r = .99 to r = 1.00 across multiple test batteries, leading them to conclude there is "Just one g."

The evidence is clear: intelligence is not an arbitrary collection of test items. Any cognitive task measures intelligence to some extent, and when combined through factor analysis, different tests produce essentially identical g factors. Intelligence, as measured by g, represents a unitary ability regardless of which tasks are used to assess it.

From Chapter 1 of "In the Know: Debunking 35 Myths About Human Intelligence" by Dr. Russell Warne (2020)