What are basal and ceiling rules? How examiners decide where to start and stop

Basal and ceiling rules are the administration rules that tell an examiner where to begin a subtest and when to stop, so that each person answers only the items near their own level instead of every item in the book. The "basal" is a run of correct answers that lets the examiner assume everything easier would also have been passed, and the "ceiling" is a run of errors that lets the examiner assume everything harder would also have been failed.

These rules are written into the manual of almost every individually administered ability, achievement and language test. This page defines the terms, shows real rules from published tests, explains why the rules exist and the scoring assumption they depend on, and reviews what research has found about how often examiners apply them wrongly.

Items within a subtest are arranged from easiest to hardest, and the rules below govern which stretch of that sequence a given person sees.

• Start point: The item where testing begins, usually set by age or grade and sometimes by the examiner's estimate of ability. A ten-year-old does not begin a vocabulary subtest at the item written for a six-year-old.

• Basal: A specified number of consecutive correct responses at the low end of what was administered. Once it is established, the examiner stops working downward.

• Ceiling or discontinue rule: A specified number of consecutive incorrect responses (or, on some tests, a number of errors within a block of items) after which the subtest ends. Wechsler manuals call this a "discontinue" rule, while other publishers say "ceiling".

• Reverse rule: If the person misses one of the first items given, the examiner tests backward from the start point until the basal is met or item 1 has been given.

• Credit below the basal: Items below the basal that were never administered are scored as correct and added to the raw score.

One point of terminology causes real confusion. A ceiling rule is an administration procedure for stopping a single subtest. A ceiling effect is a property of a whole test that lacks enough difficult items to measure its highest scorers, and the two ideas are unrelated apart from the shared word.

The exact numbers differ from test to test, and sometimes from subtest to subtest within one test, which is why examiners are trained to check the rule printed at the top of each subtest before starting it. The examples below come from publisher documentation.

• Woodcock-Johnson IV Tests of Achievement: Riverside's examiner training workbook lists a basal of six consecutive correct and a ceiling of six consecutive incorrect for most of the achievement tests, including Letter-Word Identification, Spelling, Passage Comprehension and Calculation. The two Reading Vocabulary subtests use five and five. Timed tests such as Sentence Reading Fluency have no basal or ceiling, and Oral Reading uses "continuation instructions" based on points earned in blocks of items. When the basal is not met on the first items, the examiner tests backward until it is met or item 1 is given, in which case item 1 serves as the basal. When stimulus material is visible to the examinee, the WJ IV asks examiners to test by complete pages, so a ceiling reached mid-page does not end testing until the page is finished.

• Peabody Picture Vocabulary Test: Pearson's comparison of editions shows the rule changing between versions. The PPVT-4 was given in 12-item sets, with a basal of 11 or 12 correct within a set and a ceiling of 8 or more incorrect within a set. The PPVT-5 uses a continuous item sequence, with a basal of 3 consecutive correct and a ceiling of 6 consecutive incorrect, and the companion EVT-3 uses the same 3 and 6. Our page on the Peabody Picture Vocabulary Test describes the test itself.

• WISC-V: The fifth edition shortened its discontinue rules relative to earlier Wechsler scales. In their independent review, Canivez and Watkins report that for most primary and secondary subtests the rule is now three consecutive zero-point responses. Pearson's FAQ for the edition reports that the primary index scores take about 10 minutes less to administer than on the WISC-IV with the same number of subtests, a saving the publisher credits to development work aimed at the shortest possible testing time.

• CELF-5: Pearson's documentation for the Australian and New Zealand edition says age-specific start points and performance-based discontinue rules were added across selected tests to shorten administration, and that the discontinue rule on Word Classes was shortened to four consecutive zero scores. Its scoring documentation treats a test as invalid if required items were omitted.

The WJ IV training material states the purpose directly: basal and ceiling requirements limit the number of items administered "but still be able to estimate, with high probability, the score that would have been obtained if all items were administered." By skipping items that are far too easy or far too hard, the rules keep testing short and, in the workbook's words, maximize "the examinee's tolerance for the testing situation." A struggling child is not made to fail item after item long past the point of possible success.

That efficiency depends on an assumption about how people respond to ordered items. Crediting every unadministered item below the basal assumes the person would have passed them all, and scoring every item above the ceiling as zero assumes they would have failed them all. In reality responses are probabilistic. Someone who misses three in a row might well have answered the fourth correctly, and someone who passes six in a row might have slipped on an easy item lower down.

For ordinary number-correct scoring this is largely absorbed by the norms, because the standardization sample was tested under the same rules, so a score is comparable to the norms only when the rules are followed exactly as written. The assumption matters more when item-level data are analyzed with modern models. Von Davier, Cho and Pan examined discontinue rules analytically and by simulation and, consistent with earlier studies, found that ability estimates are biased most when the unobserved items above the discontinue point are scored as wrong. They note that this is the scoring used operationally and call for more research to improve practice.

Basal and ceiling rules look mechanical, but studies of completed test protocols show they are misapplied often enough to change scores.

Belk's master's thesis, which shares its title and authorship with the 2002 journal article by Belk, LoBello, Ray and Zachar, examined 100 WISC-III protocols completed by 21 graduate student examiners. It found an incorrect ceiling on 22 percent of protocols and an incorrect basal on 18 percent. Fourteen of the 21 examiners made at least one ceiling error and ten made at least one basal error. Counting all error types, errors changed the Full Scale IQ on about two-thirds of the protocols.

The rule-related mistakes take a few predictable forms.

• Stopping too early or too late: Miscounting a run of errors, or overlooking a correct answer that resets the count, ends the subtest at the wrong item.

• Skipping the reversal: Moving forward after a miss on the first items without testing backward leaves the basal unestablished.

• Forgetting the free credit: Failing to add one point for each unadministered item below the basal lowers the raw score, and the WJ IV workbook lists this among its reminders for exactly that reason.

The problem is not confined to students. Styck and Walsh's meta-analysis of examiner errors on the Wechsler scales found that when failures to record responses were excluded, a mean of 41.2 percent of protocols contained at least one error, that psychologists made more errors per protocol than graduate students, and that on average 73.1 percent of Full Scale IQ scores changed as a result of examiner errors. Scoring adds its own variability. When Ryan and Schnakenberg-Ott had 19 psychologists and 19 graduate students score the same two WAIS-III protocols, only 42.1 percent of the psychologists' Full Scale IQs matched the correct value, although 89.5 percent fell within one standard error of measurement, and both groups were confident in their accuracy.

A reported score assumes the standardized procedure was followed, start points, reversals, ceilings and credits included. Styck and Walsh concluded that published estimates of the standard error of measurement may not capture the variance due to the examiner, so a score's real uncertainty is probably somewhat wider than its printed confidence interval. If a result looks inconsistent with everything else known about a person, asking whether the protocol was double-scored is a reasonable question for the examiner.

A basal is a required run of consecutive correct answers, such as three on the PPVT-5 or six on most WJ IV achievement tests. Once it is reached, the examiner assumes all easier items would have been passed and gives credit for them without administering them.

A ceiling (or discontinue) rule ends a subtest after a set number of consecutive errors, such as three zero-point responses on most WISC-V subtests. Items above the ceiling are scored as incorrect.

No. A ceiling rule is an instruction for stopping one subtest during administration. A ceiling effect is when a test as a whole does not have enough hard items to distinguish among the highest scorers.

If the first items are failed, the reverse rule sends the examiner backward until a basal is established or item 1 is reached. If the examiner skips that step or starts in the wrong place without correcting it, the raw score and the norm-based score can both be wrong.

Most fixed-form online tests give every person the same items, so these rules do not apply. Computerized adaptive tests pursue the same goal of matching items to ability, using a statistical model in place of fixed start and stop rules.

Basal and ceiling rules let an individually administered test home in on each person's level, saving time and sparing frustration at both ends. They rest on the assumption that a run of passes or failures predicts what would have happened on the items never given, and the norms are valid only if the rules are applied exactly as the standardization sample experienced them. The numbers differ by test and edition, and research on completed protocols shows that trained examiners misapply them often enough to move scores.

If you want to see how a standardized test reports a score with its uncertainty, you can take the RIOT IQ test, the Reasoning and Intelligence Online Test, which reports index scores with confidence intervals.

1. Wendling, B. J. (2014). Woodcock-Johnson IV Tests of Achievement examiner training workbook. Riverside. info.riversideinsights.com

2. Pearson. (2018). PPVT/EVT: What's changed between versions? Pearson Clinical Assessment. pearsonclinical.com.au

3. Canivez, G. L., & Watkins, M. W. (2016). Review of the Wechsler Intelligence Scale for Children-Fifth Edition: Critique, commentary, and independent analyses. In A. S. Kaufman, S. E. Raiford, & D. L. Coalson, Intelligent testing with the WISC-V (pp. 683-702). Wiley. ux1.eiu.edu

4. Pearson. (n.d.). WISC-V Canadian frequently asked questions. Pearson Canada Assessment. pearsonclinical.ca

5. Pearson. (n.d.). About the CELF-5 A&NZ. Pearson Clinical Assessment. pearsonclinical.com.au

6. von Davier, M., Cho, Y., & Pan, T. (2019). Effects of discontinue rules on psychometric properties of test scores. Psychometrika, 84(1), 147-163. doi.org

7. Belk, M. S. (2001). WISC-III administration, clerical, and scoring errors made by student examiners [Master's thesis, Auburn University Montgomery]. digitalarchives.aum.edu

8. Belk, M. S., LoBello, S. G., Ray, G. E., & Zachar, P. (2002). WISC-III administration, clerical, and scoring errors made by student examiners. Journal of Psychoeducational Assessment, 20(3), 290-300. doi.org

9. Styck, K. M., & Walsh, S. M. (2016). Evaluating the prevalence and impact of examiner errors on the Wechsler scales of intelligence: A meta-analysis. Psychological Assessment, 28(1), 3-17. doi.org

10. Ryan, J. J., & Schnakenberg-Ott, S. D. (2003). Scoring reliability on the Wechsler Adult Intelligence Scale-Third Edition (WAIS-III). Assessment, 10(2), 151-159. doi.org

Hero image: a high jumper at Kincaid Field, 1913, photographer unknown, public domain. Via the Digital Public Library of America and Wikimedia Commons.