A quick overview of the RIOT's test flags for you all...
As an online intelligence test, the RIOT has excellent potential as a psychological instrument; however, it also presents risks that are not present in traditional tests. Most examinees are expected to take the RIOT without supervision, which increases the risk for cheating and other behaviors that threaten the trustworthiness of scores. The RIOT team has created flags on score reports that are designed to alert score users to irregularities that may make the reported score less accurate. The list of flags that can be reported on the RIOT will continue to grow, but currently comprises these flags 👇
Low IQ Score Flag
The Low IQ Score flag appears when overall cognitive performance suggests potential difficulty understanding test instructions. This flag carries a high level of caution as it indicates the examinee may not have possessed the foundational cognitive abilities necessary to engage with the test materials properly. When this flag is present, score interpreters should carefully consider whether the results accurately reflect the individual's capabilities.
Low Subtest Scores Flag
The Low Subtest Scores flag is activated when performance on specific cognitive domains falls significantly below expected ranges. These unusually low performances may suggest that an examinee experienced confusion with instructions or response methods for particular subtests. The flag operates on three levels of severity, based on the extent of affected domains. Mild, moderate, and high caution levels help identify instances where specific cognitive areas may have been compromised during testing.
Language Flag
The Language flag appears for examinees whose primary language differs from the language of the test. This flag carries moderate caution because linguistic barriers may prevent full comprehension of test questions, particularly affecting verbal reasoning abilities. The resulting scores may underestimate the examinee's true cognitive potential, making this flag essential for ensuring equitable assessment across diverse linguistic backgrounds.
Non-Normed Population Flag
The Non-Normed Population flag is triggered when an examinee's demographic characteristics fall outside the established normative sample. This flag carries moderate caution because valid score interpretation depends on the use of appropriate comparison groups. When examinees don't match the normative population, standard score comparisons may not accurately reflect their standing relative to their actual peer group, potentially leading to misinterpretation of abilities.
The Retake flag appears when our system detects previous RIOT exposure within a specified timeframe. This flag carries high caution due to well-documented practice effects in cognitive testing. Familiarity with test content and format can artificially inflate performance, making it difficult to distinguish between genuine ability improvements and score inflation from prior exposure.
The Subtest Window Inactive flag is activated when our monitoring system detects attention patterns that suggest divided focus during testing. This may indicate environmental distractions or concurrent activities that could compromise test validity. The flag operates on three severity levels based on the frequency and pattern of detected attention shifts, helping identify examinees who may not have maintained proper focus throughout the assessment.
The Subtests Exited flag appears when testing sessions show irregular completion patterns across multiple subtests. Since subtests are designed for continuous completion, interruptions may indicate technical difficulties, environmental disruptions, or insufficient engagement with the testing process. The flag severity increases based on the extent of affected subtests, helping identify potentially compromised testing conditions.
The Highly Rapid Responses flag is triggered when response patterns suggest insufficient consideration of test items. Speedy responses often indicate a lack of serious engagement with the assessment materials, which can potentially compromise the validity of the results. The flag operates across multiple severity levels depending on how extensively rapid responding occurs throughout the battery, helping identify cases where examinees may not have attempted genuine problem-solving.
The Excessive Reaction Time Trials Repeated flag appears when performance on timed tasks suggests potential difficulties with task comprehension or compliance. Our system monitors completion patterns to identify cases where examinees may struggle with understanding or following specific procedural requirements. This flag carries mild caution and helps identify instances where performance may not accurately reflect processing speed abilities.
When circumstances trigger a flag, the examinee’s score is not altered. A flag on a score report is displayed for informational purposes only and is accompanied by a short explanation detailing the circumstances that triggered the flag. Whether the flag invalidates the examinee’s score(s) on the RIOT is left to the judgment of the person reading the score report. If multiple flags are triggered, then all of them are displayed on the examinee’s score report. The person(s) interpreting the scores must decide whether the flags are severe enough to invalidate RIOT scores.
