Professional IQ Tests

Dr. Russell T. Warne

Sep 10, 2023

IQ tests were first created 120 years ago, and in that time the number of tests has proliferated. With so many tests available, it can be difficult to know which tests are professional tests. This article is designed to help non-experts determine whether the test they are going to take is a professional test.

Characteristics of Professional IQ Tests

A good professional IQ test has these characteristics:

A professional IQ test is created by experts with training in the science of testing (a discipline called “psychometrics”). It should be easy to identify a test’s author(s) and their professional training and education. A test created by an anonymous author should always be avoided. Legitimate test creators are proud to have their name on their test and are willing to stake their professional reputation on the quality of the test. Anonymity helps a bad actor avoid accountability
A professional IQ test has its technical properties documented. Often, this documentation is in the form of a test manual, which is a book-length explanation of all the technical aspects of the test’s creation and evaluation. Technical documentation may only be accessible to experts (i.e., a test creator may not make the manual available to the general public), but there should be some indication that a manual exists. Shorter technical documentation about specific issues (e.g., research bulletins, technical updates) may also exist.
A professional IQ test has been independently evaluated by experts. Some of this evaluation occurs while the test is in development. For example, the Reasoning and Intelligence Online Test (RIOT) had its content evaluated by a panel of diverse experts from cognitive, educational, and developmental psychology. Other evaluations can occur after publication, such as in the form of endorsements from outside experts. One of the most powerful forms of evidence that a test has been independently evaluated by experts is if the test has been used in peer-reviewed research. Searching for the name of the test in Google Scholar is a quick way to determine whether the test has been used in scholarly research.
A professional IQ test has been created to meet relevant technical and ethical standards for psychological tests. While there is no organization that certifies psychological tests, the American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education have published guidelines called the Standards for Educational and Psychological Testing. Professional test creators strive to meet these guidelines and provide information about their efforts on their web sites and in their technical documentation. The Standards are designed for an American context, but professional organizations in some other countries have published similar guidelines. The International Test Commission has also published its own guidelines for international and cross-cultural testing. Professional test creators reference relevant guidelines and publicly make efforts to adhere to them.
A professional IQ test is designed for a specific population. It is impossible to make a test that works for everyone. Language barriers, cultural differences, age and developmental differences, disabilities, and other characteristics make some tests inappropriate for some populations. Professional test creators are clear about who the intended test takers are for their tests. Other people may be allowed to take the test, but professional test creators will encourage users to use caution when administering the test outside of the target population(s) or to gather data that the test functions properly for people outside of the target group. It is a red flag when a test creator implies that their test is appropriate for anyone to take.
A professional IQ test should produce reliable scores. “Reliability” is the degree to which test scores are consistent. Because intelligence is believed to be a stable trait throughout most of the lifespan, IQ scores should also be stable. Reliability data should be reported in scholarly publications, a test manual, or publicly. (Notice that reliability is a property of test scores, not of the test itself.)
A professional IQ test has evidence that its scores are valid for a specific purpose or for a specific interpretation. Blanket statements that “the test is valid” are scientifically inaccurate and encourage confusion. Like reliability, validity is not a property of tests. Rather, validity is a property of test score uses or interpretations. Test scores have limits to their valid uses or interpretations. For example it is valid to use IQ test scores for hiring employees for a cognitively intense job, but using those same scores to determine who should be an organ donor is not justified. Professional test creators are clear about what applications are valid uses of their test scores, and they caution against likely misuses or bad interpretations.
A professional IQ test is aligned with a scientific theory. There are competing theories of intelligence, the most popular of which are the Cattell-Horn-Carroll theory and the bifactor model. There are other theories that are used to create intelligence tests, such as the PASS (Planning, Attention, Simultaneous and Successive processing) theory. Searching for the name of a theory on Google Scholar can help non-experts to determine whether a theory is legitimate. Savvy users can even read some of the theoretical articles and come to their own judgments about whether the test is aligned with the theory.
A professional IQ test has a norm sample that is representative of its intended population. A “norm sample” is a sample of test takers that serves as the standard to which future examinees are compared. Whether examinees are labeled as “below average” or “above average” (and how far above or below average they are) is based on how their test performance compares with the norm sample’s. A representative norm sample is important because non-representative samples give distorted results. Professional IQ test creators state, usually in their technical documentation, the number of people in the norm sample, how the norm sample’s data was collected, and the demographic statistics of the norm sample. This information is usually in their technical manual, but may be available elsewhere. If a test’s norm sample is not clearly stated, or if it is a self-selected group of test takers, then the test cannot provide accurate scores to examinees.

Is the test not a professional IQ test?

If a test does not meet all of these criteria, then you should avoid it. Some of these tests are created by well-meaning amateurs, though others are created by scammers. Doing a little bit of investigation may be inconvenient, but it is often needed. Professionally-developed IQ tests are powerful tools, and it is worth the effort to ensure that they provide trustworthy data.

We hope you found this information useful. For further questions, please join our Discord server to ask a Riot IQ team member or email us at support@riotiq.com. If you are interested in IQ and Intelligence, we co-moderate a related subreddit forum and have started a Youtube channel. Please feel free to join us.