Experimental Control of Confounding Factors

Any psychological test essentially can be regarded as a scientific experiment, in which confounding or unintended factors are controlled for. This is certainly the case with the ACT. The following factors have been taken into consideration.

Knowledge

Lack of knowledge (mind block) and incorrect knowledge (mind set) should not play any part in intelligence tests. No matter what opinion one may have about what intelligence tests should measure, it can never be knowledge. The latter is the purpose of achievement tests. If knowledge is not allowed to play any part in intelligence tests, then one should only use very simple problems, which will always be solved by the testee given unlimited time. The knowledge factor has been eliminated by using only very simple problems for which no knowledge whatsoever is needed.

Learning

Learning within the Test

In all existing intelligence tests practice during the test may play a very important role for the final test score. Some testees may quickly get used to the task while others are getting used to the task more slowly. It may well be the case that although some subjects may be different regarding their rate of learning, they may be the same regarding their performance when progress in learning has ended. This means that, if practice was allowed, those subjects would have obtained the same test results. If the purpose of the test is to measure rate of learning, then one should construct the test accordingly. Moreover, learning during the test also produces a trend in the reaction time curve. The resulting trend in the performance series may be a disturbing factor in the assessment of the distraction process. The practice factor is eliminated by giving the testee ample opportunity to practice the test before actually taking them.

Learning accross Tests

Learning effects between different administrations of the same test are referred to as reminiscence effects. The study of reminiscence has a long history, which is shortly described in Eysenck and Frith (1977, chapter 1):

Reminiscence is a technical term, coined by Ballard in 1913, denoting improvement in the performance of a partially learned act that occurs while the subject is resting, that is, not performing the act in question. (Eysenck and Frith, 1977, page 3).
The reality of the phenomenon was first experimentally demonstrated by Oehrn (1895). In experiments on reminiscence the same task is always administered twice or more. One is mainly interested in the effect of the rest periods between test administrations. Learning is not only apparent within tests but also, and very distinctive, across tests. If one wants to cancel out all learning effects, one should also include possible reminiscence effects. Therefore, the testee is given the opportunity and also is encouraged to do the test several times until complete habituation is obtained.

Inaccuracy

It is a well-known fact that subjects are able to exchange speed against accuracy. Routine tasks can be done faster, but at the cost of accuracy and subjects are able to work more accurately, but at the cost of speed. This phenomenon is referred to as the speed-accuracy trade-off and can be described in terms of the so-called speed-accuracy trade-off function, in which the probability of a correct answer is given as a function of the time needed to answer the item. It is a monotone increasing function and it may vary from subject to subject. Depending on the subject, the function may be shifted to the right (a lower ability) or to the left (a higher ability). As long as one does not know the ability of the subject, one cannot know whether the observed reaction times are high or low when they would have been corrected for errors. This is only possible when the trade-off function of the subject in question is known. This, however, is generally not the case. In the case of the ACT the whole problem has been circumvented by only accepting test results without any error.

References

Eysenck, H.J. and Frith, C.D. (1977). Reminiscence, motivation and personality. London Plenum Press.

Oehrn, A. (1896). Experimentelle Studiëen zur Individualpsychologie [Experimental research on the study of individual differences]. Psychologische Arbeiten, 1, 92-151.