Experimental Control of Confounding Factors
Any psychological test essentially can be regarded as a scientific
experiment, in which
confounding or unintended factors are controlled for. This is
certainly the case with
the ACT. The following factors have been taken into consideration.
Knowledge
Lack of knowledge (mind block) and incorrect knowledge (mind set)
should not play any part in
intelligence tests. No matter what opinion one may have about
what intelligence tests should measure, it can never be knowledge.
The latter is the purpose of achievement tests.
If knowledge is not allowed to play any part in intelligence tests,
then one should only use very simple problems, which will always
be solved by the testee given unlimited time.
The knowledge factor has been eliminated by using
only very simple problems for which no knowledge whatsoever is needed.
Learning
Learning within the Test
In all existing intelligence tests practice during the test may
play a very important role for the final test score. Some testees
may quickly get used to the task while others are getting used to
the task more slowly. It may well be the case that although
some subjects may be different regarding their rate of learning,
they may be the same regarding their performance when progress in
learning has ended. This means that, if practice was allowed,
those subjects would have obtained the same test results.
If the purpose of the test is to measure rate of learning,
then one should construct the test accordingly.
Moreover, learning during the test also produces a trend in
the reaction time curve. The resulting trend in the
performance series may be a disturbing factor in the
assessment of the distraction process.
The practice factor
is eliminated by giving the testee ample opportunity to practice
the test before actually taking them.
Learning accross Tests
Learning effects between different administrations of the same
test are referred to as reminiscence effects. The study of
reminiscence has a long history, which is shortly described
in Eysenck and Frith (1977, chapter 1):
Reminiscence is a technical term, coined by Ballard in
1913, denoting improvement in the performance of a partially
learned act that occurs while the subject is resting, that is, not
performing the act in question. (Eysenck and Frith, 1977, page 3).
The reality of the phenomenon was first experimentally
demonstrated by Oehrn (1895). In experiments on reminiscence
the same task is always administered twice or more.
One is mainly interested in the effect of the rest
periods between test administrations. Learning is not only
apparent within tests but also, and very distinctive, across tests.
If one wants to cancel out all learning effects,
one should also include possible reminiscence effects.
Therefore, the testee is given the opportunity and also is
encouraged to do the test several times until complete
habituation is obtained.
Inaccuracy
It is a well-known fact that subjects are able to exchange
speed against accuracy. Routine tasks can be done faster, but at
the cost of accuracy and subjects are able to work more
accurately, but at the cost of speed. This phenomenon is
referred to as the speed-accuracy trade-off and can be
described in terms of the so-called speed-accuracy
trade-off function, in which the probability of a
correct answer is given as a function of the time
needed to answer the item. It is a monotone
increasing function and it may vary
from subject to subject. Depending on the subject,
the function may be shifted to the right (a lower ability)
or to the left (a higher ability). As long as
one does not know the ability of the subject,
one cannot know whether the observed reaction times
are high or low when they would have been
corrected for errors. This is only possible when
the trade-off function of the subject in question
is known. This, however, is generally not the case.
In the case of the ACT the whole problem has been
circumvented by only accepting test results without
any error.
References
Eysenck, H.J. and Frith, C.D. (1977).
Reminiscence, motivation and personality.
London Plenum Press.
Oehrn, A. (1896).
Experimentelle Studiëen zur Individualpsychologie
[Experimental research on the study of individual differences].
Psychologische Arbeiten, 1, 92-151.