SMABS 2004 Jena University
SMABS 2004 Home Organization About Jena Sponsors Links Imprint / Contact SMABS Home

European Association of Methodology

Department of methodology and evaluation research

Jena University

Contributions: Abstract

Myths and paradoxes in the realm of classical test theory

Safir Yousfi
University of Jena
Germany

The construction of most psychological tests is based on two principles: aggregation and the elimination of items with low item-scale correlations. It is shown that this practice cannot be justified by the formulas of classical test unless very strong assumptions hold. If the items under consideration are not t-congeneric (one-dimensional) or if the errors are correlated then neither aggregation nor the selection with respect to item-scale correlations is an appropriate method to assure the validity and reliability of a scale. In this case it is even possible that aggregation leads to monotonous decrease of reliability whereas validity increases.

It is argued that this hitherto unnoticed kind of attenuation paradox is likely to occur in practice whereas the constellations discussed so far are practically irrelevant. Paradoxical effects of item selection with regard to item-scale correlations are also discussed. It is concluded that the conventional method of scale construction is not founded on statistical theorems but on (more or less implicit) assumptions that are logically inconsistent. Therefore, this practice should rather be called naïve than classic.