SMABS 2004 Jena University
SMABS 2004 Home Organization About Jena Sponsors Links Imprint / Contact SMABS Home

European Association of Methodology

Department of methodology and evaluation research

Jena University

Contributions: Abstract

Test optimization for the nominal response model

Martijn P.F. Berger Valeria Lima Passos
University of Maastricht
The Netherlands

Modern testing in Education and Psychology is mostly based on Item Response Theory (IRT), and different dichotomous and polytomous IRT models have been proposed to analyse test data. As in any research situation where data are obtained, the costs of collecting test data may be enormous. Optimal design techniques can be applied for the reduction of administration time and the costs of producing items while parameter estimation remains as efficiently as possible. One of the problems, however, is that optimal tests may be different for different objective functions.

In this paper the optimal composition of a test by means of two most often used objective functions, namely the A- and the D-optimality criterion, will be considered. Previous research has shown that for dichotomous IRT models, the resulting optimal tests may be quite different for these two criteria. In this paper we will investigate their performance for the more general case of polytomous items. More specifically, the performance of the two objective functions will be evaluated for different (homogeneous and heterogeneous) samples of examinees and for item banks with different forms of their information functions. The composition of the item banks is based on parameter values from tests administered to medical students from the University of Maastricht.

The results show that, for the nominal response model, the A- and D-optimality criteria have different item selection mechanisms and that the degree of difference depends on the latent trait distribution of the sample of test-takers. The A-optimality criterion tends to reveal a test with a more peaked information function, while the D-optimality criterion tends to result in tests with a more flattened information function. We will explain why these criteria have different performances and in what kind of situations these criteria are more suitable to apply.