SMABS 2004 Jena University
SMABS 2004 Home Organization About Jena Sponsors Links Imprint / Contact SMABS Home

European Association of Methodology

Department of methodology and evaluation research

Jena University

Contributions: Abstract

Test design optimisation in the early stage of CAT with the Nominal Response Model

Valeria Lima Passos Martijn P.F. Berger Frans E.S. Tan
University of Maastricht
The Netherlands

The early stage of CAT is a term used to denote the instable phase of trait estimation after the administration of the first few items. In this study, two procedures are presented in an attempt to lessen this instability. A first one involves a change in the test format. A small-sized fixed-test is introduced as a starting premise for the subsequent sequential items administration, a test-form referred to as semi-adaptive. A second procedure relates to the application of an item selection criterion hitherto not used in connection with CAT: the D-optimality criterion. A polytomous CAT simulation is carried out to evaluate this criterion's performance under different test premises. Other established criteria based on Fisher's and Kullback-Leibler information are considered for comparative purposes.

The simulation shows that the extent of early stage instability depends primarily on the quality of the item pool information and its size, and secondarily on the item selection criteria. A comparison between fully and semi-adaptive tests reveals that the latter can considerably lessen the instability. The efficiency of D-optimality criterion is similar to the efficiency of other known item selection criteria, often yielding more stable estimates in the course of items administration.