SMABS 2004 Jena University
SMABS 2004 Home Organization About Jena Sponsors Links Imprint / Contact SMABS Home

European Association of Methodology

Department of methodology and evaluation research

Jena University

Contributions: Abstract

Dynamic test administration using missing data imputation: a model-free way of adaptive testing

Andreas WolfUlf Kröhne
University of Jena
Germany

Many large-scale surveys suffer from problems like non-compliance due to high respondent's burden, ongoing rigidity in answering behaviour, lack of concentration or displeasure of respondents. The selectivity of the sample increases with the test length as a - presumably non-random - share of subjects is not willing to complete lengthy questionnaires.

A desirable solution would be a significant reduction of the test length with minimal loss of information and statistical power, and the opportunity to calculate unbiased estimates. The aim of the proposed approach is to optimally distribute item sets to different test-takers without prior knowledge of item interrelationships (as necessary for "classical" Multiple Matrix Sampling).

The proposed solution is a flexible tool for test administration - call it an "individualized" Multiple Matrix Sampling - using (a) empirical relationships in the data based on an optimization criterion such as the multiple correlation of item blocks from all previous test participants, and (b) the progress in multiple imputation techniques. Item selection is determined by a probabilistic function based on the coefficient of determination of all items not yet administered for this person on evidence of all administered items. Afterwards, missing values are imputed on manifest item level using multiple imputations.

First results using simulated data are presented. The proposed test administration model is compared to different sampling techniques, and similarities and differences to adaptive testing are shown. The current approach offers a data-based optimization of the sampling problem for a matrix-sampling approach on the individual level.