SMABS 2004 Jena University
SMABS 2004 Home Organization About Jena Sponsors Links Imprint / Contact SMABS Home

European Association of Methodology

Department of methodology and evaluation research

Jena University

Contributions: Abstract

Analytical, numerical, and visual investigations of parameter invariance in IRT models

AndrĂ© A. Rupp Bruno D. Zumbo
University of Ottawa University of British Columbia
Canada

This paper presents analytical, numerical, and visual arguments that clarify what is meant by item parameter invariance in item response theory (IRT) models and how a lack of parameter invariance (LOI) impacts practical decision-making and model choice. Item parameter invariance refers to the identity of item parameters across different examinee populations or measurement conditions under a given model and is a theoretical property of latent variable models under ideal conditions of perfect model fit. In practice, LOI is common, however, as exemplified by numerous investigations of differential item or bundle functioning (e.g., Clauser & Mazor, 1998) and item parameter drift (e.g., Wells, Subkoviak, & Serlin, 2002). To help clarify the sometimes confusing implications of parameter invariance for modeling practice, this research investigates three complementary questions based on a statistical formalization of a LOI.

First, it is shown, both theoretically and with real data, how using a correlation coefficient to compare item parameter estimates from different populations or measurement conditions, as if sometimes done in simulation studies (e.g., Fan, 1998; McDonald & Paunonen, 2002) is technically inappropriate to detect a LOI as additive shifts in parameter values and nonlinear relationships between estimates remain undetected. Second, using the mathematical structure of the two-parameter logistic IRT model, it is shown what kinds of biases get introduced due to a LOI and how their effect on response probabilities can be directly computed and visualized. Third, that investigation is generalized to a cross-model comparison where it is shown how, practically, the three basic unidimensional IRT models are relatively robust toward different degrees of LOI but how, theoretically, neither one is universally optimal under all LOI conditions that can be considered. Taken together, these results can aid practitioners and theoreticians alike in engaging in meaningful discussions about latent variable model properties.