SMABS 2004 Jena University
SMABS 2004 Home Organization About Jena Sponsors Links Imprint / Contact SMABS Home

European Association of Methodology

Department of methodology and evaluation research

Jena University

Contributions: Abstract

Robustness of factor mean comparisons across populations in the case when one factor indicator shows measurement non-invariance across populations.

Alain De Beuckelaer
Unilever Bestfoods NV
Belgium

This paper presents a simulation study which assesses the reliability of factor mean comparisons across two populations when one factor indicator fails to satisfy the condition of measurement invariance across populations. A one-factor Means And Covariance Structure (MACS) model with 3 or 4 indicators is specified, and five-point Likert-type scales are used.

Measurement invariance of indicators across populations is defined as the equality of factor loadings and indicator intercepts across populations. In "non-invariance conditions" the non-invariant indicator has at least one non-invariant measurement parameter (i.e. factor loading or indicator intercept) across populations. In "invariance conditions" all measurement parameters of all indicators are invariant across populations.

Factor mean scores for both populations are estimated under the (false) assumption that measurement invariance across populations holds for all indicators. The simulation experiment represents a full-factorial design using the following design factors: (1) the type of (empirical) distribution of the indicators, (2) the number of observations from each population, (3) the degree of non-measurement invariance of the (non-invariant) factor loading and / or indicator intercept, and (4) the size of the difference between the two population factor means (zero in some conditions).

A statistical difference test between the estimated factor means from both populations determines the correctness of each replication of simulated conditions (50 replications). Non-invariance conditions are considered "robust" if the number of correct statistical conclusions fall within a 99% confidence interval around the number of correct statistical conclusions of the corresponding invariance condition.

The main conclusion from the simulation study is that non-invariant measurement parameters, and in particular a non-invariant indicator intercept, form a serious threat to the robustness of the factor means difference test. It is therefore concluded that a test of measurement invariance (across populations) is crucial to ensure that factor mean comparisons across populations are meaningful.