SMABS 2004 Jena University
SMABS 2004 Home Organization About Jena Sponsors Links Imprint / Contact SMABS Home

European Association of Methodology

Department of methodology and evaluation research

Jena University

Contributions: Abstract

Statistical performance of the I2 index in assessing heterogeneity in meta-analysis

Tania Huedo-Medina Fulgencio Marín-Martínez
Julio Sánchez-Meca
University of Murcia
Spain

Meta-analysis aims to integrate the results of a set of studies about a common topic (Hedges and Olkin, 1985; Rosenthal,1991; Glass et al., 1981; Hunter and Smith, 1990). One of the main purposes in a meta-analysis is to assess the heterogeneity of the results of the integrated studies. The usual way of assessing the extent of heterogeneity in a meta-analysis has been the Q statistic (Cochran, 1954).

However, the Q statistics is a significance test that only informs about the presence or absence of heterogeneity in a population, without indicating the magnitude of such heterogeneity. Recently, Higgins et al. (2002) have proposed a new index, I2, which is easily interpretable and quantifies the degree of heterogeneity following scale invariance independent of the type of outcome data (eg. dichotomous of quantitative) and choice of effect measure (eg. standardized mean difference, odds ratio) in the meta-analysis.

By means of Monte Carlo simulation and assuming a random-effect model, we assessed the bias and efficiency of the I2 index when applied on different effect size indexes and through different number of studies in the meta-analyses. Deviations of our findings with respect to the theoretically expected performance of I2 are discussed.

References

Hedges, L.V. and Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.
Hunter, J.E. and Schmidt, F.L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
Rosenthal, R. (1991). Meta-Analytic Procedures for Social Research (revised edn). Newbury Park, CA: Russell Sage Foundation.
Glass, G.V., McGaw, B. and Smith, M. L. (1981). Meta-Analysis in Social Research. Beverly Hills, CA: Russell Sage Foundation.
Cochran, W.G. (1954). The combination of estimates from different experiments. Biometrics, 10, 101-129.
Higgins, J.P.T., and Thompson, S.G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539-1558.