Getting started with semTests

What semTests does

semTests computes p-values for the chi-square tests of structural equation models fitted with lavaan. When the data are not normal, the chi-square statistic \(T\) does not have a chi-square distribution, not even in large samples. Its limit is a weighted sum of independent \(\chi^2_1\) variables,

\[ T \xrightarrow{d} \sum_{j=1}^{d} \lambda_j Z_j^2, \qquad Z_1, \ldots, Z_d \sim N(0, 1), \]

where \(d\) is the degrees of freedom and \(\lambda_1, \ldots, \lambda_d\) are the nonzero eigenvalues of \(U\Gamma\). Here \(\Gamma\) is the asymptotic covariance matrix of the sample moments and \(U\) depends on the model and the estimator. Under normality every \(\lambda_j\) equals 1 and the chi-square reference is correct.

The Satorra–Bentler correction matches the mean of this distribution, and the scaled-and-shifted correction matches its mean and variance. semTests estimates the eigenvalues and uses all of them. Single eigenvalue estimates are noisy, so the recommended methods shrink them toward their common mean: penalized eigenvalue block averaging (pEBA) for overall fit (Foldnes, Moss, and Grønneberg, 2025), and penalized use of all eigenvalues (PALL) for nested comparisons (Foldnes, Grønneberg, and Moss, 2026).

The package has three entry points:

This vignette covers the first two for complete continuous data.

Overall fit

library("semTests")
library("lavaan")

model <- "
  visual  =~ x1 + x2 + x3
  textual =~ x4 + x5 + x6
  speed   =~ x7 + x8 + x9
"
fit <- cfa(model, HolzingerSwineford1939)
pvalues(fit)
#>    peba4_rls 
#> 5.165712e-07 
#> estimator: ML | data: continuous | information: expected | df: 24

The fit needs no robust lavaan test. When lavaan has not stored an estimate of \(\Gamma\), semTests computes one from the raw data. The line below the p-values records the estimator, the data type, the fit’s information option, and the degrees of freedom. attr(result, "semtests") holds the full record.

Choosing a test

A test name has up to three parts, as in PEBA4_UG_ML: the method, an optional UG, and an optional statistic. The methods are

Name Method
PEBA<j> Penalized eigenvalue block averaging with j blocks. Values from 2 to 6 usually work well; j cannot exceed the degrees of freedom.
POLS<g> Penalized regression with penalty g. Foldnes, Moss, and Grønneberg (2025) studied POLS2.
PALL All eigenvalues, penalized.
ALL All eigenvalues, unpenalized.
EBA<j> Eigenvalue block averaging without penalization (Foldnes and Grønneberg, 2018).
STD The ordinary chi-square reference.
SB The Satorra–Bentler scaled test (Satorra and Bentler, 1994).
SS The scaled-and-shifted test (Asparouhov and Muthén, 2010).
SF The scaled F test (Wu and Lin, 2016).

UG replaces the usual estimate of \(\Gamma\) with the unbiased estimate of Du and Bentler (2022). The statistic is ML, the normal-theory likelihood-ratio statistic, or RLS, Browne’s (1974) reweighted least-squares statistic. Both UG and RLS are defined only for complete continuous data fitted by maximum likelihood, which includes MLM and MLR fits. Names are case-insensitive.

For maximum likelihood fits, pvalues() defaults to PEBA4_RLS and pvalues_nested() to PALL_UG_ML, the combinations recommended in the two papers. Other fits use PEBA4 and PALL with their standard statistic.

A name without a statistic uses RLS on maximum likelihood fits. So SB here is the Satorra–Bentler correction applied to the RLS statistic, and lavaan’s Satorra–Bentler p-value is SB_ML:

pvalues(fit, c("STD_ML", "SB_ML", "SS_ML", "PEBA4_ML", "SB", "PEBA4_UG_RLS"))
#>       std_ml        sb_ml        ss_ml     peba4_ml       sb_rls peba4_ug_rls 
#> 8.502553e-09 4.416190e-08 2.746479e-07 1.529609e-07 1.726558e-07 6.669703e-07 
#> estimator: ML | data: continuous | information: expected | df: 24
fit_mlm <- cfa(model, HolzingerSwineford1939, estimator = "MLM")
as.numeric(fitMeasures(fit_mlm, "pvalue.scaled"))
#> [1] 4.41619e-08

Nested models

To test whether the two visual loadings that are not fixed for scaling are equal, fit the restricted model and pass it to pvalues_nested() together with the original fit. The restricted model goes first:

restricted <- "
  visual  =~ x1 + a*x2 + a*x3
  textual =~ x4 + x5 + x6
  speed   =~ x7 + x8 + x9
"
fit0 <- cfa(restricted, HolzingerSwineford1939)
pvalues_nested(fit0, fit, tests = c("SB_ML", "PALL_UG_ML"))
#>      sb_ml pall_ug_ml 
#> 0.05607963 0.05722534 
#> estimator: ML | data: continuous | information: expected | df: 1 | nested (method 2000, A.method weighted)

The test is Satorra’s (2000) difference test. Its restriction matrix defaults to A.method = "weighted", which works for any nesting, including models that fix or drop parameters. It depends neither on how the factors are identified nor on the units of the observed variables. lavaan’s own construction, A.method = "delta", differs only in using an unweighted projection, so its p-values change when a variable is rescaled. Use it to reproduce lavaan: for MLM fits, SB_ML with A.method = "delta" equals lavTestLRT(fit, fit0, method = "satorra.2000"). Here x1 is measured in tenths instead:

rescaled <- HolzingerSwineford1939
rescaled$x1 <- 10 * rescaled$x1
fit_rescaled <- cfa(model, rescaled)
fit0_rescaled <- cfa(restricted, rescaled)
sapply(c(weighted = "weighted", delta = "delta"), function(A) {
  c(
    original = pvalues_nested(fit0, fit, tests = "SB_ML", A.method = A)[[1]],
    rescaled = pvalues_nested(fit0_rescaled, fit_rescaled,
      tests = "SB_ML", A.method = A
    )[[1]]
  )
})
#>            weighted      delta
#> original 0.05607963 0.05940942
#> rescaled 0.05607961 0.05440472

A.method = "exact" is lavaan’s other alternative; see ?pvalues.

Both fits must use the same data, estimator, and fitting options. semTests checks this, and if the models arrive in the wrong order it warns and swaps them. It does not check that one model is a special case of the other.

Other estimators

GLS and ULS fits work the same way:

pvalues(cfa(model, HolzingerSwineford1939, estimator = "GLS"))
#>     peba4_ml 
#> 8.854857e-07 
#> estimator: GLS | data: continuous | information: expected | df: 24
pvalues(cfa(model, HolzingerSwineford1939, estimator = "ULS"))
#>     peba4_ml 
#> 2.527109e-08 
#> estimator: ULS | data: continuous | information: expected | df: 24

The other vignettes cover the remaining settings:

?semTests-support lists every supported combination of estimator, data type, and test.

References

Asparouhov, T., & Muthén, B. O. (2010). Simple second order chi-square correction. Mplus Technical Appendix. https://www.statmodel.com/download/WLSMV_new_chi21.pdf

Browne, M. W. (1974). Generalized least squares estimators in the analysis of covariance structures. South African Statistical Journal, 8, 1–24.

Du, H., & Bentler, P. M. (2022). 40-year old unbiased distribution free estimator reliably improves SEM statistics for nonnormal data. Structural Equation Modeling, 29(6), 872–887. https://doi.org/10.1080/10705511.2022.2063870

Foldnes, N., & Grønneberg, S. (2018). Approximating test statistics using eigenvalue block averaging. Structural Equation Modeling, 25(1), 101–114. https://doi.org/10.1080/10705511.2017.1373021

Foldnes, N., Grønneberg, S., & Moss, J. (2026). Penalized eigenvalue block averaging: Extension to nested model comparison and Monte Carlo evaluations. Behavior Research Methods, 58, article 107. https://doi.org/10.3758/s13428-026-02968-4

Foldnes, N., Moss, J., & Grønneberg, S. (2025). Improved goodness of fit procedures for structural equation models. Structural Equation Modeling, 32(1), 1–13. https://doi.org/10.1080/10705511.2024.2372028

Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In R. D. H. Heijmans, D. S. G. Pollock, & A. Satorra (Eds.), Innovations in Multivariate Statistical Analysis (pp. 233–247). Kluwer Academic. https://doi.org/10.1007/978-1-4615-4603-0_17

Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent Variables Analysis: Applications for Developmental Research (pp. 399–419). Sage.

Wu, H., & Lin, J. (2016). A scaled F distribution as an approximation to the distribution of test statistics in covariance structure analysis. Structural Equation Modeling. https://doi.org/10.1080/10705511.2015.1057733