Structural equation models with intensive longitudinal data quickly become very extensive – with numerous manifest variables and parameters to be estimated. The evaluation of structural equation models relies on fit indices such as χ², CFI, TLI or RMSEA and rules of thumb when these indices imply that the data fit the model well enough. Those rules of thumb (or cut-off values) are derived from simulation studies with much smaller models, such as Hu & Bentler (1999) who used models with 15 manifest variables loading on three latent constructs. However, with more variables in the model, the χ² estimate is inflated, CFI and TLI tend to worsen, while RMSEA tends to improve (Kenny & McCoach, 2003). Consequently, relying on common cut-off values for structural equation models with intensive longitudinal data will likely lead more often to the rejection of models that should be acceptable.

Therefore, we want to show alternative ways to evaluate model fit for large structural equation models with intensive longitudinal data. More precisely, we first suggest block-wise model fit assessment. The model-implied covariance matrix and model parameters are estimated based on the entire model. Subsequently, fit indices are calculated based on smaller blocks (e.g. days in experience sampling data) of the covariance matrix. We illustrate this strategy by applying it to data from the Conflict of Interests scale of the Interdependence in Daily Life Study (Columbus, Molho, Righetti & Balliet, 2018). After having a social interaction, participants scored the conflict of interest in that situation on a two-item scale. Participants were asked to respond on 49 measurement occasions over seven days. The data is modelled in a latent-state-trait theory framework (Steyer, Schmitt & Eid, 1999; Steyer, Mayer, Geiser & Cole, 2015) with day- and indicatorspecific traits (Eid, Courvoisier & Lischetzke, 2012) and autoregressive effects (Eid, Holtmann, Santangelo & Ebner-Priemer, 2017). Fit measures are then provided for each day.

Additionally, a simulation study is conducted to investigate how fit indices behave differently in large structural equation (latent-state-trait) models with experience sampling data compared to smaller models. The structural equation model used for the simulation study is based on the same real-data example where data was collected seven times a day on seven days. We compare how fit indices change in the simulated data for two model sizes: a smaller model for two days (28 manifest variables) and a large model for all seven days (98 manifest variables). We also compare different sample sizes and different types and degrees of misspecification (no misspecification, correlated errors within blocks, correlated errors between blocks, structural misspecification). We manage to replicate the bias of common fit indices in large models, and can show that block-wise indices to not follow this bias.

References

Columbus, S., Molho, C., Righetti, F., & Balliet, D. (2018). Interdependence in Daily
Life

Eid, M., Courvoisier, D. S., & Lischetzke, T. (2012). Structural equation modeling of ambulatory assessment data.

Eid, M., Holtmann, J., Santangelo, P., & Ebner-Priemer, U. (2017). On the definition of latent-state-trait models with autoregressive effects. European Journal of Psychological Assessment.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1), 1-55.

Kenny, D. A., & McCoach, D. B. (2003). Effect of the number of variables on measures of fit in structural equation modeling. Structural equation modeling, 10(3), 333-351.

Steyer, R., Mayer, A., Geiser, C., & Cole, D. A. (2015). A theory of states and traits—Revised. Annual review of clinical psychology, 11, 71-98.

Steyer, R., Schmitt, M., & Eid, M. (1999). Latent state–trait theory and research in personality and individual differences. European Journal of Personality, 13(5), 389-408.