Nhlanhla Ndebele
This blog is based on part of my PhD research focusing on the measurement of quality of work and employment (QWE) of UK employees and modelling how this has changed over time. This might be of particular interest to students who want to pursue studies in advanced quantitative methods as well as researchers conducting research focusing on measuring unobservable or latent concepts.
Standard analysis of the labour market tends to focus on quantity, but there is an increasing interest in the quality of jobs created. However, there is no consensus on the conceptualisation and operationalisation of QWE, demonstrating not only the novelty of research and policy development in this area, but also the considerable challenges in measuring this concept (Muñoz de Bustillo et al. 2011). Quality, in general, is a complex concept to measure as it is unobservable. To measure such a concept, observable variables thought to be indicators of different aspects of the concept are measured and methods, such as latent variable modelling (Bartholomew et al. 2008), applied to create the measure. A further complication is that QWE as a concept is multidimensional; thus, employers offer jobs made up of bundles of rewards; with the general or overall QWE being a combination of these bundles or dimensions (Muñoz de Bustillo et al. 2011) but it is debatable how this should be modelled.
This blog focuses on the comparison of measurement models of QWE using data from Wave 8 (2016 – 2018) of Understanding Society: The UK Household Longitudinal Study (UKHLS) (University of Essex, Institute for Social and Economic Research 2018). It is only limited to model comparison to show which measurement model best fits the data and reports global fit statistics but not the model estimates. The sample (n = 17,196) was limited to employees over 16 years old in a paid job and the data had 20 indicators / items measuring different aspects of QWE. The indicators had ordinal levels of measurement (mixture of dichotomous and polytomous) and their descriptions are shown in Table 1. These are grouped by dimension of QWE based on a conceptual framework developed from different schools of thought of the social sciences (Muñoz de Bustillo et al. 2011).
Exploring Dimensionality of the Data
To better understand the dimensionality of the data, I examined the polychoric correlation matrix of the indicators. The weighted coefficients are displayed in a heat map (Figure 1), with darker colours indicating higher correlations. The coefficients were generally positive and if negative, they were close to zero. Based on Cohen's (1988) rule of thumb, economic compensationindicators were weak to moderately correlated, while training and progression indicators had a strong relationship as did working conditions indicators. Employment security indicators had a moderate relationship, while the relationship between work-life balance indicators was weak to strong. Furthermore, there were also weak to strong relationships between some indicators in different dimensions.
Figure 1: Heat Map of Weighted Polychoric Correlation Matrix
The polychoric correlation matrix suggested that in addition to the hypothesised dimensions, there might also be an association between the dimensions or an overall factor explaining the relationship between the indicators. Therefore, measurement models should consider dimensions of QWE, their associations and / or an overall factor of QWE.
Measurement Models
I considered four measurement models, thus 1) unidimensional model, 2) correlated factors model, 3) second-order factor model,and 4) bifactor model. The path diagrams for these models are shown in Figure 2 (a – d), with indicators; Item1 – Item20; being the observable variables, and latent variables G (general factor) measuring overall QWE, specific factors S1 measuring economic compensation, S2 measuring training and progression, S3 measuring employment security, S4 measuring working conditions, and S5 measuring work-life balance.
The unidimensional model (Figure 2 (a)) hypothesises that the covariance between a set of item responses can be explained by one general factor accounting for the common variance shared by all the items (Chen and Zhang 2018). In a correlated factors model (Figure 2 (b)), the covariance between a set of item responses is explained by multiple correlated specific factors (Chen and Zhang 2018) accounting for the common variance shared by items within that specific factor. The second-order factor model(Figure 2 (c)) is similar to the correlated-factors model but hypothesises that there is a higher order factor; second-order (or general) factor; consisting of lower order factors; first-order (or specific) factors; accounting for the common variance shared by the items. The specific factors are assumed to be orthogonal (Chen et al. 2012). Like in a unidimensional model, a bifactor model(Figure 2 (d)) hypothesises that the covariance between a set of item responses can be explained by a general factor accounting for the common variance shared by all the items, but also multiple specific factors accounting for any remaining common variance among items within that specific factor over and above the general factor (Chen and Zhang 2018; Reise 2012). The general andspecific factors are assumed to be orthogonal (Reise 2012).
In all these models, given the general and / or specific factors, the item responses are conditionally independent (Bartolucci, Bacci, and Gnaldi 2016; Raykov and Marcoulides 2018). The unidimensional, correlated factors, and second-order factor models are all nested within the bifactor model. The unidimensional model can be derived from the bifactor model by fixing all factor loadings from the specific factors in the bifactor model to zero, while fixing the factor loadings from the general factor in the bifactor model to zero and relaxing the orthogonality constraint on the specific factors will reduce the bifactor model to a correlated-factors model (Chen and Zhang 2018). The bifactor model can be reduced to a second-order factor model by fixing direct effects of the general factor on the items in the bifactor model to zero and introducing indirect effects of the general factor on the items through the specific factors (Yung, Thissen, and McLeod 1999).
Figure 2: Path Diagrams for Measurement Models
The models were estimated using the item response theory framework which is a class of latent variable models appropriate if the observed variables are categorical, and the latent variables are assumed to be continuous (Bartholomew et al. 2008). Goodness-of-fit measures of the models were examined using the cut-off criteria for approximate fit indices; root mean square error of approximation (RMSEA), standardised root mean square residual (SRMSR), Tucker-Lewis index (TLI), and comparative fit index(CFI); in Table 2.
Table 2: Cut-off Criteria for Approximate Fit Indices
Criterion
| *RMSEA | *SRMSR | **TLI | **CFI |
Adequate fit | ≤ 0.089 | ≤ 0.05 | ≥ 0.95 | ≥ 0.95 |
Close fit | ≤ 0.05 | ≤ 0.027 |
|
|
Excellent fit | ≤ 0.05 / (k – 1) | ≤ 0.027 / (k – 1) |
|
|
Notes: k is the number of categories. * Maydeu-Olivares and Joe (2014). ** Kline (2016).
From Table 3, overall fit (M2) tests for all the models were statistically significant (p-values < 0.001) suggesting that the overall models did not fit the data, however this test is sensitive to sample size and likely to yield statistically significant results for large samples (Morizot, Ainsworth, and Reise 2007). For approximate fit indices, the RMSEA suggested adequate fit for the unidimensional, correlated factors, and second-order factor models (upper 90% CIs < 0.089) and a close fit for the bifactor model (RMSEA = 0.036, 90% CI [0.035, 0.037]). However, for the other indices; SRMSR, TLI, and CFI; the unidimensional, correlated factors, and second-order factor models had poor fit, while the bifactor model had adequate fit, the SRMSR was < 0.05 and the TLI and CFI were both > 0.95. If the other three models had adequate fit, the likelihood ratio test would have been used to determine the most parsimonious model as the models are nested within the bifactor model, however in this instance the bifactor model was retained.
Model | M2 | df | RMSEA [90% CI] | SRMSR | TLI | CFI |
Unidimensional model | 12605.32*** | 156 | 0.072 [0.071, 0.073] | 0.094 | 0.839 | 0.857 |
Correlated factors model | 10907.81*** | 146 | 0.070 [0.068, 0.071] | 0.099 | 0.851 | 0.877 |
Second-order factor model | 11407.32*** | 156 | 0.069 [0.068, 0.070] | 0.099 | 0.855 | 0.871 |
Bifactor model | 2837.45*** | 136 | 0.036 [0.035, 0.037] | 0.039 | 0.960 | 0.969 |
* p < 0.05. ** p < 0.01. *** p < 0.001.
The empirical evidence from this short analysis suggests the bifactor model fits this data better compared to other models considered here. The bifactor model has some advantages in modelling QWE compared to the unidimensional, correlated factors, and second-order factor models. While the unidimensional model is a useful representation of the general factor and the correlated factors model is useful if the interest is in specific factors, both the second-order factor and bifactor models are useful if the general and specific factors are of interest (Brown and Croudace 2015), as is the case in the study of QWE. However, the orthogonal nature of the general and specific factors in a bifactor model mean that 1) we can study the dimensions of QWE independent of overall QWE, whereas in the second-order factor model, these dimensions are dependent on overall QWE, 2) the relationship between overall QWE and the items, as well as the dimensions of QWE and the items can be directly examined, while in a second-order factor model there is no direct relationship between overall QWE and the items, and 3) latent mean differences for both overall QWE and dimensions of QWE over and above overall QWE can be compared between groups in a bifactor model; subject to adequate measurement invariance; but only the latent mean differences of overall QWE between groups can be compared in a second-order factor model (Chen et al. 2012).
Conceptually, QWE can be better modelled by the bifactor model as this not only models overall QWE but also different dimensions of QWE over and above overall QWE, and the bifactor model was a better fit for the data compared to the other models.
Nhlanhla Ndebele is a PhD Researcher / Q-Step Teaching Assistant with the Q-Step Centre at City, University of London.
Comments
Post a Comment