Chapter 11 Dealing with Too Many Variables

In the previous chapter, one variable is measured in single dimension. However, we often have multiple measurements of the one variable (e.g., multi-item rating in a survey). You want to combine these multiple ratings into a composite to measure the variable. Whether it is reasonable to combine them depends on the correlations among the ratings.

But how this correlation as a group instead of correlation between two ratings determines the reliability of the composite? Principal component analysis (PCA) is a method to deal with this problem by examining the internal correlation structure and present in a compact way. For instance, let’s assume that there are five input variables (five principal components). If there is common variance among the input variables, the most common variance is explained by the first principal component. Whatever else is left is explained by the second principal component, and so on until the last component explains the remaining variance. In this way, we can reduce the number of variables by combining them into a smaller number of principal components. A component is a linear combination (a set of loadings) of the original variables. Other exploratory factor analysis methods such as factor analysis and exploratory factor analysis are also used to reduce the number of variables. In this chapter, we will focus on PCA.