Many times, the terms principal components and factors analysis are often confused, and sometimes used as synonyms. However, there is a technical distinction in that a principal component can be explicitly defined as a unique combination of variables, while a factor cannot be. Because of this, some statisticians treat principal components and factor analysis as completely disparate methods.
- The latent variable model of Factor Analysis was first introduced by Spearman in 1904 for one factor and in 1947 for more than one factor by Thurstone.
- The PCA model was invented more than 100 years ago by Karl Pearson, and it has been widely used in diverse fields since then.
Both are dimension reduction techniques, but while Principal Component Analysis is used to reduce the number of variables by creating principal components, extracting the essence of the dataset in the means of artificially created variables, which best describe the variance of the data, Factor Analysis tries to identify, unknown latent variables to explain the original data. Often principal axis factoring is used when there is interest in studying relations among the variables, while principal components is used when there is a greater emphasis on data reduction and less on interpretation.
This said, there are some fundamental differences between of them, let’s see it!