Ordinary Least Squares: the Adequacy of Linear Regression Solutions under Multicollinearity and without it
Tyzhnenko A. G., Ryeznik Y. V.

Tyzhnenko, Alexander G., and Ryeznik, Yevgen V. (2019) “Ordinary Least Squares: the Adequacy of Linear Regression Solutions under Multicollinearity and without it.” The Problems of Economy 1:217–227.
https://doi.org/10.32983/2222-0712-2019-1-217-227

Section: Mathematical methods and models in economy

UDC 330.43 (075.8)

Abstract:
The article deals with the problem of economic adequacy of solving a linear regression problem by the OLS method. The study uses the following definition of adequacy: a linear regression solution is considered adequate if it not only has correct signs but also correctly reflects the relationship between coefficients of regression in the population. If in this case the coefficient of determination is greater than 0.8, the solution is considered economically adequate. As an indicator of adequacy of a linear regression problem solution it is proposed to use a 10 % level of the coefficient of variability (CV) of the regression coefficients. It is shown that OLS solutions may be not adequate to the solution in the population, although they may be physically correct (with correct signs) and statistically significant. The mentioned result is obtained by using the artificial data population (ADP) algorithm. The ADP allows generating data of any size with known regression coefficients in the whole population, which can be calculated with the aid of the OLS solution for a very large sample. The ADP algorithm makes it possible to change the regular component of the influence of the regressors on the response. Besides, the random changes of regressors in the ADP are divided into two parts. The first part is coherent to the response changes, but the second part is completely random (incoherent). This one allows changing the near-collinearity level of the data by changing the variance of the incoherent noise in regressors. Studies using ADP have shown that with a high probability the OLS solutions are physically incorrect if the sample sizes (n) are less than 23; physically correct but not adequate for 23 < n < 400; adequate for n > 400. Furthermore, it is noted that if the elimination of strongly correlated regressors is not economically justified but is rather a measure of lowering the value of the VIF-factor, the results may be far from the reality. In this regard, it is stated that the use of the MOLS eliminates the need to exclude strongly correlated regressors at all, since the accuracy of the MOLS solution increases with an increase in the VIF.

Keywords: multicollinearity, OLS, data simulation, artificial population, physical correctness, adequacy.

Tabl.: 3. Formulae: 5. Bibl.: 29.

Tyzhnenko Alexander G. – Candidate of Sciences (Physics and Mathematics), Associate Professor, Associate Professor, Department of Mathematics and Mathematcal Methods in Economics, Simon Kuznets Kharkiv National University of Economics (9a Nauky Ave., Kharkіv, 61166, Ukraine)
Email: olersandr.tyzhnenko@m.hneu.edu.ua
Ryeznik Yevgen V. – Candidate of Sciences (Physics and Mathematics), Lecturer, Department of Mathematics, Uppsala University (buildings 1, 6 and 7, 1 Lagerhyddsvagen, Uppsala, 75106, Sweden)
Email: yevgen.ryeznik@math.uu.se

List of references in article

Tyzhnenko, A. G. “A new stable solution to the linear regression problem under multicollinearity“. Economics of Development. 2018. http://www.ed.ksue.edu.ua/ER/knt/ee182_86/e182tyz.pdf
Seber, G. A. F. Linear Regression Analysis. New York: Wiley-Blackwell, 1977.
Seber, G. A. F., and Lee, A. J. Linear Regression Analysis. New York: Wiley, 2003.
Spanos, A. Probability Theory and Statistical Inference: econometric modeling with observational data. Cambridge: Cambridge University Press, 1999.
Gujarati, D. N. Basic econometrics. New York: McGraw-Hill, 2002.
Wooldridge, J. M. Introductory Econometrics: Modern Approach. Ohio: South-Western, 2009.
Baltagi, B. Econometrics. New York: Springer, 2011.
Greene, W. H. Econometric Analysis. New York: Pearson, 2012.
Draper, N., and Smith, H. Applied Regression Analysis. New York: Wiley, 1966.
Farrar, D., and Glauber, R. “Multicollinearity in Regression Analysis: The problem revisited“. Review of Economics and Statistics, vol. 49 (1967): 92-107.
Hoerl, A. E., and Kennard, R. W. “Ridge regression: Biased estimation for nonorthogonal problems“. Technometrics, no. 12 (1) (1970): 55-67.
Marquardt, D. V. “Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation“. Technometrics, no. 12 (1970): 591-612.
Blanchard, O. J. “Comment“. Journal of Business and Economic Statistics, no. 5 (1987): 449-451.
Adkins, L. C., and Hill, R. C. “Collinearity“. In Companion in Theoretical Econometrics, 256-278. Oxford: Blackwell Publishers, Ltd., 2001.
Belsley, D. A., Kun, E., and Welsh, R. T. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York: Wiley, 2004.
Belsley, D. A. “Demeaning conditioning diagnostics through centering“. The American Statistician, vol. 38 (2) (1984): 73-77.
Rao, C., and Toutenberg, H. Linear Models: Least Squares and Alternatives. New York: Springer, 1999.
Spanos, A., and McGuirk, A. “The Problem of Near-Multicollinearity Revisited: erratic vs. systematic volatility“. Journal of Econometrics, vol. 108 (2002): 365-393.
Adkins, L. C., Waters, M. S., and Hill, R. C. “Collinearity Diagnostics in gretl“. In Economics Working Paper Series 1506. Oklahoma: Oklahoma State University, Department of Economics and Legal Studies in Business, 2015.
Tikhonov, A. N. “On the stability of inverse problems“. In Doklady Acad. Sci. USSR, vol. 39 (1943): 176-179.
Tikhonov, A. N., and Arsenin, V. Y. Solutions of Ill-Posed Problems. New York: Winston & Sons, 1977.
Kabanichin, S. I. “Definitions and Examples of Inverse and Ill-posed Problems“. J. Inv. Ill-Posed Problems, vol. 16 (2008): 317-357.
Harvey, A. C. “Some Comments on Multicollinearity in Regression“. Applied Statistics, vol. 26 (2) (1977): 188-191.
Kutner, M. H., Nachtsheim, C. J., and Neter, J. Applied Linear Regression Models. New York: McGraw-Hill / Irwin, 2004.
Sheather, S. J. A modern approach to regression with R. New York: Springer, 2009.
Moore, D. S., Notz, W. I., and Flinger, M. A. The basic practice of statistics. New York: W. H. Freeman and Company, 2013.
Dougherty, C. Introduction to Econometrics. New York: Oxford University Press, 1992.
Maddalla, G. S. Introduction to Economics. New York: Macmillan, 1992.
Kennedy, P. E. “Oh no! I got the wrong sign! What should I do?“ The Journal of Economic Education, vol. 36, no. 1 (2005): 77-92.