Estimating the Linear Regression Model in High-Dimensional Data and collinearity
Hassan, S., Helmyand, N., Elbadawy, A. (2020). Estimating the Linear Regression Model in High-Dimensional Data and collinearity. EKB Journal Management System, 24(1), 67-98. doi: 10.21608/jsfc.2020.248230
Sahar Hassan; Nahed Helmyand; Amira Elbadawy. "Estimating the Linear Regression Model in High-Dimensional Data and collinearity". EKB Journal Management System, 24, 1, 2020, 67-98. doi: 10.21608/jsfc.2020.248230
Hassan, S., Helmyand, N., Elbadawy, A. (2020). 'Estimating the Linear Regression Model in High-Dimensional Data and collinearity', EKB Journal Management System, 24(1), pp. 67-98. doi: 10.21608/jsfc.2020.248230
Hassan, S., Helmyand, N., Elbadawy, A. Estimating the Linear Regression Model in High-Dimensional Data and collinearity. EKB Journal Management System, 2020; 24(1): 67-98. doi: 10.21608/jsfc.2020.248230

Estimating the Linear Regression Model in High-Dimensional Data and collinearity

المجلة العلمية لقطاع کليات التجارة بجامعة الأزهر
Article 6, Volume 24, Issue 1, June 2020, Page 67-98  XML PDF (1.42 MB)
Document Type: المقالة الأصلية
DOI: 10.21608/jsfc.2020.248230
View on SCiNiTO View on SCiNiTO
Authors
Sahar Hassan* ; Nahed Helmyand; Amira Elbadawy
Department of statistics, Faculty of Commerce (Girls’ Branch), Al-Azhar University, Cairo, Egypt
Abstract
This paper is concerned with introducing the most used penalized regression methods, including ridge regression (RR), least absolute shrinkage and selection operator (LASSO), and elastic net (EN) regression for estimating the linear regression model. These models are used in two cases low and high-dimensional data when data iscontain outliers when the explanatory variables have collinearity among them. The Monte Carlo simulation study is conducted to evaluate and compare the performance of these estimators. The simulation results indicate that the obtained estimators using EN are efficient and reliable than the other estimators.
Keywords
penalized regression; Ridge regression; least absolute shrinkage and selection operator; Elastic net; High-dimensional data; Collinearity; Outliers
References
  1. Adelegoke, A.Adewuyi, E.Ayinde,K. and Lukman,A. (2016). “A Comparative Study of some Robust Ridge and Liu Estimators”, Science World journal,Vol. 11, pp.16-20.
  2. Alauddin,M. and Nghiemb,H. (2010). “Do Instructional Attributes Pose Multicollinearity Problems? An Empirical Exploration.” Economic Analysis and Policy, Vol. 40, pp. 351–361.
  3. Alkan, B.B, Atakan, C. (2013). Visualizing Diagnostic of Multicollinearity: Table Plot and Biplot Methods. Pakistan Journal of statistics, Vol. 29, pp. 59-78.
  4. Arslan, O. (2012). Weighted LAD-LASSO method for robust parameter estimation and variable selection in regression. Computational Statistics and Data Analysis, Vol 56, 1952– 1965.
  5. Emmert-Streib, F. and Dehmer,M. (2019). High-Dimensional LASSO-Based Computational Regression Models: Regularization, Shrinkage, and Selection. Machine Learning and knowledge Extraction, Vol. 1, pp. 359-383.
  6. Fan, J. and Li, R. (2001) Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties. JASA,Vol 96, 1348-1360.
  7. Fonti,V. and Belitser, E. (2017). “Feature Selection Using LASSO”, VrijeUniversiteit Amsterdam.
  8. Hebiri,M. and Van,S. (2011). “The Smooth-Lasso and other 12l1+l2"> -Penalized Methods”. arXiv:1003.4885v2 [math.ST]
  9. Hoerl, A. E. (1962). Application of Ridge Analysis to Regression Problems. Chemical Engineering Progress, Vol 58, no 3, pp. 54-59.
  10. Hoerl, A.E. and Kennard, R. W. (1970a). Ridge Regression: Biased Estimation for Non orthogonal Problems. Technometrics, Vol. 12, no. 1, pp. 55 - 67.
  11. Hoerl, A.E. and Kennard, R. W. (1970b). Ridge Regression: Application to Non-Orthogonal Problems. Technometrics, vol. 12, no. 3, pp. 591 - 612.
  12. Huang, J., Jiao, Y., Liu, Y. and Lu, X. in (2018). A Constructive Approach to 12l0"> Penalized Regression, Machine Learning Research, Tong Zhang.
  13. Hui andZou. (2006). “The Adaptive Lasso and its Oracle Properties”. Journal of the American statistical association, Vol. 101, no. 476, pp.1418–1429.
  14. Januaviani,T. M., Gusriani, N., Subiyanto and Bon,A. T. (2019). “The LASSO (Least Absolute Shrinkage and Selection Operator) Method to Predict Indonesian Foreign Exchange Deposit Data” Proceedings of the International Conference on Industrial Engineering and Operations ManagementBangkok, Thailand.pp.3195-3202
  15. James,G.Witten,D. Hastie,T. and Tibshirani,R. (2013). An Introduction to Statistical Learning with Applications in R. Springer New York.  
  16. Karl.W and Simar.L (2015). Applied Multivariate Statistical Analysis, Springer.
  17. Melkumova, L. E. and Shatskikh, S.Y. (2017). “Comparing Ridge and LASSO Estimators for Data Analysis”, Procedia Engineering 201 pp.746–755
  18. Neter, J., Wasserman,W. and Kutner,M.H. (2005). Applied Linear Statistical Models. McGraw-Hill/Irwin New York. 
  19. Osborne M, Presnell B, Turlach B (2000). “A New Approach to Variable Selection in LeastSquares Problems.” IMA Journal of Numerical Analysis, Vol20, p.p389–404.
  20. Saleh,E., Arashi,M. and Kibria,G. (2019). Theory of Ridge Regression Estimation with Applications. WILEY.
  21. Tibshirani, R. (1996). “Regression Shrinkage and Selection Via the Lasso”. Journal of the Royal Statistics. Vol. 58, pp. 267-288.
  22. Tibshirani, R. (1997).  The Lasso Method for Variable Selection in The COX Model. Statistics in Medicine, Vol. 16, pp.385—395.
  23. Weige, Yu.et al. (2019). “Analysis of Vegetable Price Fluctuation Law and Causes based on Lasso Regression Model”. Journal of Physics: Conference Series.

24. Zou,H., Hastie,T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol, Vol 67,p.p 301–320.

25. Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association,Vol101, p.p 1418–1429.

Statistics
Article View: 115
PDF Download: 155