A Review of Penalized Regression and Machine Learning Methods in High-Dimensional Data
ElSheikh, A., Abonazel, M., Ali, M. (2025). A Review of Penalized Regression and Machine Learning Methods in High-Dimensional Data. EKB Journal Management System, 69(1), 250-261. doi: 10.21608/esju.2025.368665.1080
Ahmed ElSheikh; Mohamed R. Abonazel; Mohamed Cherif Ali. "A Review of Penalized Regression and Machine Learning Methods in High-Dimensional Data". EKB Journal Management System, 69, 1, 2025, 250-261. doi: 10.21608/esju.2025.368665.1080
ElSheikh, A., Abonazel, M., Ali, M. (2025). 'A Review of Penalized Regression and Machine Learning Methods in High-Dimensional Data', EKB Journal Management System, 69(1), pp. 250-261. doi: 10.21608/esju.2025.368665.1080
ElSheikh, A., Abonazel, M., Ali, M. A Review of Penalized Regression and Machine Learning Methods in High-Dimensional Data. EKB Journal Management System, 2025; 69(1): 250-261. doi: 10.21608/esju.2025.368665.1080

A Review of Penalized Regression and Machine Learning Methods in High-Dimensional Data

The Egyptian Statistical Journal
Volume 69, Issue 1, June 2025, Page 250-261  XML PDF (1.42 MB)
Document Type: Review
DOI: 10.21608/esju.2025.368665.1080
View on SCiNiTO View on SCiNiTO
Authors
Ahmed ElSheikh1; Mohamed R. Abonazelorcid 1; Mohamed Cherif Ali email 2
1Department of Applied Statistics and Econometrics, Faculty of Graduate Studies for Statistical Research, Cairo University, Egypt.
2Faculty of Business Administration, Deraya University, Minya, Egypt
Abstract
In recent years, penalized regression techniques and machine learning methods have emerged as powerful tools for statistical modeling, particularly in high-dimensional data analysis. Penalized regression methods, such as Ridge, least absolute shrinkage and selection operator, and Elastic Net, mitigate multicollinearity and overfitting through regularization, enhancing model stability, accuracy, and interpretability. Meanwhile, machine learning techniques, including decision trees, random forests, support vector machines, and neural networks, provide strong predictive capabilities across various applications, research domains, and real-world case studies. This review systematically examines these methodologies, discussing their theoretical foundations, advancements, practical implementations, and computational efficiency. A comparative analysis highlights their strengths, limitations, and performance in different analytical contexts. Additionally, emerging hybrid techniques that integrate penalized regression with machine learning are explored, demonstrating their potential to improve model efficiency, scalability, accuracy, and interpretability. The review concludes that combining these approaches offers a robust framework for handling complex, high-dimensional data, making them valuable tools for modern statistical analysis, predictive modeling, and data-driven decision-making.
Keywords
Ridge Regression; LASSO; Elastic Net; Random Forests; Support Vector Machines
Statistics
Article View: 177
PDF Download: 200