The Performance of Ridge Regression, LASSO, and Elastic-Net in Controlling Multicollinearity: A Simulation and Application

Abstract

This research aims to compare the performance of Ordinary Least Square (OLS), Least Absolute Shrinkage and Selection Operator (LASSO), Ridge Regression (RR) and Elastic-Net in controlling multicollinearity problems between independent variables in multiple regression analysis using simulation data and case data. Data simulation uses a multiple regression model with p = 6 with a high level of multicollinearity (ρ = 0.99) at several sample sizes (n = 25, 50, 75). The best method is measured based on the smallest Average Mean Square Error (AMSE) and AIC values. The research results show that Elastic-Net is the best method for simulated data compared to LASSO and Ridge because it has the smallest AMSE and AIC values for each sample size studied. Similar things were also obtained when applying these three methods to data on stunting toddler cases in Indonesia which had high multicollinearity. By using the best method, namely the Elastic Net method, real data shows that cases of stunted toddlers in Indonesia are influenced by the percentage of toddlers who are malnourished (????????), the percentage of toddlers who receive exclusive breast milk (????????), the percentage of toddlers whose growth is monitored (????????), coverage of health services for pregnant women (????????), number of nutrition workers (????????), percentage of households with adequate drinking water (????????), percentage of households with adequate sanitation (????????????), human development index (????????????), and population density (????????????).