When Do Logistic Regression and Generalized Maximum Entropy Agree? Evidence from Simulation Studies

Abstract

Logistic regression is widely used for binary outcomes, yet generalized maximum entropy (GME) estimators are often recommended when data are limited or ill-conditioned. Applied guidance on when GME and logistic regression provide practically similar predictive performance—particularly probability calibration—remains limited. We conducted a Monte Carlo study comparing SAS PROC LOGISTIC (maximum likelihood) with SAS PROC ENTROPY (GME-D; GMED) across four baseline scenarios: (S1) large well-specified samples (n = 5000), (S2) small samples with high collinearity (n = 100; corr(x1,x2) ≈ 0.95), (S3) rare events with a quasi-separation tendency (n = 1000; β0 = −3.50; 1% of observations with x1 shifted by +12), and (S4) heavy-tailed contamination/outliers (n = 1000; 5% contaminated with t(1) noise). Each condition was replicated R = 200 times with a 70/30 train–test split. Performance was evaluated on test data using discrimination (AUC), proper scoring rules (Brier score, log loss), and calibration via logistic recalibration (intercept and slope). Across S1–S2, AUC and scoring rules were nearly identical between methods, but PROC ENTROPY produced modestly larger calibration slopes. In S3, AUC remained similar (ΔAUC ≈ 0.0014), while calibration diverged substantially (Δ intercept ≈ 0.456; Δ slope ≈ 0.278). In S4, PROC LOGISTIC showed better discrimination (ΔAUC ≈ −0.0055) and slightly better scoring rules, while PROC ENTROPY again produced larger calibration slopes. Sensitivity analyses varying event prevalence, ESUPPORTS width, and sample size confirmed that calibration differences are most pronounced under rare events and depend on entropy support specifications. Routine calibration reporting is essential when comparing binary classifiers and PROC ENTROPY users should assess sensitivity to ESUPPORTS choices when probability estimation is the goal.