Measuring Persistent Professor Effects in Ordinal University Grades: A Separation-Robust Alternative to Mixed-Effects Cumulative Logit

Abstract

We study persistent instructor effects in university grading when grades are discrete, ordinal, and pile up at institutional thresholds (18 = legal passing grade; 30 e lode = honors above 30). In such data, the canonical hierarchical specification—a mixed-effects proportional-odds (cumulative logit) model with professor-level random intercepts—is not estimable in practice: quasi-complete separation at the professor level drives the random intercepts toward ±∞ and prevents the likelihood from attaining a finite maximum. We propose a separation-robust alternative. First, we fit a pooled proportional-odds model for the grade as a function of observed student characteristics (gender, off schedule status, age), exam year, and disciplinary area, explicitly excluding professor identifiers. This model yields, for each exam, the full predicted grade distribution and its expected value on the transcript scale. Second, for each professor we define a grading severity index as the average deviation between realized grades and these predicted benchmarks. This index is always finite, directly interpretable in transcript points, and can be analyzed using standard sampling, shrinkage, and persistence tools. Using over 1.2 million exam records from a large Italian university (2007--2019), we find that grading standards differ sharply across professors: the gap between the severe and generous tails approaches five grade points even after conditioning on observables. These differences are highly persistent over time (year-to-year persistence around 0.8), are not explained by professor gender or by broad disciplinary area, and do not generate systematic subgroup undercoverage in predictive calibration.