Edgar Bahilo Rodríguez

Lead AI Engineer @ B2-Impact

Unveiling Precision: A Novel Machine Learning Framework for Accurate Probability Estimates in Financial Industries

Nowadays, probability prediction models play an essential role in many industries. To cite only a few examples, these applications can range from models in the engineering sector, meteorology (weather forecasting) to modeling frameworks in the financial sector such as the prediction of the number of claims in insurance, credit scoring or propensity to pay forecasts.

Probability calibration is typically assessed graphically via reliability diagrams that plot an estimated version of the conditional event probability (CEP) against the forecast value, with deviations from the diagonal suggesting lack of calibration. Classical approaches to estimating CEP rely on binning and counting and have been hampered by ad hoc implementation decisions, instability under unavoidable choices regarding binning, and inefficiency.

This session shows a practical implementation of alternative calibration metrics applied to the debt management industry, where having a good calibration of probabilities can be critical and can significantly lead to increased cash flows (e.g. in the hundreds of thousands or millions of euros). An advanced machine learning framework is designed to address the proper scoring rules for the Hyperparameter Optimization (HPO) and the use of the ECCE-MAD and ECCE-R scores to have the best model selection in terms of calibration.