netcal.metrics.QuantileLoss

class netcal.metrics.QuantileLoss

Pinball aka quantile loss within regression calibration to test for quantile calibration of a probabilistic regression model. The Pinball loss is an asymmetric loss that measures the quality of the predicted quantiles. Given a probabilistic regression model that outputs a probability density function (PDF) \(f_Y(y)\) targeting the ground-truth \(y\), we further denote the cumulative as \(F_Y(y)\) and the (inverse) percent point function (PPF) as \(F_Y^{-1}(\tau)\) for a certain quantile level \(\tau \in [0, 1]\).

The Pinball loss is given by

\[\begin{split}L_{\text{Pin}}(\tau) = \begin{cases} \big( y-F_Y^{-1}(\tau) \big)\tau \quad &\text{if } y \geq F_Y^{-1}(\tau)\\ \big( F_Y^{-1}(\tau)-y \big)(1-\tau) \quad &\text{if } y < F_Y^{-1}(\tau) \end{cases} .\end{split}\]

Methods

__init__()
measure(X, y, q, *[, kind, reduction])	Measure quantile loss for given input data either as tuple consisting of mean and stddev estimates or as NumPy array consisting of a sample distribution.