netcal.metrics.regression.QuantileLoss

class netcal.metrics.regression.QuantileLoss

Pinball aka quantile loss within regression calibration to test for quantile calibration of a probabilistic regression model. The Pinball loss is an asymmetric loss that measures the quality of the predicted quantiles. Given a probabilistic regression model that outputs a probability density function (PDF) $f_Y(y)$ targeting the ground-truth $y$, we further denote the cumulative as $F_Y(y)$ and the (inverse) percent point function (PPF) as $F_Y^{-1}(\tau )$ for a certain quantile level $\tau \in [0,1]$.

The Pinball loss is given by

$$\begin{array}{r}{L}_{\text{Pin}}(\tau )=\{\begin{array}{ll}(y-F_Y^{-1}(\tau ))\tau & \text{if }y\ge F_Y^{-1}(\tau )\\ (F_Y^{-1}(\tau )-y)(1-\tau )& \text{if }y<F_Y^{-1}(\tau )\end{array}.\end{array}$$

Methods

__init__()
measure(X, y, q, *[, kind, reduction])	Measure quantile loss for given input data either as tuple consisting of mean and stddev estimates or as NumPy array consisting of a sample distribution.