netcal.regression.IsotonicRegression¶
- class netcal.regression.IsotonicRegression¶
Isotonic regression calibration for probabilistic regression models with multiple independent output dimensions (optionally). Isotonic regression is a piecewise constant, monotonically increasing mapping function used to recalibrate the estimated cumulative density function (CDF) of a probabilistic forecaster [1]. The goal of regression calibration using Isotonic regression is to achieve quantile calibration.
On the one hand, this method accepts as input X either a tuple X = (mean, stddev) using two NumPy arrays of shape N with N number of samples that express the estimated mean and standard deviation of a probabilistic forecaster. On the other hand, a NumPy array of shape (R, N) is also accepted where R denotes the number of probabilistic forecasts. For example, if probabilistic outputs are obtained by Monte-Carlo sampling using N samples and R stochastic forward passes, it is possible to pass all outputs to the calibration function in a single NumPy array.
This method is capable of multiple independent data dimensions where separate calibration models are fitted for each data dimension. This method outputs a tuple consisting of three NumPy arrays:
1st array: T points where the density/cumulative distribution functions are defined, shape: (T, N, D)
2nd array: calibrated probability density function, shape: (T, N, D)
3rd array: calibrated cumulative density function, shape: (T, N, D)
Mathematical background: In [1], regression calibration is defined in terms of quantile calibration. A probabilistic forecaster \(h(X)\) outputs for any input \(X \in \mathbb{R}\) a probability density distribution \(f_Y(y)\) for the target domain \(Y \in \mathcal{Y} = \mathbb{R}\). The according cumulative density function (CDF) is denoted as \(F_Y(y)\), the respective (inverse) quantile function \(F^{-1}_Y(\tau)\) for a certain confidence level \(\tau \in [0, 1]\). The quantile function denotes the quantile boundaries in \(\mathcal{Y}\) given a certain confidence level \(\tau\). Using this notation, quantile calibration [1] is defined as
\[\mathbb{P}(Y \leq F^{-1}_Y(\tau)) = \tau, \quad \forall \tau \in [0, 1] ,\]which is equivalent to
\[\mathbb{P}(F^{-1}_Y(\tau_1) \leq Y \leq F^{-1}_Y(\tau_2)) = \tau_2 - \tau_1, \quad \forall \tau_1, \tau_2 \in [0, 1] .\]In other words, the estimated quantiles should match the observed quantiles. For example, if we inspect the 90% quantiles of a forecaster over multiple samples, we would expect that 90% of all ground-truth estimates fall into these quantiles.
The Isotonic Regression consumes the input cumulative distribution function (CDF) and compares it with the empirical data CDF. With this comparison, it is possible to map the uncalibrated CDF estimates to calibrated ones using a monotonically increasing step function.
References
Methods
__init__()Constructor.
clear()Clear model parameters.
epsilon(dtype)Get the smallest digit that is representable depending on the passed dtype (NumPy or PyTorch).
fit(X, y[, tensorboard])Fit a isotonic regression calibration method to the provided data.
fit_transform(X[, y])Fit to data, then transform it.
get_metadata_routing()Get metadata routing of this object.
get_params([deep])Get parameters for this estimator.
load_model(filename)Load model from saved torch dump.
save_model(filename)Save model instance as with torch's save function as this is safer for torch tensors.
set_fit_request(*[, tensorboard])Request metadata passed to the
fitmethod.set_output(*[, transform])Set output container.
set_params(**params)Set the parameters of this estimator.
set_transform_request(*[, t])Request metadata passed to the
transformmethod.transform(X[, t])Transform uncalibrated distributional estimates (mean and stddev or stochastic samples) to calibrated ones by applying isotonic regression calibration of quantiles.