netcal.binning.HistogramBinning

class netcal.binning.HistogramBinning(bins: int | Iterable[int] = 10, equal_intervals: bool = True, detection: bool = False, independent_probabilities: bool = False)

Simple Histogram Binning calibration method. This method has been proposed by [1]. Each prediction is sorted into a bin and assigned its calibrated confidence estimate. This method normally works for binary classification. For multiclass classification, this method is applied into a 1-vs-all manner [2].

The bin boundaries are either chosen to be equal length intervals or to equalize the number of samples in each bin.

On object detection, use a multidimensional binning to include additional information of the box regression branch [3].

Parameters:

• bins (int or iterable, default: 10) – Number of bins used by the Histogram Binning. On detection mode: if int, use same amount of bins for each dimension (nx1 = nx2 = … = bins). If iterable, use different amount of bins for each dimension (nx1, nx2, … = bins).

• equal_intervals (bool, optional, default: True) – If True, the bins have the same width. If False, the bins are splitted to equalize the number of samples in each bin.

• detection (bool, default: False) – If False, the input array ‘X’ is treated as multi-class confidence input (softmax) with shape (n_samples, [n_classes]). If True, the input array ‘X’ is treated as a box predictions with several box features (at least box confidence must be present) with shape (n_samples, [n_box_features]).

• independent_probabilities (bool, optional, default: False) – Boolean for multi class probabilities. If set to True, the probability estimates for each class are treated as independent of each other (sigmoid).

References

[1]

Zadrozny, Bianca and Elkan, Charles: “Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers.” In ICML, pp. 609–616, 2001. Get source online

[2]

Zadrozny, Bianca and Elkan, Charles: “Transforming classifier scores into accurate multiclass probability estimates.” In KDD, pp. 694–699, 2002. Get source online

[3]

Fabian Küppers, Jan Kronenberger, Amirhossein Shantia and Anselm Haselhoff: “Multivariate Confidence Calibration for Object Detection.” The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020.

Methods

__init__([bins, equal_intervals, detection, ...])	Create an instance of HistogramBinning.
clear()	Clear model parameters.
epsilon(dtype)	Get the smallest digit that is representable depending on the passed dtype (NumPy or PyTorch).
fit(X, y[, random_state, tensorboard, log_dir])	Function call to build the calibration model.
fit_transform(X[, y])	Fit to data, then transform it.
get_degrees_of_freedom()	Needed for BIC.
get_metadata_routing()	Get metadata routing of this object.
get_params([deep])	Get parameters for this estimator.
load_model(filename)	Load model from saved torch dump.
save_model(filename)	Save model instance as with torch's save function as this is safer for torch tensors.
set_fit_request(*[, log_dir, random_state, ...])	Request metadata passed to the fit method.
set_output(*[, transform])	Set output container.
set_params(**params)	Set the parameters of this estimator.
set_transform_request(*[, num_samples, ...])	Request metadata passed to the transform method.
transform(X[, num_samples, random_state])	After model calibration, this function is used to get calibrated outputs of uncalibrated confidence estimates.