crps#

pymc_marketing.metrics.crps(y_true, y_pred, sample_weight=None)[source]#

Compute the (possibly weighted) average of the continuous ranked probability score.

Parameters:

y_truearray_like: The ground truth values.
y_predarray_like: The predicted values. It is expected that y_pred has one extra sample dimension on the left.
sample_weightarray_like, optional: The sample weights.

Returns:

float: The CRPS value as a (possibly weighted) average of the per-observation CRPS values.

References

This implementation is a minimal adaptation from the one in the Pyro project: https://docs.pyro.ai/en/dev/_modules/pyro/ops/stats.html#crps_empirical
For an introduction to CRPS, see https://towardsdatascience.com/crps-a-scoring-function-for-bayesian-machine-learning-models-dd55a7a337a8

Examples

import numpy as np

from pymc_marketing.metrics import crps

# y_true shape is (3,)
y_true = np.array([1, 1, 1])
# y_pred shape is (10, 3). The extra dimension on the left is the number of samples.
y_pred = np.repeat(np.array([[0, 1, 0]]), 10, axis=0)

# The result is a scalar.
crps(y_true, y_pred)

>> 0.666