per_observation_crps#

pymc_marketing.metrics.per_observation_crps(y_true, y_pred)[source]#

Compute the continuous ranked probability score (CRPS) for each observation.

The CRPS — Continuous Ranked Probability Score — is a score function that compares a single ground truth value to a Cumulative Distribution Function.

Parameters:

y_truearray_like: The ground truth values.
y_predarray_like: The predicted values. It is expected that y_pred has one extra sample dimension on the left.

Returns:

array_like: The CRPS for each observation.

References

This implementation is a minimal adaptation from the one in the Pyro project: https://docs.pyro.ai/en/dev/_modules/pyro/ops/stats.html#crps_empirical
For an introduction to CRPS, see https://towardsdatascience.com/crps-a-scoring-function-for-bayesian-machine-learning-models-dd55a7a337a8

Examples

import numpy as np

from pymc_marketing.metrics import per_observation_crps

# y_true shape is (3,)
y_true = np.array([1, 1, 1])
# y_pred shape is (10, 3). The extra dimension on the left is the number of samples.
y_pred = np.repeat(np.array([[0, 1, 0]]), 10, axis=0)

# The result has shape (3,), one value per observation.
per_observation_crps(y_true, y_pred)

>> array([1., 0., 1.])