counterfactual#

Evaluating a fitted MMM under counterfactual spend.

incrementality asks what would have happened at a different level of spend. Answering it means running the fitted graph again on perturbed inputs, many times over, and this module is the part that runs it. It knows about windows, scenarios and compiled graphs; it knows nothing about estimands, link functions or ROAS, which is where the dependency between the two modules stops.

The unit of work is a scenario: one period’s spend, perturbed for one channel or for all of them at once, evaluated over a stretch of dates wide enough to carry the perturbation’s whole effect. Three pieces make that up:

  • EvaluationWindows decides which dates each period is evaluated over and cuts every date-indexed array to match. Once a counterfactual is evaluated on a window rather than on the full axis, every date-indexed input of the graph has to be cut in lockstep, or the graph is handed a window-length spend array and a full-length mediator array.

  • CounterfactualScenarios holds the perturbed spend together with the bookkeeping that says which row answers which question.

  • CounterfactualEvaluator compiles the model graph once, conditioned on the posterior, and evaluates the scenarios in bounded batches.

The intervention itself is expressed through pymc.do(). The evaluator grafts a symbolic input onto the intervened node, and only then conditions the graph on the posterior and vectorizes it over scenarios:

do(model, {target: intervention})            # the intervention
  -> extract_response_distribution(...)      # condition on posterior draws
  -> vectorize_graph(...)                    # batch over scenarios

do states what is intervened on; the batching states how many values it takes. The split matters because do requires the intervention to have the target’s own dimensions, so the scenario axis can only be introduced afterwards.

Which node is intervened on decides which question the result answers. Intervening on channel_data – the default – is the spend counterfactual: the perturbation propagates through adstock, saturation and any mediated effect, which is the total effect of moving spend. Scaling channel_contribution by a zero mask instead answers effect removal: what would have happened without this channel’s contribution, whatever its spend was. The two coincide only when the media transform maps zero spend to zero contribution and no time-varying multiplier scales it, and only for a factor of zero: for fractional factors they must differ, because saturation is nonlinear in spend while the mask is linear in contribution.

Module Attributes

Estimand

Which counterfactual an increment answers for.

InterventionMode

How an intervention grafts onto the target node.

Functions

find_named_node(roots, name, *[, exclude])

Find the one node of a graph that carries a given name.

Classes

CounterfactualEvaluator(*, pymc_model, ...)

Compiled batched evaluator for the nodes a counterfactual intervention reaches.

CounterfactualScenarios(spend, period_index, ...)

Perturbed spend arrays for every scenario that has to be evaluated.

DateIndexedInput(name, values, dtype, dims, ...)

A date-indexed pm.Data replaced by a batched evaluator input.

EvaluationWindows(windows, max_window, dates)

Per-period evaluation windows, and the cutting of arrays to fit them.

PeriodWindow(start, end, in_window, ...)

The stretch of fitted dates one period is evaluated over.