Responsible AI packIBM telco holdout

Interactive holdout

Move a control. Read the same definitions.

Fairness rates, local TreeSHAP contributions, and PSI are loaded from the committed holdout. A control recomputes those definitions. Reset puts the published figures back.

This is the public IBM telco sample. A gap on a cutoff you chose is still one split, and it is still not a fairness certificate. Subtracting a SHAP value does not rescore the model and does not cause a customer to stay. A PSI change here is a shift applied to the published bin counts.

Fairness

Move the cutoff.

Hard labels on the held-out rows follow the cutoff. Demographic parity difference and equalized odds difference use the same definitions as the Fairlearn report. Gender and SeniorCitizen are model inputs.

0.50

The mark in the write-up is the saved pipeline rule: positive when P(Churn=Yes) > 0.5. An exact tie stays negative.

Demographic parity difference0.223259Absolute gap between group selection rates
Equalized odds difference0.167878Larger of the true-positive-rate gap and the false-positive-rate gap
Selected348/1761Selection rate 0.197615

SeniorCitizen 0 n=1475, positives 345

Selection rate0.161356
True positive rate0.455072
False positive rate0.071681

SeniorCitizen 1 n=286, positives 122

Selection rate0.384615
True positive rate0.622951
False positive rate0.207317

These six-decimal rates match reports/fairness.json. ROC-AUC does not use the cutoff, so it stays in the write-up.

Source: web/data/holdout_scores.json, checked against reports/fairness.json. Fairness write-up

Explanations

Turn a contribution off.

Each control is one published local SHAP value on a held-out customer. The probability is the logistic of the decision function after that value is removed. It is an explanation arithmetic, not a cause of churn.

P(Churn=Yes) from the contributions left on0.122990Published probability 0.122990. Difference 0.000000
Decision function-1.964413Published -1.964413. Base value -1.556380
Contributions removed0.000000Every published contribution is included.
  • Other fields, kept0.013273

Positive SHAP raises the model's log-odds of churn.Negative SHAP lowers them. A pressed control keeps that contribution.

With every contribution on, the logistic of the published decision function matches the published probability at six decimals.

Source: reports/shap_summary.json. Explanations write-up

Drift

Shift one batch.

The reference counts stay the training split. The slider moves mass in the current counts, then PSI is recomputed. At zero, the figure is the committed baseline. Nothing here is a later month of operator data.

0%
Contract PSI0.000040Published PSI 0.000040. Review trigger 0.250000. above_trigger: false.
  • Month-to-month0.549688
  • One year0.207836
  • Two year0.242476

Training-split share, the reference.Current-batch share after the shift. The number is that share.

PSI matches reports/drift_psi.json. The red mark on the meter is the review trigger.

Source: reports/drift_psi.json. categorical. Drift write-up