Responsible AI packIBM telco holdout

reports/shap_summary.json

What the model uses, not why someone left.

shap.TreeExplainer on the gradient-boosting classifier inside the saved pipeline. Output units: decision_function log-odds of P(Churn=Yes). On 1761 held-out rows the values plus the base value -1.556380 rebuild decision_function with max absolute error 0.000000.

Turn a local contribution off in the demo. The figures below stay on the published values.

Global

one-hot and scaled columns seen by the booster. 46 encoded columns. Mean absolute SHAP of an original field is the sum of the mean absolute SHAP of its encoded columns.

Largest mean absolute SHAP among encoded columns.
Encoded columnMean absolute SHAP
cat__Contract_Month-to-month0.643924
num__tenure0.355529
cat__OnlineSecurity_No0.241598
Original fields. The scale of the ink bars is the largest value in this table, Contract at 0.837003.
Original fieldMean absolute SHAPValue
Contract0.837003
tenure0.355529
OnlineSecurity0.241598
TechSupport0.212835
MonthlyCharges0.204335
InternetService0.201518
TotalCharges0.155191
PaymentMethod0.141521
SeniorCitizen0.044300
gender0.019143

gender and SeniorCitizen are listed after the eight largest original fields so their mean absolute SHAP can be read on the same scale. They are not a second measurement.

Mean absolute SHAP by encoded column. The longest bar is cat__Contract_Month-to-month.
Mean absolute SHAP by encoded column.reports/shap/global_bar.png
Held-out TreeSHAP summary in the encoded column space the booster sees.
Held-out TreeSHAP summary.reports/shap/global_summary.png

Three held-out customers

Signed SHAP values summed within each original field. The plot label is field=raw value. The plotted bar is not a causal effect.

5343-SGUBI

P(Churn=Yes) 0.122990. Historical Churn No. Decision function -1.964413. SHAP base value -1.556380. SHAP sum -0.408033.

  • Contract = One year-0.511087
  • OnlineSecurity = No0.332220
  • tenure = 52-0.288727
  • TechSupport = No0.191892
  • PaymentMethod = Mailed check-0.145604

Positive SHAP raises the model's log-odds of churn.Negative SHAP lowers them. The bar is not a cause.

Local TreeSHAP plot for customer 5343-SGUBI. Signed values are summed back to original fields.
Local plot for 5343-SGUBI. The plotted bar is not a causal effect.reports/shap/local_5343-SGUBI.png
The same three rows, always listed. Plots above follow the selection.
CustomerWhy this rowHistorical ChurnP(Churn=Yes)Largest original field
5343-SGUBIFirst held-out row with numeric TotalChargesNo0.122990Contract = One year, -0.511087
0295-PPHDOHighest held-out probabilityYes0.924460Contract = Month-to-month, 0.870800
5787-KXGIYLowest held-out probabilityNo0.006431Contract = Two year, -0.886617