GOVERNANCE_CHECKLIST.md

Govern, map, measure, manage.

Checklist for the churn model in this repo, mapped to the NIST AI Risk Management Framework functions Govern, Map, Measure, and Manage. Independent portfolio project built on public data.

If a later project used personal data of people in Kenya, the statute to read is the Kenya Data Protection Act 2019. This page names that Act. It does not interpret it, and it does not claim compliance.

Metric floors

minimum = floor_to_decimals(recomputed_value - margin, 2). The margin is there so six-decimal noise does not fail CI, and a broken artifact or a real drop does. These floors are not confidence intervals. Fairness gaps are not floors.

gates.yaml writes the floors as 0.82, 0.63, 2.60. gate_check.json stores them at six decimals. passed: true. failures: none.
Metric	Recomputed	Margin	Floor in the check	Result
ROC-AUC	0.846001	0.02	0.820000	passed
PR-AUC	0.656070	0.02	0.630000	passed
Top-decile lift	2.806733	0.20	2.600000	passed

Drift is watched, not gated. Review trigger 0.250000. n_above_trigger 0. A feature at or above the trigger is a reason to read the batch. It does not by itself fail gates.yaml.

gates.yaml · MONITORING.md · INCIDENT_RUNBOOK.md

Checklist

Govern

Done
The model card names the artifact, the commit, and the person whose portfolio this is.
Done
The README states the data and scope and that the pack uses the public IBM sample only.
Done
Kenya Data Protection Act 2019 is named, with no section numbers and no compliance claim.
Open
A real deployment still needs an accountable owner inside the organisation that would act on a score. This repo cannot appoint one.

Map

Done
Intended use and out-of-scope use are written in MODEL_CARD.md and MODEL_CARD_MULTIHEAD.md.
Done
The churn training table is identified by the upstream metrics file and by the CSV SHA-256 in upstream.lock.json.
Done
gender and SeniorCitizen are documented as model inputs, and as the only group fields this pack measures.
Done
The multi-head card says when the source repo does not document data or quality.

Measure

Done
Held-out ROC-AUC, PR-AUC, and top-decile lift are recomputed in reports/metrics_recomputed.json from the saved models.
Done
TreeSHAP global and local plots are in reports/shap/, with numbers in reports/shap_summary.json.
Done
Fairlearn selection rate, true positive rate, false positive rate, and ROC-AUC by gender and SeniorCitizen are in reports/fairness.json, with demographic parity and equalized odds differences.
Done
PSI on the pinned split is in reports/drift_psi.json. That file says the split is not a time comparison.

Manage

Done
gates.yaml holds floors under the recomputed churn metrics. scripts/check_gates.py recomputes them and fails if one is below its floor. GitHub Actions runs that script.
Done
MONITORING.md says what a later batch would compare, and what this repo does not watch.
Done
INCIDENT_RUNBOOK.md covers detection, triage, rollback, communication, and a postmortem when a gated metric is below its floor.
Open
Fairness gaps are reported and are not a CI floor. A disparity budget needs an owner this repo does not have.
Open
The multi-head emergency-recall smoke floor stays in that other repo. This pack does not gate it, because the weights and gold labels are not there.

Source: GOVERNANCE_CHECKLIST.md