Library
White paperMichael Clark
The Model Is Trained on Half the Test
Explainability is unlikely to be lenders' binding constraint. CONC 5.2A requires an assessment of credit risk and affordability risk, but most credit models are trained on default, which measures only the first. Its demand for a reasonable assessment in each case calls for a way to identify and divert the cases the model gets wrong, not a nominal reviewer. The recommended steps are three numbers from existing logs, affordability outcome indicators and a record made at decision time, because CONC applies now.
Key findings
- A default model measures one limb of CONC 5.2A's two-limb test, so a customer who repays on time by falling behind on other bills is a good outcome to the model and a bad one under CONC.
- CONC 5.2A does not require a human decision-maker; it requires a reasonable assessment before each agreement, proportionate to the individual circumstances of each case.
- Adding a reviewer is not enough on its own: nominal involvement leaves a decision solely automated in substance, and rules that turn on the use of a system can attach anyway.
In the paper
- One model, two regimes
- The model is trained on half the test
- CONC does not require a human
- Adding a reviewer is not enough on its own
- The record made at decision time
- Deferred payment credit
Michael Clark
Chief Executive Officer, Standard Intelligence
Your next step