Learn / Playbooks / Scorecard build

Scorecard build

You are asked to build a new application score from scratch. Frame the problem, build the sample and the model, and turn it into score bands.

8 steps · 10 tools (2 free) · for risk analysts at banks and NBFCs

Tools you will use

This playbook is free to read. The prompt for each step is inside its tool. See the Pro tools.

Schema profile builderFree

Definition libraryFree

Problem framing and target definitionScorecard development

Development sample designScorecard development

Data leakage and look-ahead checksModel validation

Feature engineeringScorecard development

Variable selectionScorecard development

Scorecard buildScorecard development

ML model buildScorecard development

Score scaling and calibrationScorecard development

Check these first

Each tool asks the same questions and gives you a prompt to build whatever is missing.

Applications with a clear good/bad outcome after 12 months?

Have itShort windowDon't have it

Target definition agreed (default, DPD, window)?

YesNot yet

Method

ScorecardML modelNot decided

The steps, in order

Each step names the tool to open.

Frame

Tell the tools your table names

Sample tables, outcome fields and dates.

FreeSchema profile builderFreeDefinition library

Define the target

What is bad, over what window, for whom.

Problem framing and target definition

Design the sample

Time window, exclusions, and holdouts.

Development sample design

Check for leakage

No field may carry information from after the decision.

Data leakage and look-ahead checks

Build

Engineer features

Turn raw fields into candidate variables.

Feature engineering

Select variables

Keep what predicts, drop what duplicates.

Variable selection

Build the model

Scorecard with points per attribute, or an ML model.

Scorecard buildML model build

Scale and calibrate

Turn the output into score bands and probabilities.

Score scaling and calibration

Check before you send

Train, test and out-of-time samples do not share accounts.
No variable has a suspiciously perfect relationship with the outcome.
The target definition is written down and signed off before you build.
Every step is written down so someone else can repeat it.

Warning signs

One variable carries almost all the power: check it for leakage.

The sample window is too short for bad accounts to show up: the model will look better than it is.

What you hand over

A built model, with the sample design, the variable list and score bands, ready to be validated. Continue with the Model validation playbook.

In your profile

Once you are signed in, your profile shows which steps you can run with the tools you own, and which tools you still need.

Other playbooks

Monthly portfolio review Delinquency tracking Underwriting policy change Bureau data and scoring checks Model validation Early warning and leading indicators Collections strategy and agency performance Fraud and first-payment-default review Provisioning and ECL RBI reporting and asset classification Loss budget and stress testing Board and risk committee pack