CreditCompass Calibrated XGBoost · probability of default Connecting…

Applied machine learning · tabular classification · probability calibration

Estimating loan default probability from application data

An XGBoost pipeline, tuned with randomized search and wrapped in sigmoid calibration, estimates the probability that a loan in a public credit-risk dataset defaults. The interface shows the estimate, how it compares with the stored decision threshold, how it responds to each input, and how the model performs on held-out data.

Research demonstration — not a lending tool

Outputs are statistical estimates from a model trained on one public dataset with undocumented provenance and currency. They are not credit scores, lending decisions or financial advice, and the model has not been audited for fairness.

Assess an application

The 11 inputs below are exactly the fields of LoanApplication in main.py. Values are sent unchanged to POST /predict, where the saved scikit-learn pipeline imputes, one-hot encodes and scores them.

Applicant
Loan
Credit bureau

Sensitivity to one input

Hold the current application fixed, vary one feature across the range seen in training, and query the deployed model at each value (an individual conditional expectation curve). Every point is a real POST /predict call.

No sensitivity curve yetUses the values currently in the form above.

Evaluation on held-out data

re-evaluated figures score the shipped credit_risk_model.pkl on the notebook's own 20% stratified test split (reproduced with the same cleaning and random_state = 42) via evaluation/evaluate_artifact.py. notebook figures are copied from saved notebook outputs.

Model card

Pipeline

Credit_Risk.ipynb
  1. Cleaning Remove duplicates; keep age 18–100, employment length ≤ age and ≤ 60, loan amount > 0 (32,581 → 31,522 rows).
  2. Split Stratified 80/20 train/test, random_state = 42.
  3. Preprocess Numeric: median imputation. Categorical: constant imputation + one-hot encoding (unknown categories ignored).
  4. Model XGBoost with scale_pos_weight ≈ 3.63; 150-candidate randomized search, 5-fold CV, scoring = average precision.
  5. Calibration CalibratedClassifierCV(method="sigmoid", cv=5) around the tuned pipeline.
  6. Decision main.py returns predict_proba and labels “High Risk” when it is ≥ the stored threshold (best_threshold.pkl).

Intended use and limitations

  1. Intended use: demonstrating an end-to-end, calibrated tabular-ML workflow. Not for decisions about real people.
  2. Threshold selection. The notebook's threshold cell computes 2·(P − R)/(P + R) instead of F1 = 2·P·R/(P + R), searches on the test split, and uses the uncalibrated model's probabilities. The saved threshold (0.695) does not match the value printed in the notebook (1.00), so it came from a different execution. It happens to be close to the F1-optimal threshold for the shipped model (see the threshold explorer).
  3. Imperfect calibration. Despite sigmoid calibration, mid-range estimates (≈ 0.4–0.75) overstate the observed default rate on the test split.
  4. Lender-assigned inputs. Loan grade and interest rate are set by the lender and already encode its risk assessment, so the model partly reproduces existing grading.
  5. Sensitive attributes. Age is a model input; no fairness or disparate-impact analysis was performed.
  6. Data. One public CSV (32,581 loans) with undocumented origin, time period and currency. Rows with missing employment length were removed by the cleaning step.

Reproduce

cd ML/CREDIT_RISK_ML
python3.12 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --port 8000
# regenerate static/data/{evaluation,schema}.json
python evaluation/evaluate_artifact.py