oc-p04-credit-scoring/README.md

Source of truth : https://git.gregoiremureau.com/openclassroom-ai/oc-p04-credit-scoring

git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p04-credit-scoring.git
git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p04-credit-scoring.git

Dataset: see fetch-data.sh (excluded from git if large).

Credit Scoring Project - Prêt à dépenser

📘 This project is also available in French 🇫🇷

📋 Project Context

This project was carried out for "Prêt à dépenser", a financial company specializing in consumer credit lending. The company wants to set up a credit scoring tool to assess the probability that a customer will repay their loan, thereby automating the credit approval decision process.

🎯 Objectives

The main objectives of the project are:

1. Develop a scoring model: Create a robust predictive model capable of assessing the risk of payment default
2. Optimize business cost: Minimize financial losses by taking into account the real cost of false positives and false negatives
3. Ensure interpretability: Provide clear explanations of the model's decisions for customer relationship managers
4. Handle class imbalance: Effectively deal with the imbalanced data characteristic of the credit domain

📊 Data

The data comes from the Kaggle "Home Credit Default Risk" competition and includes:

• Customers' credit history
• Previous loan applications
• Credit card information
• Installment payment data
• POS (Point of Sale) and cash balances

⚠️ Important note: The raw data files are not included in this repository due to their large size. They can be downloaded here:

• URL: https://www.kaggle.com/competitions/home-credit-default-risk/data

The required files include:

• application_train.csv / application_test.csv
• bureau.csv
• previous_application.csv
• credit_card_balance.csv
• installments_payments.csv
• POS_CASH_balance.csv

📁 Project Structure

├── P4_01_notebook_analyse_exploratoire_feature_engineering.ipynb
├── P4_02_notebook_modelisation.ipynb
├── P4_03_presentation.pptx
├── explication_client_0.html
└── explication_client_2.html

1. P4_01_notebook_analyse_exploratoire_feature_engineering.ipynb

Exploratory Data Analysis (EDA) and Feature Engineering Notebook

This notebook contains the entire data preparation phase:

Data exploration:

• Loading and analysis of the various CSV files
• Study of the target variable distribution (class imbalance)
• Analysis of missing values and outliers
• Study of correlations between variables and with the target

Data enrichment:

• Intelligent merging of the various data sources (bureau, previous_application, etc.)
• Aggregation of information from multiple tables
• Creation of relevant business variables (ratios, averages, statistics)

Feature Engineering:

• Creation of new features based on business ratios
• Encoding of categorical variables
• Imputation of missing values
• Normalization and standardization of variables
• Specific preparation for models tolerant of missing values (LightGBM)

2. P4_02_notebook_modelisation.ipynb

Modeling, Optimization and Interpretation Notebook

This notebook presents the entire modeling process:

Baseline modeling:

• Logistic regression (baseline)
• Random Forest
• Evaluation with standard metrics (AUC-ROC, precision, recall)

Advanced modeling:

• Training of gradient boosting models: XGBoost, LightGBM, CatBoost
• Handling class imbalance (class_weight, scale_pos_weight)
• Hyperparameter optimization with GridSearchCV

Business optimization:

• Definition of a business score incorporating the cost of false positives and false negatives
• Determination of the optimal decision threshold to minimize financial losses
• Analysis of the cost-threshold curve

Interpretability:

• Global interpretation: Feature importance with SHAP (TreeExplainer)
• Local interpretation: Explanation of individual predictions with LIME
• Analysis of the impact of each variable on decisions

Model simplification:

• Selection of the most important features
• Reduction of the number of features while maintaining performance (AUC)

3. P4_03_presentation.pptx

Defense Presentation

This presentation summarizes:

• The business context and challenges of credit scoring
• The data preparation and enrichment methodology
• The comparison of the different models tested
• The choice of the final model and its business optimization
• The global and local interpretability approach (SHAP and LIME)
• Recommendations and improvement perspectives

4. explication_client_0.html and explication_client_2.html

Local Prediction Interpretation Reports

These interactive HTML reports illustrate concrete examples of explanations for individual customers:

• The model's predictions for each customer (default probability)
• SHAP and/or LIME explanations detailing why the model made a given decision
• The contribution of each variable to the final prediction for this specific customer
• Visualizations allowing customer relationship managers to understand and justify decisions
• Examples of acceptance and rejection cases with their justifications

These files serve as a demonstration of the local interpretability system that could be deployed in production to explain each credit decision.

🛠️ Technologies Used

• Python: Development and analysis
  • pandas, numpy: data manipulation
  • scikit-learn: baseline models, preprocessing, metrics
  • XGBoost, LightGBM, CatBoost: gradient boosting models
  • SHAP: global model interpretation
  • LIME: local prediction interpretation
  • matplotlib, seaborn, plotly: visualization
• Jupyter Notebook: development and documentation

📈 Methodology

1. Exploration and understanding: In-depth analysis of multi-source data
2. Feature Engineering: Creation of relevant business variables through aggregation and transformation
3. Modeling: Testing several algorithms with class imbalance handling
4. Business optimization: Definition and optimization of a business score incorporating real costs
5. Interpretability: Implementation of global (SHAP) and local (LIME) explanation tools
6. Validation: Evaluation on test data and verification of generalization

🎯 Expected Results

• ✅ Enriched and preprocessed dataset with new business features
• ✅ Scoring model optimized according to a business criterion (cost minimization)
• ✅ Optimal decision threshold determined
• ✅ Global and local interpretability reports
• ✅ Complete documentation for end users

💡 Key Points of the Project

Handling class imbalance

The dataset shows a strong imbalance between good and bad payers. Several techniques were implemented:

• Use of class_weight and scale_pos_weight parameters
• Optimization based on appropriate metrics (AUC-ROC, recall, business score)
• Post-modeling decision threshold adjustment

Custom business score

Rather than relying solely on classic metrics, a business score was defined taking into account:

• The cost of a false positive (wrongly rejecting a good customer → lost opportunity)
• The cost of a false negative (accepting a bad payer → significant financial loss)
• Optimization of the decision threshold to minimize total cost

Interpretability

Two levels of interpretation were implemented:

• Global (SHAP): Which variables are most important for the model in general?
• Local (LIME): Why did the model make this decision for this specific customer?

This dual approach allows customer relationship managers to understand and explain decisions to credit applicants.

🎓 Skills Developed

• Preparation and enrichment of complex multi-source data
• Business-oriented feature engineering in the financial domain
• Modeling with class imbalance handling
• Optimization based on business criteria (cost/benefit)
• Interpretability of machine learning models (SHAP, LIME)
• Communication of technical results to business audiences

👤 Author

Grégoire Mureau

---

This project illustrates the complete data science approach applied to credit scoring: from data preparation to business optimization, including model interpretability.