236 lines
6.9 KiB
Markdown
236 lines
6.9 KiB
Markdown
**Source of truth** : https://git.gregoiremureau.com/openclassroom-ai/oc-p10-content-recommendation
|
|
|
|
```bash
|
|
git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p10-content-recommendation.git
|
|
git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p10-content-recommendation.git
|
|
```
|
|
|
|
Dataset: see `fetch-data.sh` (excluded from git if large).
|
|
|
|
# Content Recommendation System - My Content
|
|
|
|
📘 This project is also available in [French 🇫🇷](./README.fr.md)
|
|
|
|
Project developing an article recommendation system based on a serverless Azure Functions architecture with a Streamlit user interface.
|
|
|
|
|
|
## Project Deliverables
|
|
1. **Working application**: Streamlit + Azure Functions deployed
|
|
2. **Versioned source code**: Complete GitHub repository
|
|
3. **Technical presentation**: PDF deck (docs/presentation.pdf)
|
|
|
|
## Overview
|
|
|
|
This project implements a content recommendation system using collaborative filtering and content-based algorithms. The serverless architecture enables automatic scaling and optimized cost.
|
|
|
|
### Main Features
|
|
|
|
- **Recommendation API**: Azure Function returning the top 5 recommended articles for a user
|
|
- **User interface**: Streamlit application to test and demonstrate the system
|
|
- **Implemented algorithms**: Content-Based Filtering with precomputed embeddings
|
|
- **Serverless architecture**: Deployment on Azure Functions with Consumption plan
|
|
|
|
|
|
## Analysis and Development
|
|
|
|
### Jupyter Notebook
|
|
- **Data exploration**: Analysis of the Kaggle Globo.com dataset
|
|
- **Algorithm implementation**: Content-Based and Collaborative Filtering
|
|
- **Approach comparison**: Evaluation and technical choice
|
|
- **PCA optimization**: Dimensionality reduction 250D → 50D
|
|
|
|
Location: `notebook/p10_notebook.ipynb`
|
|
|
|
## Architecture
|
|
|
|
### Current MVP
|
|
```
|
|
[Streamlit Interface] → [Azure Functions API] → [Recommendation Algorithm]
|
|
↓
|
|
[Embeddings data + user history]
|
|
```
|
|
|
|
### Components
|
|
- **Frontend**: Streamlit (test interface)
|
|
- **Backend**: Azure Functions (Python 3.12, Linux)
|
|
- **Algorithm**: Content-Based Filtering with cosine similarity
|
|
- **Data**: Article embeddings (50D via PCA vs 250D original embeddings) + click history
|
|
|
|
## Project Structure Overview
|
|
|
|
```
|
|
Project_10/
|
|
├── azure-app/ # Azure Function
|
|
│ ├── function_app.py # Azure Function entry point
|
|
│ ├── requirements.txt # Python dependencies
|
|
│ ├── host.json # Azure Functions configuration
|
|
│ ├── local.settings.json # Local configuration (not versioned)
|
|
│ ├── data/
|
|
│ │ ├── embeddings_azure_50d.pkl # Article embeddings (PCA 50D)
|
|
│ │ └── clicks_sample.csv # User click history
|
|
│ └── utils/
|
|
│ ├── data_loader.py # Data loading and caching
|
|
│ └── content_based.py # Recommendation algorithm
|
|
├── notebook/
|
|
│ └── p10_notebook.ipynb # Data analysis + model development
|
|
├── docs/
|
|
│ ├── presentation.pdf # Presentation deck (15-25 slides)
|
|
│ └── illustrations/ # Architecture diagrams and screenshots
|
|
├── streamlit_app.py # User interface
|
|
├── streamlit_requirements.txt # Streamlit dependencies
|
|
└── README.md
|
|
```
|
|
|
|
## Installation and Deployment
|
|
|
|
### Prerequisites
|
|
- Python 3.12+
|
|
- Azure CLI
|
|
- Azure Functions Core Tools
|
|
- Azure account
|
|
|
|
### Azure Functions Deployment
|
|
|
|
1. **Sign in to Azure**
|
|
```bash
|
|
az login
|
|
```
|
|
|
|
2. **Infrastructure creation**
|
|
```bash
|
|
# Resource group
|
|
az group create --name rg-content-recommender --location francecentral
|
|
|
|
# Storage account
|
|
az storage account create --name [your-unique-storage-name] \
|
|
--resource-group rg-content-recommender \
|
|
--location francecentral \
|
|
--sku Standard_LRS
|
|
|
|
# Linux Function App with Python 3.12
|
|
az functionapp create \
|
|
--resource-group rg-content-recommender \
|
|
--consumption-plan-location francecentral \
|
|
--runtime python \
|
|
--runtime-version 3.12 \
|
|
--functions-version 4 \
|
|
--name contentrecommender-linux \
|
|
--storage-account stcontentrecommend2025 \
|
|
--os-type Linux
|
|
```
|
|
|
|
3. **Code deployment**
|
|
```bash
|
|
# From the azure-app folder
|
|
func azure functionapp publish contentrecommender-linux
|
|
```
|
|
|
|
### Streamlit Interface
|
|
|
|
1. **Install dependencies**
|
|
```bash
|
|
pip install -r streamlit_requirements.txt
|
|
```
|
|
|
|
2. **Launch the interface**
|
|
```bash
|
|
streamlit run streamlit_app.py
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Azure Functions API
|
|
|
|
**Endpoint**: `https://contentrecommender-linux.azurewebsites.net/api/recommend`
|
|
|
|
**Parameters**:
|
|
- `user_id` (int): User identifier (0-706)
|
|
|
|
**Example call**:
|
|
```bash
|
|
curl "https://contentrecommender-linux.azurewebsites.net/api/recommend?user_id=0"
|
|
```
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"user_id": 0,
|
|
"recommendations": [157519, 157944, 159495, 156690, 162857]
|
|
}
|
|
```
|
|
|
|
### Streamlit Interface
|
|
|
|
1. Configure the API URL (local or Azure)
|
|
2. Select a user ID
|
|
3. Get the recommendations
|
|
4. Test the advanced features
|
|
|
|
## Recommendation Algorithm
|
|
|
|
### Content-Based Filtering
|
|
|
|
1. **Data loading**: Precomputed embeddings and user history
|
|
2. **User profile computation**: Average of the embeddings of the viewed articles
|
|
3. **Cosine similarity**: Computed against all articles in the catalog
|
|
4. **Filtering**: Exclusion of already-seen articles
|
|
5. **Selection**: Top 5 most similar articles
|
|
|
|
### Optimizations
|
|
|
|
- **Dimensionality reduction**: PCA 250D → 50D to optimize Azure performance
|
|
- **In-memory cache**: Embeddings and user data loaded only once
|
|
- **Serverless architecture**: Automatic scaling based on demand
|
|
|
|
## Tests and Validation
|
|
|
|
### Local tests
|
|
```bash
|
|
# Start Azure Functions locally
|
|
func start
|
|
|
|
# Test the algorithm
|
|
python test_local.py
|
|
```
|
|
|
|
### Performance metrics
|
|
- **Response time**: ~200ms per recommendation
|
|
- **Supported users**: 707 in the current dataset
|
|
- **Available articles**: 364,047 with embeddings
|
|
|
|
## Known Issues and Solutions
|
|
|
|
### Azure Deployment
|
|
- **Python on Windows**: Not supported, must use Linux
|
|
- **Authentication**: Set `http_auth_level=func.AuthLevel.ANONYMOUS`
|
|
- **"Service unavailable" errors**: Temporary Azure issues, retry later
|
|
|
|
### Current Limitations
|
|
- **Cold start**: First request is slow after a period of inactivity
|
|
- **New users**: No handling of the cold-start problem
|
|
- **Data updates**: Manual process
|
|
|
|
## Target Architecture (Future)
|
|
|
|
### Planned Improvements
|
|
- **Cold start handling**: Popular recommendations for new users
|
|
- **Data pipeline**: Automatic embeddings update
|
|
- **Database**: Persistent storage of user profiles
|
|
- **Hybrid API**: Combination of Content-Based + Collaborative Filtering
|
|
- **Distributed cache**: Precomputation of popular recommendations
|
|
|
|
## Technologies Used
|
|
|
|
- **Backend**: Azure Functions, Python 3.12
|
|
- **Frontend**: Streamlit
|
|
- **ML/Data**: scikit-learn, pandas, numpy
|
|
- **Infrastructure**: Azure CLI, Azure Storage
|
|
- **Deployment**: Azure Functions Core Tools
|
|
|
|
## Author
|
|
|
|
Project carried out as part of an OpenClassrooms training program - AI Engineer.
|
|
|
|
## License
|
|
|
|
This project is for educational purposes only.
|