Grégoire Mureau
17b906ad58
Public READMEs now expose a single git.gregoiremureau.com clone URL. The Tailscale SSH form only works on the homelab network and leaked the NAS address. The language switcher sits right under the title so an English speaker landing on the French page can leave without reading French.
235 lines
6.9 KiB
Markdown
235 lines
6.9 KiB
Markdown
# Content Recommendation System - My Content
|
|
|
|
📘 Ce projet est également disponible en [français 🇫🇷](./README.fr.md)
|
|
|
|
**Source of truth** : https://git.gregoiremureau.com/openclassroom-ai/oc-p10-content-recommendation
|
|
|
|
```bash
|
|
git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p10-content-recommendation.git
|
|
```
|
|
|
|
Dataset: see `fetch-data.sh` (excluded from git if large).
|
|
|
|
Project developing an article recommendation system based on a serverless Azure Functions architecture with a Streamlit user interface.
|
|
|
|
|
|
## Project Deliverables
|
|
1. **Working application**: Streamlit + Azure Functions deployed
|
|
2. **Versioned source code**: Complete GitHub repository
|
|
3. **Technical presentation**: PDF deck (docs/presentation.pdf)
|
|
|
|
## Overview
|
|
|
|
This project implements a content recommendation system using collaborative filtering and content-based algorithms. The serverless architecture enables automatic scaling and optimized cost.
|
|
|
|
### Main Features
|
|
|
|
- **Recommendation API**: Azure Function returning the top 5 recommended articles for a user
|
|
- **User interface**: Streamlit application to test and demonstrate the system
|
|
- **Implemented algorithms**: Content-Based Filtering with precomputed embeddings
|
|
- **Serverless architecture**: Deployment on Azure Functions with Consumption plan
|
|
|
|
|
|
## Analysis and Development
|
|
|
|
### Jupyter Notebook
|
|
- **Data exploration**: Analysis of the Kaggle Globo.com dataset
|
|
- **Algorithm implementation**: Content-Based and Collaborative Filtering
|
|
- **Approach comparison**: Evaluation and technical choice
|
|
- **PCA optimization**: Dimensionality reduction 250D → 50D
|
|
|
|
Location: `notebook/p10_notebook.ipynb`
|
|
|
|
## Architecture
|
|
|
|
### Current MVP
|
|
```
|
|
[Streamlit Interface] → [Azure Functions API] → [Recommendation Algorithm]
|
|
↓
|
|
[Embeddings data + user history]
|
|
```
|
|
|
|
### Components
|
|
- **Frontend**: Streamlit (test interface)
|
|
- **Backend**: Azure Functions (Python 3.12, Linux)
|
|
- **Algorithm**: Content-Based Filtering with cosine similarity
|
|
- **Data**: Article embeddings (50D via PCA vs 250D original embeddings) + click history
|
|
|
|
## Project Structure Overview
|
|
|
|
```
|
|
Project_10/
|
|
├── azure-app/ # Azure Function
|
|
│ ├── function_app.py # Azure Function entry point
|
|
│ ├── requirements.txt # Python dependencies
|
|
│ ├── host.json # Azure Functions configuration
|
|
│ ├── local.settings.json # Local configuration (not versioned)
|
|
│ ├── data/
|
|
│ │ ├── embeddings_azure_50d.pkl # Article embeddings (PCA 50D)
|
|
│ │ └── clicks_sample.csv # User click history
|
|
│ └── utils/
|
|
│ ├── data_loader.py # Data loading and caching
|
|
│ └── content_based.py # Recommendation algorithm
|
|
├── notebook/
|
|
│ └── p10_notebook.ipynb # Data analysis + model development
|
|
├── docs/
|
|
│ ├── presentation.pdf # Presentation deck (15-25 slides)
|
|
│ └── illustrations/ # Architecture diagrams and screenshots
|
|
├── streamlit_app.py # User interface
|
|
├── streamlit_requirements.txt # Streamlit dependencies
|
|
└── README.md
|
|
```
|
|
|
|
## Installation and Deployment
|
|
|
|
### Prerequisites
|
|
- Python 3.12+
|
|
- Azure CLI
|
|
- Azure Functions Core Tools
|
|
- Azure account
|
|
|
|
### Azure Functions Deployment
|
|
|
|
1. **Sign in to Azure**
|
|
```bash
|
|
az login
|
|
```
|
|
|
|
2. **Infrastructure creation**
|
|
```bash
|
|
# Resource group
|
|
az group create --name rg-content-recommender --location francecentral
|
|
|
|
# Storage account
|
|
az storage account create --name [your-unique-storage-name] \
|
|
--resource-group rg-content-recommender \
|
|
--location francecentral \
|
|
--sku Standard_LRS
|
|
|
|
# Linux Function App with Python 3.12
|
|
az functionapp create \
|
|
--resource-group rg-content-recommender \
|
|
--consumption-plan-location francecentral \
|
|
--runtime python \
|
|
--runtime-version 3.12 \
|
|
--functions-version 4 \
|
|
--name contentrecommender-linux \
|
|
--storage-account stcontentrecommend2025 \
|
|
--os-type Linux
|
|
```
|
|
|
|
3. **Code deployment**
|
|
```bash
|
|
# From the azure-app folder
|
|
func azure functionapp publish contentrecommender-linux
|
|
```
|
|
|
|
### Streamlit Interface
|
|
|
|
1. **Install dependencies**
|
|
```bash
|
|
pip install -r streamlit_requirements.txt
|
|
```
|
|
|
|
2. **Launch the interface**
|
|
```bash
|
|
streamlit run streamlit_app.py
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Azure Functions API
|
|
|
|
**Endpoint**: `https://contentrecommender-linux.azurewebsites.net/api/recommend`
|
|
|
|
**Parameters**:
|
|
- `user_id` (int): User identifier (0-706)
|
|
|
|
**Example call**:
|
|
```bash
|
|
curl "https://contentrecommender-linux.azurewebsites.net/api/recommend?user_id=0"
|
|
```
|
|
|
|
**Response**:
|
|
```json
|
|
{
|
|
"user_id": 0,
|
|
"recommendations": [157519, 157944, 159495, 156690, 162857]
|
|
}
|
|
```
|
|
|
|
### Streamlit Interface
|
|
|
|
1. Configure the API URL (local or Azure)
|
|
2. Select a user ID
|
|
3. Get the recommendations
|
|
4. Test the advanced features
|
|
|
|
## Recommendation Algorithm
|
|
|
|
### Content-Based Filtering
|
|
|
|
1. **Data loading**: Precomputed embeddings and user history
|
|
2. **User profile computation**: Average of the embeddings of the viewed articles
|
|
3. **Cosine similarity**: Computed against all articles in the catalog
|
|
4. **Filtering**: Exclusion of already-seen articles
|
|
5. **Selection**: Top 5 most similar articles
|
|
|
|
### Optimizations
|
|
|
|
- **Dimensionality reduction**: PCA 250D → 50D to optimize Azure performance
|
|
- **In-memory cache**: Embeddings and user data loaded only once
|
|
- **Serverless architecture**: Automatic scaling based on demand
|
|
|
|
## Tests and Validation
|
|
|
|
### Local tests
|
|
```bash
|
|
# Start Azure Functions locally
|
|
func start
|
|
|
|
# Test the algorithm
|
|
python test_local.py
|
|
```
|
|
|
|
### Performance metrics
|
|
- **Response time**: ~200ms per recommendation
|
|
- **Supported users**: 707 in the current dataset
|
|
- **Available articles**: 364,047 with embeddings
|
|
|
|
## Known Issues and Solutions
|
|
|
|
### Azure Deployment
|
|
- **Python on Windows**: Not supported, must use Linux
|
|
- **Authentication**: Set `http_auth_level=func.AuthLevel.ANONYMOUS`
|
|
- **"Service unavailable" errors**: Temporary Azure issues, retry later
|
|
|
|
### Current Limitations
|
|
- **Cold start**: First request is slow after a period of inactivity
|
|
- **New users**: No handling of the cold-start problem
|
|
- **Data updates**: Manual process
|
|
|
|
## Target Architecture (Future)
|
|
|
|
### Planned Improvements
|
|
- **Cold start handling**: Popular recommendations for new users
|
|
- **Data pipeline**: Automatic embeddings update
|
|
- **Database**: Persistent storage of user profiles
|
|
- **Hybrid API**: Combination of Content-Based + Collaborative Filtering
|
|
- **Distributed cache**: Precomputation of popular recommendations
|
|
|
|
## Technologies Used
|
|
|
|
- **Backend**: Azure Functions, Python 3.12
|
|
- **Frontend**: Streamlit
|
|
- **ML/Data**: scikit-learn, pandas, numpy
|
|
- **Infrastructure**: Azure CLI, Azure Storage
|
|
- **Deployment**: Azure Functions Core Tools
|
|
|
|
## Author
|
|
|
|
Project carried out as part of an OpenClassrooms training program - AI Engineer.
|
|
|
|
## License
|
|
|
|
This project is for educational purposes only.
|