diff --git a/README.MD b/README.fr.md similarity index 99% rename from README.MD rename to README.fr.md index af6bec6..8df6a57 100644 --- a/README.MD +++ b/README.fr.md @@ -9,6 +9,8 @@ Dataset : voir `fetch-data.sh` (hors git si volumineux). # SystΓ¨me de Recommandation de Contenu - My Content +πŸ“˜ Ce projet est Γ©galement disponible en [anglais πŸ‡¬πŸ‡§](./README.md) + Projet de dΓ©veloppement d'un systΓ¨me de recommandation d'articles basΓ© sur une architecture serverless Azure Functions avec interface utilisateur Streamlit. diff --git a/README.md b/README.md new file mode 100644 index 0000000..62e0ea1 --- /dev/null +++ b/README.md @@ -0,0 +1,236 @@ +**Source of truth** : https://git.gregoiremureau.com/openclassroom-ai/oc-p10-content-recommendation + +```bash +git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p10-content-recommendation.git +git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p10-content-recommendation.git +``` + +Dataset: see `fetch-data.sh` (excluded from git if large). + +# Content Recommendation System - My Content + +πŸ“˜ This project is also available in [French πŸ‡«πŸ‡·](./README.fr.md) + +Project developing an article recommendation system based on a serverless Azure Functions architecture with a Streamlit user interface. + + +## Project Deliverables +1. **Working application**: Streamlit + Azure Functions deployed +2. **Versioned source code**: Complete GitHub repository +3. **Technical presentation**: PDF deck (docs/presentation.pdf) + +## Overview + +This project implements a content recommendation system using collaborative filtering and content-based algorithms. The serverless architecture enables automatic scaling and optimized cost. + +### Main Features + +- **Recommendation API**: Azure Function returning the top 5 recommended articles for a user +- **User interface**: Streamlit application to test and demonstrate the system +- **Implemented algorithms**: Content-Based Filtering with precomputed embeddings +- **Serverless architecture**: Deployment on Azure Functions with Consumption plan + + +## Analysis and Development + +### Jupyter Notebook +- **Data exploration**: Analysis of the Kaggle Globo.com dataset +- **Algorithm implementation**: Content-Based and Collaborative Filtering +- **Approach comparison**: Evaluation and technical choice +- **PCA optimization**: Dimensionality reduction 250D β†’ 50D + +Location: `notebook/p10_notebook.ipynb` + +## Architecture + +### Current MVP +``` +[Streamlit Interface] β†’ [Azure Functions API] β†’ [Recommendation Algorithm] + ↓ + [Embeddings data + user history] +``` + +### Components +- **Frontend**: Streamlit (test interface) +- **Backend**: Azure Functions (Python 3.12, Linux) +- **Algorithm**: Content-Based Filtering with cosine similarity +- **Data**: Article embeddings (50D via PCA vs 250D original embeddings) + click history + +## Project Structure Overview + +``` +Project_10/ +β”œβ”€β”€ azure-app/ # Azure Function +β”‚ β”œβ”€β”€ function_app.py # Azure Function entry point +β”‚ β”œβ”€β”€ requirements.txt # Python dependencies +β”‚ β”œβ”€β”€ host.json # Azure Functions configuration +β”‚ β”œβ”€β”€ local.settings.json # Local configuration (not versioned) +β”‚ β”œβ”€β”€ data/ +β”‚ β”‚ β”œβ”€β”€ embeddings_azure_50d.pkl # Article embeddings (PCA 50D) +β”‚ β”‚ └── clicks_sample.csv # User click history +β”‚ └── utils/ +β”‚ β”œβ”€β”€ data_loader.py # Data loading and caching +β”‚ └── content_based.py # Recommendation algorithm +β”œβ”€β”€ notebook/ +β”‚ └── p10_notebook.ipynb # Data analysis + model development +β”œβ”€β”€ docs/ +β”‚ β”œβ”€β”€ presentation.pdf # Presentation deck (15-25 slides) +β”‚ └── illustrations/ # Architecture diagrams and screenshots +β”œβ”€β”€ streamlit_app.py # User interface +β”œβ”€β”€ streamlit_requirements.txt # Streamlit dependencies +└── README.md +``` + +## Installation and Deployment + +### Prerequisites +- Python 3.12+ +- Azure CLI +- Azure Functions Core Tools +- Azure account + +### Azure Functions Deployment + +1. **Sign in to Azure** +```bash +az login +``` + +2. **Infrastructure creation** +```bash +# Resource group +az group create --name rg-content-recommender --location francecentral + +# Storage account +az storage account create --name [your-unique-storage-name] \ + --resource-group rg-content-recommender \ + --location francecentral \ + --sku Standard_LRS + +# Linux Function App with Python 3.12 +az functionapp create \ + --resource-group rg-content-recommender \ + --consumption-plan-location francecentral \ + --runtime python \ + --runtime-version 3.12 \ + --functions-version 4 \ + --name contentrecommender-linux \ + --storage-account stcontentrecommend2025 \ + --os-type Linux +``` + +3. **Code deployment** +```bash +# From the azure-app folder +func azure functionapp publish contentrecommender-linux +``` + +### Streamlit Interface + +1. **Install dependencies** +```bash +pip install -r streamlit_requirements.txt +``` + +2. **Launch the interface** +```bash +streamlit run streamlit_app.py +``` + +## Usage + +### Azure Functions API + +**Endpoint**: `https://contentrecommender-linux.azurewebsites.net/api/recommend` + +**Parameters**: +- `user_id` (int): User identifier (0-706) + +**Example call**: +```bash +curl "https://contentrecommender-linux.azurewebsites.net/api/recommend?user_id=0" +``` + +**Response**: +```json +{ + "user_id": 0, + "recommendations": [157519, 157944, 159495, 156690, 162857] +} +``` + +### Streamlit Interface + +1. Configure the API URL (local or Azure) +2. Select a user ID +3. Get the recommendations +4. Test the advanced features + +## Recommendation Algorithm + +### Content-Based Filtering + +1. **Data loading**: Precomputed embeddings and user history +2. **User profile computation**: Average of the embeddings of the viewed articles +3. **Cosine similarity**: Computed against all articles in the catalog +4. **Filtering**: Exclusion of already-seen articles +5. **Selection**: Top 5 most similar articles + +### Optimizations + +- **Dimensionality reduction**: PCA 250D β†’ 50D to optimize Azure performance +- **In-memory cache**: Embeddings and user data loaded only once +- **Serverless architecture**: Automatic scaling based on demand + +## Tests and Validation + +### Local tests +```bash +# Start Azure Functions locally +func start + +# Test the algorithm +python test_local.py +``` + +### Performance metrics +- **Response time**: ~200ms per recommendation +- **Supported users**: 707 in the current dataset +- **Available articles**: 364,047 with embeddings + +## Known Issues and Solutions + +### Azure Deployment +- **Python on Windows**: Not supported, must use Linux +- **Authentication**: Set `http_auth_level=func.AuthLevel.ANONYMOUS` +- **"Service unavailable" errors**: Temporary Azure issues, retry later + +### Current Limitations +- **Cold start**: First request is slow after a period of inactivity +- **New users**: No handling of the cold-start problem +- **Data updates**: Manual process + +## Target Architecture (Future) + +### Planned Improvements +- **Cold start handling**: Popular recommendations for new users +- **Data pipeline**: Automatic embeddings update +- **Database**: Persistent storage of user profiles +- **Hybrid API**: Combination of Content-Based + Collaborative Filtering +- **Distributed cache**: Precomputation of popular recommendations + +## Technologies Used + +- **Backend**: Azure Functions, Python 3.12 +- **Frontend**: Streamlit +- **ML/Data**: scikit-learn, pandas, numpy +- **Infrastructure**: Azure CLI, Azure Storage +- **Deployment**: Azure Functions Core Tools + +## Author + +Project carried out as part of an OpenClassrooms training program - AI Engineer. + +## License + +This project is for educational purposes only.