Préparer le repo P7 autonome : CI preprocessing, fetch-data, URLs prod.

Le dataset Sentiment140 reste hors git ; les tests TF/MLflow ne tournent pas encore sur le runner k3s.
This commit is contained in:
Grégoire Mureau 2026-09-09 12:58:39 +02:00
parent 6cd377eee7
commit 5b58da9f01
4 changed files with 67 additions and 2 deletions

30
.github/workflows/ci.yml vendored Normal file
View file

@ -0,0 +1,30 @@
# CI — P7 Sentiment API
# Même fichier lu par Forgejo Actions et GitHub Actions (OC-ADR-001).
# Image job = node:22-bookworm (label runner mini-pc-k3s). Pas d'image ubuntu native.
name: CI
on:
push:
branches: [main]
workflow_dispatch:
jobs:
test-preprocessing:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Python + pytest (preprocessing only)
run: |
set -eu
apt-get update
apt-get install -y --no-install-recommends python3 python3-pip python3-venv
python3 -m pip install --break-system-packages nltk pytest
python3 -m nltk.downloader punkt punkt_tab stopwords wordnet omw-1.4
python3 -m pytest deployment/api/tests/test_preprocessing.py -q
# test_main.py charge TensorFlow + artefacts MLflow (prod). Pas dans ce job :
# trop lourd pour le runner (DinD 8 Gio) et exige des secrets MLflow.
# Build/push image : Phase 6 (dual registry), après secrets Forgejo.

8
.gitignore vendored
View file

@ -1,2 +1,10 @@
**/mlruns
**/downloaded_artifacts
**/.env
**/.env.*
**/.pytest_cache
**/__pycache__
*.pem
*aws*
sentiment-api-*.json

View file

@ -4,6 +4,14 @@
This repository contains the notebooks and source code for the sentiment analysis project on tweets, conducted as part of the OpenClassrooms **"Air Paradis"** program.
**Source of truth** : [Forgejo](https://git.gregoiremureau.com/grgmro/oc-p07-sentiment-api) (self-hosted). GitHub, when mirrored, is the public shop window.
```bash
git clone https://git.gregoiremureau.com/grgmro/oc-p07-sentiment-api.git
# HTTPS (Cloudflare, fine for clone) :
git clone https://git.gregoiremureau.com/grgmro/oc-p07-sentiment-api.git
```
---
## Business Context
@ -33,7 +41,8 @@ The project aims to develop a complete sentiment analysis system, including:
- `deployment/`: FastAPI API code, unit tests, Dockerfile and deployment configuration,
- `docs/`: detailed documentation (blog.md), diagrams, screenshots,
- `presentation/`: project presentation slides (PDF),
- `../.github/workflows/`: CI/CD pipelines (automated tests, Docker build, DockerHub push).
- `.github/workflows/`: CI (Forgejo Actions on the homelab runner; same YAML on GitHub),
- `fetch-data.sh`: where to get Sentiment140 (dataset **not** in git).
---
@ -48,7 +57,7 @@ Training phases were primarily conducted on **Google Colab** to leverage GPU res
Notebooks detail the analysis and training steps.
The API is available in the `deployment/api` folder with Streamlit testing interface.
To test the production API: https://sentiment-api.greg-madman-nas.duckdns.org
To test the production API: https://api.gregoiremureau.com/ml/sentiment/
---
@ -56,6 +65,7 @@ To test the production API: https://sentiment-api.greg-madman-nas.duckdns.org
- The `requirements.txt` files in `notebooks/` come from Google Colab environments and may contain dependencies not optimized for local use.
- The `.env` file, excluded from the repository, contains necessary environment variables (MLFlow, MinIO, alerting configuration).
- Training data: run `./fetch-data.sh` (Sentiment140 is **not** stored in git).
---

17
fetch-data.sh Executable file
View file

@ -0,0 +1,17 @@
#!/usr/bin/env bash
# Datasets hors git (OC-ADR-003). Ce script n'en télécharge pas : il documente la source.
set -euo pipefail
cat <<'EOF'
P7 — Sentiment140 (tweets annotés positif/négatif)
Source amont :
http://help.sentiment140.com/for-students
archive historique : http://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip
miroir Kaggle : https://www.kaggle.com/datasets/kazanova/sentiment140
Ne pas versionner le CSV/zip dans ce repo.
Copie locale éventuelle : archive scolaire Nextcloud
Master_openclassroom_AI_Engineer/07 Réalisez une analyse de sentiments...
EOF