Préparer le repo P7 autonome : CI preprocessing, fetch-data, URLs prod.
Le dataset Sentiment140 reste hors git ; les tests TF/MLflow ne tournent pas encore sur le runner k3s.
This commit is contained in:
parent
6cd377eee7
commit
5b58da9f01
4 changed files with 67 additions and 2 deletions
30
.github/workflows/ci.yml
vendored
Normal file
30
.github/workflows/ci.yml
vendored
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
# CI — P7 Sentiment API
|
||||
# Même fichier lu par Forgejo Actions et GitHub Actions (OC-ADR-001).
|
||||
# Image job = node:22-bookworm (label runner mini-pc-k3s). Pas d'image ubuntu native.
|
||||
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
test-preprocessing:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Python + pytest (preprocessing only)
|
||||
run: |
|
||||
set -eu
|
||||
apt-get update
|
||||
apt-get install -y --no-install-recommends python3 python3-pip python3-venv
|
||||
python3 -m pip install --break-system-packages nltk pytest
|
||||
python3 -m nltk.downloader punkt punkt_tab stopwords wordnet omw-1.4
|
||||
python3 -m pytest deployment/api/tests/test_preprocessing.py -q
|
||||
|
||||
# test_main.py charge TensorFlow + artefacts MLflow (prod). Pas dans ce job :
|
||||
# trop lourd pour le runner (DinD 8 Gio) et exige des secrets MLflow.
|
||||
# Build/push image : Phase 6 (dual registry), après secrets Forgejo.
|
||||
8
.gitignore
vendored
8
.gitignore
vendored
|
|
@ -1,2 +1,10 @@
|
|||
**/mlruns
|
||||
**/downloaded_artifacts
|
||||
**/.env
|
||||
**/.env.*
|
||||
**/.pytest_cache
|
||||
**/__pycache__
|
||||
*.pem
|
||||
*aws*
|
||||
sentiment-api-*.json
|
||||
|
||||
|
|
|
|||
14
README.md
14
README.md
|
|
@ -4,6 +4,14 @@
|
|||
|
||||
This repository contains the notebooks and source code for the sentiment analysis project on tweets, conducted as part of the OpenClassrooms **"Air Paradis"** program.
|
||||
|
||||
**Source of truth** : [Forgejo](https://git.gregoiremureau.com/grgmro/oc-p07-sentiment-api) (self-hosted). GitHub, when mirrored, is the public shop window.
|
||||
|
||||
```bash
|
||||
git clone https://git.gregoiremureau.com/grgmro/oc-p07-sentiment-api.git
|
||||
# HTTPS (Cloudflare, fine for clone) :
|
||||
git clone https://git.gregoiremureau.com/grgmro/oc-p07-sentiment-api.git
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Business Context
|
||||
|
|
@ -33,7 +41,8 @@ The project aims to develop a complete sentiment analysis system, including:
|
|||
- `deployment/`: FastAPI API code, unit tests, Dockerfile and deployment configuration,
|
||||
- `docs/`: detailed documentation (blog.md), diagrams, screenshots,
|
||||
- `presentation/`: project presentation slides (PDF),
|
||||
- `../.github/workflows/`: CI/CD pipelines (automated tests, Docker build, DockerHub push).
|
||||
- `.github/workflows/`: CI (Forgejo Actions on the homelab runner; same YAML on GitHub),
|
||||
- `fetch-data.sh`: where to get Sentiment140 (dataset **not** in git).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -48,7 +57,7 @@ Training phases were primarily conducted on **Google Colab** to leverage GPU res
|
|||
Notebooks detail the analysis and training steps.
|
||||
The API is available in the `deployment/api` folder with Streamlit testing interface.
|
||||
|
||||
To test the production API: https://sentiment-api.greg-madman-nas.duckdns.org
|
||||
To test the production API: https://api.gregoiremureau.com/ml/sentiment/
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -56,6 +65,7 @@ To test the production API: https://sentiment-api.greg-madman-nas.duckdns.org
|
|||
|
||||
- The `requirements.txt` files in `notebooks/` come from Google Colab environments and may contain dependencies not optimized for local use.
|
||||
- The `.env` file, excluded from the repository, contains necessary environment variables (MLFlow, MinIO, alerting configuration).
|
||||
- Training data: run `./fetch-data.sh` (Sentiment140 is **not** stored in git).
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
17
fetch-data.sh
Executable file
17
fetch-data.sh
Executable file
|
|
@ -0,0 +1,17 @@
|
|||
#!/usr/bin/env bash
|
||||
# Datasets hors git (OC-ADR-003). Ce script n'en télécharge pas : il documente la source.
|
||||
set -euo pipefail
|
||||
|
||||
cat <<'EOF'
|
||||
P7 — Sentiment140 (tweets annotés positif/négatif)
|
||||
|
||||
Source amont :
|
||||
http://help.sentiment140.com/for-students
|
||||
archive historique : http://cs.stanford.edu/people/alecmgo/trainingandtestdata.zip
|
||||
miroir Kaggle : https://www.kaggle.com/datasets/kazanova/sentiment140
|
||||
|
||||
Ne pas versionner le CSV/zip dans ce repo.
|
||||
|
||||
Copie locale éventuelle : archive scolaire Nextcloud
|
||||
Master_openclassroom_AI_Engineer/07 Réalisez une analyse de sentiments...
|
||||
EOF
|
||||
Loading…
Add table
Reference in a new issue