Grégoire Mureau
02a2cd0a28
All checks were successful
CI / syntax (push) Successful in 8s
Public READMEs now expose a single git.gregoiremureau.com clone URL. The Tailscale SSH form only works on the homelab network and leaked the NAS address. The language switcher sits right under the title so an English speaker landing on the French page can leave without reading French.
130 lines
5 KiB
Markdown
130 lines
5 KiB
Markdown
# Street Vision PoC - Urban Semantic Segmentation
|
||
|
||
📘 Ce projet est également disponible en [français 🇫🇷](./README.fr.md)
|
||
|
||
**Source of truth** : [Forgejo](https://git.gregoiremureau.com/openclassroom-ai/oc-p09-street-vision-poc) (self-hosted).
|
||
|
||
```bash
|
||
git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p09-street-vision-poc.git
|
||
```
|
||
|
||
Prod: https://api.gregoiremureau.com/ml/street-vision-poc/
|
||
|
||
Dataset: `./fetch-data.sh` (Cityscapes **not** in git).
|
||
|
||
## 🎯 Project Description
|
||
|
||
**Street Vision PoC** is a proof of concept demonstrating the improvement of urban semantic segmentation performance by comparing traditional CNN architectures (FPN) with modern Transformer approaches (SegFormer).
|
||
|
||
This project empirically evaluates the potential of **Vision Transformers** for understanding complex urban scenes, with a focus on safety-critical classes (pedestrians, vehicles, signage).
|
||
|
||
### 🔬 Technical Context
|
||
|
||
- **Dataset**: Cityscapes (2,975 high-resolution pixel-by-pixel annotated images)
|
||
- **Compared architectures**:
|
||
- **Traditional CNN**: FPN + EfficientNet-B0/ResNet34
|
||
- **Vision Transformer**: SegFormer-B0/B1
|
||
- **Evaluation resolutions**: 512×512 and 768×768 pixels
|
||
- **Main metric**: Mean Intersection over Union (mIoU)
|
||
|
||
## 📊 Main Results
|
||
|
||
| **Model** | **mIoU** | **Parameters** | **Size** | **Performance** |
|
||
|------------|----------|----------------|------------|-----------------|
|
||
| **SegFormer-B1** | **79.98%** | 13.68M | 52MB | ⭐ **Best** |
|
||
| **FPN+ResNet34** | 79.22% | 23.16M | 88MB | 🔄 Strong baseline |
|
||
| **FPN+EfficientNet-B0** | 78.67% | 5.76M | 22MB | 💡 Compact |
|
||
| **SegFormer-B0** | 77.91% | 3.72M | 14MB | ⚡ Ultra-lightweight |
|
||
|
||
**Comparative analysis**: SegFormer-B1 and FPN+ResNet34 achieve similar performance (~79-80% mIoU), but SegFormer-B1 has the advantage of being more compact and likely faster on GPU. The SegFormer-B2 to B5 versions would promise significant gains, but my hardware constraints did not allow their evaluation.
|
||
|
||
### 🎯 Significant Gains by Critical Classes
|
||
|
||
- **Human**: +1.78% IoU (safety-critical)
|
||
- **Vehicle**: +1.05% IoU (mobile obstacles)
|
||
- **Object**: +0.56% IoU (urban signage)
|
||
|
||
## 🚀 Quick Deployment
|
||
|
||
### Docker Hub
|
||
|
||
The application is containerized and available on Docker Hub:
|
||
|
||
```bash
|
||
docker pull grgmdmn/street-vision-poc_api
|
||
# live prod:
|
||
# https://api.gregoiremureau.com/ml/street-vision-poc/
|
||
```
|
||
|
||
### Configuration with Docker Compose
|
||
|
||
1. Copy the example configuration file:
|
||
```bash
|
||
cp docker-compose.yml.example docker-compose.yml
|
||
```
|
||
|
||
2. Configure the environment variables in `docker-compose.yml`.
|
||
|
||
3. Launch the application:
|
||
```bash
|
||
docker-compose up -d
|
||
```
|
||
|
||
4. Access the interface: `http://localhost:$(port)`
|
||
|
||
## 🎮 Interface Features
|
||
|
||
### Interactive Dashboard
|
||
|
||
- **Image upload**: Drag-and-drop of urban images
|
||
- **Dual inference**: Simultaneous comparison of FPN vs SegFormer
|
||
- **Visualization**: Segmentation masks colored by class
|
||
- **Real-time metrics**: Inference time and confidence scores
|
||
- **Export**: Download of segmentation results
|
||
- **WCAG accessibility**: Interface compliant with accessibility standards
|
||
|
||
### Exploratory Data Analysis
|
||
|
||
- **Cityscapes statistics**: Class distribution, native resolutions
|
||
- **Interactive charts**: Pixel breakdown by meta-class
|
||
- **Annotated examples**: Visualization of dataset samples
|
||
|
||
## 📈 Evaluation Methodology
|
||
|
||
### Comparative Approach
|
||
|
||
1. **CNN baseline**: FPN+EfficientNet/ResNet retraining at higher resolutions
|
||
2. **New approach**: SegFormer evaluation with global attention
|
||
3. **Rigorous protocol**: Cross-validation, standardized metrics
|
||
4. **Per-class analysis**: Detailed performance across 8 urban meta-classes
|
||
|
||
### Identified Limitations
|
||
|
||
- **Fixed resolution**: Evaluation without sliding window (vs. original protocol)
|
||
- **Limited resources**: Tests only on SegFormer-B0/B1
|
||
- **Controlled conditions**: Cityscapes dataset only (favorable weather)
|
||
|
||
## 🔬 Technical Insights
|
||
|
||
### SegFormer Advantages
|
||
|
||
- **Global attention**: Every pixel "sees" the entire image simultaneously
|
||
- **Lighter architecture**: Compact model with fewer parameters
|
||
- **Contextual robustness**: Better handling of occlusions and complex objects
|
||
- **Improvement potential**: B2-B5 versions would promise significant gains
|
||
|
||
### Identified Trade-offs
|
||
|
||
- **Classic CPU speed**: ~2x slower than FPN on high-performance processors (Xeon, 9600X)
|
||
- **Energy-efficient CPU speed**: 3x faster on energy-efficient processors (N100)
|
||
- **GPU optimization**: Transformer architecture favored by hardware acceleration
|
||
|
||
## 📚 References
|
||
|
||
- **Xie, E. et al.** (2021). *SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers*. NeurIPS 2021.
|
||
- **Cordts, M. et al.** (2016). *The Cityscapes Dataset for Semantic Urban Scene Understanding*. CVPR 2016.
|
||
- **Segmentation Models PyTorch** - [Implementation framework used](https://github.com/qubvel/segmentation_models.pytorch)
|
||
|
||
---
|
||
|
||
**Note**: This proof of concept demonstrates the potential of Transformer architectures for urban segmentation, while highlighting the importance of computational resources to fully leverage their capabilities.
|