**Source of truth** : https://git.gregoiremureau.com/openclassroom-ai/oc-p02-paris-trees

```bash
git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p02-paris-trees.git
git clone https://git.gregoiremureau.com/openclassroom-ai/oc-p02-paris-trees.git
```

Dataset: see `fetch-data.sh` (out of git if large).

# Paris Trees Analysis Project

📘 This project is also available in [French 🇫🇷](./README.fr.md)

## 📋 Project Context

This project was carried out as part of a data analysis assignment for the **City of Paris**. The objective is to explore and analyze Paris' tree heritage from a public dataset listing the trees present across the territory of the capital.

The project focuses on the fundamentals of data science: data cleaning, anomaly handling, exploratory analysis, and statistical visualization to prepare the dataset for further analysis or business applications (urban management, biodiversity, planning).

## 🎯 Objectives

The main objectives of the project are:

1. **Explore the dataset**: Understand the structure, variable types, and characteristics of Paris' trees
2. **Clean the data**: Identify and handle missing, aberrant, and inconsistent values
3. **Analyze distributions**: Perform univariate and bivariate analyses to understand the variables
4. **Visualize the data**: Create charts and interactive maps to facilitate understanding
5. **Prepare the data**: Provide a clean and usable dataset for future analyses

## 📊 Data

The data comes from the **City of Paris Open Data** platform and contains detailed information about the trees:
- Geographic location (GPS coordinates, district, address)
- Botanical characteristics (genus, species, variety)
- Dimensions (height, circumference)
- Status (remarkable tree, ownership)
- Planting date

The dataset is included in the project under the name `p2-trees-fr.csv`.

**Source**: [Open Data City of Paris - Trees](https://opendata.paris.fr/explore/dataset/les-arbres/)

## 📁 Project Structure

```
├── Paris_trees_analysis.ipynb
├── soutenance_projet2.pptx
├── data_analysis_functions.py
├── map_arrondissement.html
├── map_lieu.html
└── p2-trees-fr.csv
```

### 1. `Paris_trees_analysis.ipynb`

**Main Notebook for Paris Trees Analysis**

This notebook is the core of the project and covers the entire exploratory analysis process:

**Data loading and overview**:
- Import of the CSV file containing the tree data
- Display of the first rows for a quick understanding
- Inspection of the DataFrame structure (types, dimensions)

**Initial exploratory analysis**:
- Use of pandas methods (`info()`, `head()`, `describe()`, `value_counts()`)
- Identification of the dataset size
- Analysis of column types and missing values
- Descriptive statistics for numerical variables

**Identification and cleaning of anomalies**:
- **Outliers**:
  - Detection for tree circumference and height
  - Use of statistical methods (IQR, z-score)
  - Correction or replacement of extreme values with `NaN`
- **Missing values**:
  - In-depth analysis of columns with NaN
  - Smart imputation for certain variables (`domanialite`, `libelle_francais`)
  - Use of correlated columns (`genre`, `espece`, `variete`) for imputation

**Univariate analysis**:
- **Quantitative variables**: distribution of height, circumference, planting year
- **Categorical variables**: breakdown by genus, species, district, remarkable status
- Visualizations: histograms, boxplots, pie charts, barplots

**Bivariate analysis**:
- Relationships between the `remarquable` status and quantitative variables (`hauteur_m`, `circonference_cm`)
- Relationships between `remarquable` and categorical variables (`espece`, `genre`)
- Correlations between numerical variables
- Statistical tests to validate observations

**Geographic analysis**:
- Distribution of trees by district
- Interactive mapping of Paris trees
- Identification of areas with high tree density

**Export of cleaned data**:
- Saving the prepared DataFrame for further analyses

### 2. `soutenance_projet2.pptx`

**Final Presentation**

This presentation summarizes:
- The context and objectives of the project
- The exploration and data cleaning methodology
- The main findings about Paris' tree heritage
- Key statistical analyses and visualizations
- Business insights (remarkable trees, distribution by district, dominant species)
- Recommendations for managing the tree heritage
- Perspectives for improvement and exploitation of the data

### 3. `data_analysis_functions.py`

**Analysis Utility Functions Module**

This Python module contains reusable functions for:
- Data cleaning (duplicate removal, NaN handling)
- Outlier identification (IQR, z-score, percentile methods)
- Outlier treatment (replacement, removal, capping)
- Variable transformation (normalization, encoding)
- Generation of data quality reports
- Standardized visualization functions

These functions were developed to be reusable on other data analysis projects.

### 4. `map_arrondissement.html`

**Interactive Map of Trees by District**

HTML file containing an interactive map (likely Folium or Plotly) that visualizes:
- The distribution of trees by Paris district
- Aggregated statistics by geographic area
- Interactive visualizations allowing zooming and exploring the data
- Potentially indicators on tree density, dominant species by district

### 5. `map_lieu.html`

**Interactive Map of Trees by Location**

HTML file containing a detailed interactive map that presents:
- The precise location of individual trees across the Paris territory
- Interactive markers with detailed information (species, height, circumference)
- Filters to explore trees by characteristics
- Clusters to improve readability in high-density areas

### 6. `p2-trees-fr.csv`

**Paris Trees Dataset**

CSV file containing raw or lightly preprocessed data on Paris trees, including:
- Identifiers and references
- Location (GPS coordinates, district, address)
- Botanical characteristics (genus, species, variety, French name)
- Dimensions (height in meters, circumference in cm)
- Status (remarkable or not, ownership)
- Planting date

## 🛠️ Technologies Used

- **Python**: Main language for data analysis
  - pandas, numpy: data manipulation and analysis
  - matplotlib, seaborn: data visualization
  - folium or plotly: interactive mapping
  - scipy: statistical tests
  - scikit-learn: outlier detection
- **Jupyter Notebook**: development and documentation
- **HTML/JavaScript**: interactive maps

## 📈 Methodology

1. **Initial exploration**: Understanding the structure and content of the dataset
2. **Cleaning**: Identification and handling of missing and aberrant values
3. **Univariate analyses**: Study of the distribution of each variable
4. **Bivariate analyses**: Exploration of relationships between variables
5. **Geographic visualization**: Interactive mapping of trees
6. **Documentation**: Presentation of results and insights

## 🎯 Expected Results

- ✅ In-depth understanding of Paris' tree heritage
- ✅ Cleaned dataset ready for advanced analyses
- ✅ Identification of remarkable trees and their characteristics
- ✅ Insights into the geographic distribution of trees
- ✅ Interactive maps to visually explore the data
- ✅ Reusable Python module for similar analyses

## 💡 Key Points of the Project

### Data Quality
Like any real-world dataset, this dataset presents challenges:
- **Missing values**: some incomplete botanical characteristics
- **Outliers**: data entry errors in measurements (impossible heights and circumferences)
- **Inconsistencies**: variations in formats and naming

### Remarkable Trees
Special attention was given to remarkable trees:
- Characteristics differentiating remarkable trees from others
- Relationships between dimensions (height, circumference) and remarkable status
- Geographic distribution of remarkable trees

### Geographic Analysis
The project integrates an important spatial dimension:
- Distribution of trees by district
- Identification of areas with high tree density
- Interactive maps for intuitive exploration
- Possibility of identifying geographic patterns

### Urban Biodiversity
The analyses reveal information about biodiversity:
- Most frequent tree species in Paris
- Botanical diversity by district
- Average age of trees (via planting dates)

## 🎓 Skills Developed

- Loading and exploring tabular data
- Cleaning real-world data with anomalies
- Handling missing values with imputation strategies
- Detection and treatment of outliers
- Univariate and bivariate statistical analyses
- Data visualization (charts and maps)
- Development of reusable Python functions
- Communication of data analysis results

## 👤 Author

**Grégoire Mureau**

---

*This project illustrates the fundamentals of exploratory data analysis: exploration, cleaning, visualization, and communication of insights, applied to a real case of urban open data.*