Pablo.md
Português

Olist: an End-to-End Brazilian E-Commerce Case Study

This project is different from any playlist here. A playlist is a class, one concept at a time, each post independent from the last. This is an end-to-end data science case study: I take a real, messy dataset with nine tables tied together, and go all the way through, EDA, visualization, feature engineering, model training, interpretability, and a business conclusion. The chapters build on each other, chapter 3 assumes you've already read chapters 1 and 2.

The dataset is the Brazilian E-Commerce Public Dataset, by Olist, available on Kaggle. Real (anonymized) data from over 100 thousand orders placed between 2016 and 2018 on an actual Brazilian marketplace, customer, seller, product, payment, review, all tied together by keys. All the heavy processing (pandas, scikit-learn, XGBoost, SHAP) runs in a notebook I execute on Google Colab, with a T4 GPU when a chapter needs one. The real results of those notebooks become the interactive charts here on the blog, never an invented number.

The four scenarios

Defined right in the first chapter, because they steer the whole project:

  1. Delivery delay prediction (binary classification): does the order arrive after the estimated date or not?
  2. Review score prediction (classification/regression): how many stars will the customer give?
  3. Freight or order value prediction (regression): how much will it cost?
  4. Customer segmentation (clustering, RFM): what kinds of customers exist in this base?

Each pulls a different slice of the same dataframe, and the goal is to show you how the same base dataset serves very different business questions.

Start with Chapter 1: relational model and first exploration.

Projects · 4 chapters

  1. Nine Tables, One Business: Olist's Relational Model

    First chapter of the Olist case study: I load the nine CSVs, build the master dataframe, check real data quality, and define the four ML scenarios the rest of the project will cover.

  2. Brazil Shops on Weekdays: Olist's Exploratory Storytelling

    Second chapter of the Olist case study: a purchase-time heatmap, a delay choropleth by state, top revenue categories, payment method, and the correlation that delivers the punchline before any model gets trained: delay kills the review score.

  3. Data Leakage: the Easiest Way to Fool Yourself

    Third chapter of the Olist case study: feature engineering and selection for the four scenarios, with a live demonstration of data leakage (a model with 100% AUC that's useless) and a real discovery about seasonality.

  4. Good AUC, Zero F1: the Default Threshold Trap

    Fourth chapter of the Olist case study: training the four real models (delay, review, freight, RFM) with MLflow tracking, and finding a Random Forest with 0.74 AUC that doesn't catch a single positive case at the default threshold.