Pablo.md
Português

Olist: an End-to-End Brazilian E-Commerce Case Study

This project is different from any playlist here. A playlist is a class, one concept at a time, each post independent from the last. This is an end-to-end data science case study: I take a real, messy dataset with nine tables tied together, and go all the way through, EDA, visualization, feature engineering, model training, interpretability, and a business conclusion. The chapters build on each other, chapter 3 assumes you've already read chapters 1 and 2.

The dataset is the Brazilian E-Commerce Public Dataset, by Olist, available on Kaggle. Real (anonymized) data from over 100 thousand orders placed between 2016 and 2018 on an actual Brazilian marketplace, customer, seller, product, payment, review, all tied together by keys. All the heavy processing (pandas, scikit-learn, XGBoost, SHAP) runs in a notebook I execute on Google Colab, with a T4 GPU when a chapter needs one. The real results of those notebooks become the interactive charts here on the blog, never an invented number.

The four scenarios

Defined right in the first chapter, because they steer the whole project:

  1. Delivery delay prediction (binary classification): does the order arrive after the estimated date or not?
  2. Review score prediction (classification/regression): how many stars will the customer give?
  3. Freight or order value prediction (regression): how much will it cost?
  4. Customer segmentation (clustering, RFM): what kinds of customers exist in this base?

Each pulls a different slice of the same dataframe, and the goal is to show you how the same base dataset serves very different business questions.

A fifth angle, as a bonus

The four scenarios above never touched real time series. Chapter 6 closes that gap with month-by-month demand forecasting, outside the original numbering, but built on the same master dataframe.

Just want to play with the results before understanding how I got there? Start with Chapter 0, the invitation. Want the full process from the first CSV? Start with Chapter 1: relational model and first exploration.

Projects · 7 chapters

  1. Chapter 0: Play First, Understand Later

    Before the five technical chapters of the Olist case study, an invitation: several different ways to play with the whole project's real results, including a freight simulator and a revenue panel with a period filter, no explanation behind them yet, just the curiosity of figuring out what the numbers hide on your own.

  2. Chapter 1: Nine Tables, One Business: Olist's Relational Model

    First chapter of the Olist case study: I load the nine CSVs, build the master dataframe, check real data quality, and define the four ML scenarios the rest of the project will cover.

  3. Chapter 2: Brazil Shops on Weekdays: Olist's Exploratory Storytelling

    Second chapter of the Olist case study: a purchase-time heatmap, a delay choropleth by state, top revenue categories, payment method, and the correlation that delivers the punchline before any model gets trained: delay kills the review score.

  4. Chapter 3: Data Leakage: the Easiest Way to Fool Yourself

    Third chapter of the Olist case study: feature engineering and selection for the four scenarios, with a live demonstration of data leakage (a model with 100% AUC that's useless) and a real discovery about seasonality.

  5. Chapter 4: Good AUC, Zero F1: the Default Threshold Trap

    Fourth chapter of the Olist case study: training the four real models (delay, review, freight, RFM) with MLflow tracking, and finding a Random Forest with 0.74 AUC that doesn't catch a single positive case at the default threshold.

  6. Chapter 5: RFM with K=4: Four Clusters, But Only Two Actually Say Something

    Fifth chapter of the Olist case study: SHAP opens up the delay and review models' black box, confirming Chapter 3's findings, and the final RFM segmentation shows 94% of the customer base falls into two nearly identical clusters, only the other two actually differentiate anyone.

  7. Chapter 6: Demand Forecasting: When the Simple Model Beats the Sophisticated One

    Sixth chapter of the Olist case study, the project's only time-series angle: I compare three ways of predicting monthly orders on a real holdout, and the most naive model wins by a wide margin over a regression with trend and seasonality.