[{"data":1,"prerenderedAt":456},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fprojects\u002Folist-ecommerce\u002F03-features-e-selecao":3,"chapter-en-olist-ecommerce-03-features-e-selecao":4},null,{"id":5,"title":6,"body":7,"cover":3,"date":442,"description":443,"extension":444,"meta":445,"navigation":57,"order":61,"path":446,"project":447,"seo":448,"status":449,"stem":450,"tags":451,"__hash__":455},"projectChapters\u002Fen\u002Fprojects\u002Folist-ecommerce\u002F03-features-e-selecao.md","Data Leakage: the Easiest Way to Fool Yourself",{"type":8,"value":9,"toc":434},"minimark",[10,22,27,35,101,116,120,157,172,175,180,196,199,204,216,220,229,238,307,315,319,322,376,379,383,386,416,423,427,430],[11,12,13,14,21],"p",{},"Chapters 1 and 2 built the master dataframe and showed six exploratory angles. Now I need to pick, for each of the four scenarios defined back in Chapter 1, which features actually make sense to use. Full executed notebook, ",[15,16,20],"a",{"href":17,"rel":18},"https:\u002F\u002Fcolab.research.google.com\u002Fdrive\u002F1nt9wVC9_2H646HEKG-XTo6P4cvx-zPPr?usp=sharing",[19],"nofollow","public on Colab",".",[23,24,26],"h2",{"id":25},"a-new-feature-first-customer-seller-distance","A new feature first: customer-seller distance",[11,28,29,30,34],{},"Chapter 2 showed that customer state correlates with delivery time. Before diving into the scenarios, I computed a real distance, not just \"same state or not\": average latitude\u002Flongitude per zip code prefix (",[31,32,33],"code",{},"olist_geolocation_dataset",", over a million rows, hence averaging first), and the Haversine formula between customer and seller.",[36,37,42],"pre",{"className":38,"code":39,"language":40,"meta":41,"style":41},"language-python shiki shiki-themes github-light github-dark","geo_media = geolocation.groupby('geolocation_zip_code_prefix')[['geolocation_lat', 'geolocation_lng']].mean().reset_index()\n\ndef haversine(lat1, lon1, lat2, lon2):\n    R = 6371\n    phi1, phi2 = np.radians(lat1), np.radians(lat2)\n    dphi = np.radians(lat2 - lat1)\n    dlambda = np.radians(lon2 - lon1)\n    a = np.sin(dphi \u002F 2) ** 2 + np.cos(phi1) * np.cos(phi2) * np.sin(dlambda \u002F 2) ** 2\n    return 2 * R * np.arcsin(np.sqrt(a))\n","python","",[31,43,44,52,59,65,71,77,83,89,95],{"__ignoreMap":41},[45,46,49],"span",{"class":47,"line":48},"line",1,[45,50,51],{},"geo_media = geolocation.groupby('geolocation_zip_code_prefix')[['geolocation_lat', 'geolocation_lng']].mean().reset_index()\n",[45,53,55],{"class":47,"line":54},2,[45,56,58],{"emptyLinePlaceholder":57},true,"\n",[45,60,62],{"class":47,"line":61},3,[45,63,64],{},"def haversine(lat1, lon1, lat2, lon2):\n",[45,66,68],{"class":47,"line":67},4,[45,69,70],{},"    R = 6371\n",[45,72,74],{"class":47,"line":73},5,[45,75,76],{},"    phi1, phi2 = np.radians(lat1), np.radians(lat2)\n",[45,78,80],{"class":47,"line":79},6,[45,81,82],{},"    dphi = np.radians(lat2 - lat1)\n",[45,84,86],{"class":47,"line":85},7,[45,87,88],{},"    dlambda = np.radians(lon2 - lon1)\n",[45,90,92],{"class":47,"line":91},8,[45,93,94],{},"    a = np.sin(dphi \u002F 2) ** 2 + np.cos(phi1) * np.cos(phi2) * np.sin(dlambda \u002F 2) ** 2\n",[45,96,98],{"class":47,"line":97},9,[45,99,100],{},"    return 2 * R * np.arcsin(np.sqrt(a))\n",[11,102,103,104,107,108,111,112,115],{},"Found a real bug at this step before publishing: on the first run, the merge between ",[31,105,106],{},"customer_zip_code_prefix","\u002F",[31,109,110],{},"seller_zip_code_prefix"," and the geolocation table lost 70% of the data (only 30% coverage), because the zip-code column's data type wasn't guaranteed to match on both sides across every environment. Forced explicit ",[31,113,114],{},"int64"," on all three before merging, and coverage went back to what it should be: only 581 orders out of 118,310 ended up without a distance (99.5% coverage). Average customer-seller distance across all of Brazil is 597 km, with a max of 8,678 km, a number that proves just how huge the country is.",[23,117,119],{"id":118},"scenario-1-delivery-delay-and-the-data-leakage-demonstration","Scenario 1: delivery delay, and the data leakage demonstration",[11,121,122,123,126,127,126,130,126,133,136,137,140,141,144,145,148,149,152,153,156],{},"Candidates: ",[31,124,125],{},"distance_km",", ",[31,128,129],{},"product_weight_g",[31,131,132],{},"price",[31,134,135],{},"freight_value",", approval time (",[31,138,139],{},"approval_hours","), purchase month. The ",[31,142,143],{},"atrasado"," target is defined by comparing ",[31,146,147],{},"order_delivered_customer_date"," against ",[31,150,151],{},"order_estimated_delivery_date",". And this is where the classic trap of every ML course lives: if I include ",[31,154,155],{},"atraso_dias"," (the actual difference between those two dates) as a feature, the model isn't learning to predict delay, it's reading the answer straight off the exam.",[36,158,160],{"className":38,"code":159,"language":40,"meta":41,"style":41},"features_honestas = ['distance_km', 'product_weight_g', 'price', 'freight_value', 'approval_hours', 'month']\nfeatures_vazadas = features_honestas + ['atraso_dias']\n",[31,161,162,167],{"__ignoreMap":41},[45,163,164],{"class":47,"line":48},[45,165,166],{},"features_honestas = ['distance_km', 'product_weight_g', 'price', 'freight_value', 'approval_hours', 'month']\n",[45,168,169],{"class":47,"line":54},[45,170,171],{},"features_vazadas = features_honestas + ['atraso_dias']\n",[11,173,174],{},"Trained both, a plain RandomForest, same seed, same train\u002Ftest split:",[176,177],"olist-leakage-comparison-chart",{"x-label":178,"y-label":179},"model","AUC",[11,181,182,183,187,188,191,192,195],{},"The honest model lands at ",[184,185,186],"strong",{},"AUC 0.7388"," (95,968 orders used, real delay rate of 6.76%). The leaky model lands at ",[184,189,190],{},"AUC 1.0",", perfect, because ",[31,193,194],{},"atraso_dias > 0"," is practically the target's own definition. One detail worth calling out: the honest model's raw accuracy (93.24%) looks great at first glance, but since only 6.76% of orders are late, a dumb model that always guesses \"on time\" already scores 93.24% without learning anything. Accuracy alone is misleading with classes this imbalanced, AUC is the metric that tells the real story here.",[11,197,198],{},"Also worth noting, the honest model's feature importance:",[200,201],"olist-feature-importance-chart",{"x-label":202,"y-label":203},"feature","importance",[11,205,206,212,213,215],{},[184,207,208,211],{},[31,209,210],{},"month"," leads by a wide margin, at 0.36 importance",", practically double the runner-up (",[31,214,125],{},", 0.18). This ties directly back to Chapter 2's finding: November (Black Friday) strains logistics in a way that physical distance alone can't explain. Seasonality matters more for predicting delay than how far the seller is from the customer.",[23,217,219],{"id":218},"scenario-2-review-score","Scenario 2: review score",[11,221,122,222,224,225,228],{},[31,223,155],{},", delivery time, price, freight, number of installments. Since ",[31,226,227],{},"review_score"," is a bimodal ordinal scale (Chapter 2 already showed this), I used mutual information instead of plain Pearson correlation:",[36,230,232],{"className":38,"code":231,"language":40,"meta":41,"style":41},"mi = mutual_info_classif(X2, y2, random_state=42)\n",[31,233,234],{"__ignoreMap":41},[45,235,236],{"class":47,"line":48},[45,237,231],{},[239,240,241,256],"table",{},[242,243,244],"thead",{},[245,246,247,252],"tr",{},[248,249,251],"th",{"align":250},"left","Feature",[248,253,255],{"align":254},"right","Mutual information",[257,258,259,269,279,288,297],"tbody",{},[245,260,261,266],{},[262,263,264],"td",{"align":250},[31,265,155],{},[262,267,268],{"align":254},"0.0684",[245,270,271,276],{},[262,272,273],{"align":250},[31,274,275],{},"delivery_days",[262,277,278],{"align":254},"0.0571",[245,280,281,285],{},[262,282,283],{"align":250},[31,284,135],{},[262,286,287],{"align":254},"0.0077",[245,289,290,294],{},[262,291,292],{"align":250},[31,293,132],{},[262,295,296],{"align":254},"0.0074",[245,298,299,304],{},[262,300,301],{"align":250},[31,302,303],{},"payment_installments",[262,305,306],{"align":254},"0.0026",[11,308,309,311,312,314],{},[31,310,155],{}," and ",[31,313,275],{}," dominate, almost 10 times more informative than price or freight (95,829 orders used). Confirms, with a different technique, exactly what Chapter 2's correlation already pointed at.",[23,316,318],{"id":317},"scenario-3-freight-value","Scenario 3: freight value",[11,320,321],{},"Candidates: weight and the product's three dimensions. Plain Pearson correlation is enough here (98,650 orders used):",[239,323,324,335],{},[242,325,326],{},[245,327,328,330],{},[248,329,251],{"align":250},[248,331,332,333],{"align":254},"Correlation with ",[31,334,135],{},[257,336,337,346,356,366],{},[245,338,339,343],{},[262,340,341],{"align":250},[31,342,129],{},[262,344,345],{"align":254},"0.615",[245,347,348,353],{},[262,349,350],{"align":250},[31,351,352],{},"product_height_cm",[262,354,355],{"align":254},"0.393",[245,357,358,363],{},[262,359,360],{"align":250},[31,361,362],{},"product_width_cm",[262,364,365],{"align":254},"0.331",[245,367,368,373],{},[262,369,370],{"align":250},[31,371,372],{},"product_length_cm",[262,374,375],{"align":254},"0.317",[11,377,378],{},"Weight moderately dominates over any single dimension, which makes physical sense: carriers charge by volumetric weight, but actual weight pulls the bill harder than any one side of the box alone.",[23,380,382],{"id":381},"scenario-4-customer-segmentation-rfm","Scenario 4: customer segmentation (RFM)",[11,384,385],{},"This isn't feature selection in the supervised sense, it's engineering the three variables that will feed Chapter 5's clustering: Recency, Frequency, and Monetary value per customer.",[36,387,389],{"className":38,"code":388,"language":40,"meta":41,"style":41},"rfm = oc.groupby('customer_unique_id').agg(\n    recencia_dias=('order_purchase_timestamp', lambda x: (data_max - x.max()).days),\n    frequencia=('order_id', 'nunique'),\n    valor_monetario=('payment_value', 'sum'),\n).reset_index()\n",[31,390,391,396,401,406,411],{"__ignoreMap":41},[45,392,393],{"class":47,"line":48},[45,394,395],{},"rfm = oc.groupby('customer_unique_id').agg(\n",[45,397,398],{"class":47,"line":54},[45,399,400],{},"    recencia_dias=('order_purchase_timestamp', lambda x: (data_max - x.max()).days),\n",[45,402,403],{"class":47,"line":61},[45,404,405],{},"    frequencia=('order_id', 'nunique'),\n",[45,407,408],{"class":47,"line":67},[45,409,410],{},"    valor_monetario=('payment_value', 'sum'),\n",[45,412,413],{"class":47,"line":73},[45,414,415],{},").reset_index()\n",[11,417,418,419,422],{},"95,560 unique customers. And the number that stood out the most: ",[184,420,421],{},"only 2,924 customers (3.06% of the total) placed more than one order",". Average frequency is 1.034, meaning the entire marketplace is dominated by one-time purchases. That completely changes how to think about segmentation in Chapter 5: there won't be a large \"loyal customer\" category, RFM here will likely separate more by amount spent and recency than by purchase frequency.",[23,424,426],{"id":425},"wrapping-up-the-chapter","Wrapping up the chapter",[11,428,429],{},"Customer-seller distance computed (and a real merge bug fixed along the way), data leakage demonstrated in practice (the gap between honest AUC 0.74 and leaky AUC 1.0 is the most concrete lesson I can give on the subject), and each scenario's features picked with a technique suited to its problem type. Seasonality turned out to matter more than distance for delay, delay dominates the review score, weight dominates freight, and the customer base is overwhelmingly one-time buyers. Next chapter I train the real models for each scenario, with experiment tracking and metric comparison.",[431,432,433],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":41,"searchDepth":54,"depth":54,"links":435},[436,437,438,439,440,441],{"id":25,"depth":54,"text":26},{"id":118,"depth":54,"text":119},{"id":218,"depth":54,"text":219},{"id":317,"depth":54,"text":318},{"id":381,"depth":54,"text":382},{"id":425,"depth":54,"text":426},"2026-08-24","Third chapter of the Olist case study: feature engineering and selection for the four scenarios, with a live demonstration of data leakage (a model with 100% AUC that's useless) and a real discovery about seasonality.","md",{},"\u002Fen\u002Fprojects\u002Folist-ecommerce\u002F03-features-e-selecao","olist-ecommerce",{"title":6,"description":443},"published","en\u002Fprojects\u002Folist-ecommerce\u002F03-features-e-selecao",[452,453,454],"scikit-learn","feature-engineering","data-leakage","hjJ-zgsW6213GQNSY5CTAs34DrjJoa-kQKoLaduH66U",1787605216032]