[{"data":1,"prerenderedAt":1368},["ShallowReactive",2],{"lang-switch-post-\u002Fen\u002Fprojects\u002Fnetwork-intrusion-detection\u002F03-avaliacao-final-e-ajuste":3,"chapter-en-network-intrusion-detection-03-avaliacao-final-e-ajuste":4},null,{"id":5,"title":6,"body":7,"cover":3,"date":1353,"description":1354,"extension":1355,"meta":1356,"navigation":94,"order":67,"path":1357,"project":1358,"seo":1359,"status":1360,"stem":1361,"tags":1362,"__hash__":1367},"projectChapters\u002Fen\u002Fprojects\u002Fnetwork-intrusion-detection\u002F03-avaliacao-final-e-ajuste.md","Final Evaluation, ROC Curve, and Hyperparameter Tuning",{"type":8,"value":9,"toc":1344},"minimark",[10,24,29,42,102,129,136,201,216,220,236,251,393,409,464,485,491,494,500,504,511,522,536,554,561,666,672,733,762,769,776,780,788,822,881,888,912,916,929,1006,1045,1053,1067,1071,1078,1122,1163,1180,1183,1237,1257,1267,1275,1279,1332,1340],[11,12,13,18,19,23],"p",{},[14,15,17],"a",{"href":16},"\u002Fen\u002Fprojects\u002Fnetwork-intrusion-detection\u002F02-metodologia-selecao-de-modelo","In chapter 2"," I defined the criterion before seeing any numbers (F1 first, AUC as tiebreaker) and Random Forest won the comparison across four model families. This closing chapter trains it for real, on the entire official training set, measures it on the official test set (the 175,341 sessions the model never saw), and doesn't stop at accuracy: I open up the decision cutoff, the ROC curve, which variables weighed most, exactly where the model gets it wrong, and whether tuning hyperparameters can improve it. The full, executed notebook, covering all three parts of this project, is linked ",[14,20,22],{"href":21},"\u002Fen\u002Fprojects\u002Fnetwork-intrusion-detection\u002F01-explorando-25-milhoes-de-sessoes","at the top of chapter 1",".",[25,26,28],"h2",{"id":27},"final-evaluation-random-forest-on-the-real-official-test-set","Final evaluation: Random Forest, on the real official test set",[11,30,31,32,36,37,41],{},"I trained the Random Forest (200 trees, with ",[33,34,35],"code",{},"class_weight=\"balanced\"",", a way of telling the model that missing an attack, the rare class, costs more than missing a normal session, without needing to duplicate or drop a single row ",[14,38,40],{"href":39},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fcredit-card-fraud","the way resampling tried to do back with credit card fraud",") on the entire official training set and measured it on the official test set, the 175,341 records the model never saw:",[43,44,49],"pre",{"className":45,"code":46,"language":47,"meta":48,"style":48},"language-python shiki shiki-themes github-light github-dark","modelo_final = Pipeline([\n    (\"prep\", preprocessador),\n    (\"clf\", RandomForestClassifier(n_estimators=200, random_state=42, class_weight=\"balanced\", n_jobs=-1)),\n])\nmodelo_final.fit(X_train, y_train)\ny_pred = modelo_final.predict(X_test)\n\nprint(classification_report(y_test, y_pred, digits=4))\n","python","",[33,50,51,59,65,71,77,83,89,96],{"__ignoreMap":48},[52,53,56],"span",{"class":54,"line":55},"line",1,[52,57,58],{},"modelo_final = Pipeline([\n",[52,60,62],{"class":54,"line":61},2,[52,63,64],{},"    (\"prep\", preprocessador),\n",[52,66,68],{"class":54,"line":67},3,[52,69,70],{},"    (\"clf\", RandomForestClassifier(n_estimators=200, random_state=42, class_weight=\"balanced\", n_jobs=-1)),\n",[52,72,74],{"class":54,"line":73},4,[52,75,76],{},"])\n",[52,78,80],{"class":54,"line":79},5,[52,81,82],{},"modelo_final.fit(X_train, y_train)\n",[52,84,86],{"class":54,"line":85},6,[52,87,88],{},"y_pred = modelo_final.predict(X_test)\n",[52,90,92],{"class":54,"line":91},7,[52,93,95],{"emptyLinePlaceholder":94},true,"\n",[52,97,99],{"class":54,"line":98},8,[52,100,101],{},"print(classification_report(y_test, y_pred, digits=4))\n",[11,103,104,105,108,109,112,113,116,117,120,121,124,125,128],{},"The same ",[33,106,107],{},"Pipeline"," as always, now with the entire ",[33,110,111],{},"preprocessador"," (the 39 ",[33,114,115],{},"StandardScaler"," plus 3 ",[33,118,119],{},"OneHotEncoder",") glued in front of the final Random Forest, a single ",[33,122,123],{},".fit()"," training both stages together. ",[33,126,127],{},"classification_report"," is what computes precision, recall, and F1 per class in one shot, the same table right below.",[130,131],"heatmap-chart",{":cells":132,":x-categories":133,":y-categories":133,"x-label":134,"y-label":135},"[{\"x\":\"Normal\",\"y\":\"Normal\",\"value\":54651},{\"x\":\"Attack\",\"y\":\"Normal\",\"value\":1349},{\"x\":\"Normal\",\"y\":\"Attack\",\"value\":15429},{\"x\":\"Attack\",\"y\":\"Attack\",\"value\":103912}]","[\"Normal\",\"Attack\"]","predicted","actual",[137,138,139,163],"table",{},[140,141,142],"thead",{},[143,144,145,150,154,157,160],"tr",{},[146,147,149],"th",{"align":148},"left","Class",[146,151,153],{"align":152},"right","Precision",[146,155,156],{"align":152},"Recall",[146,158,159],{"align":152},"F1",[146,161,162],{"align":152},"Sessions in test",[164,165,166,184],"tbody",{},[143,167,168,172,175,178,181],{},[169,170,171],"td",{"align":148},"Normal",[169,173,174],{"align":152},"0.7798",[169,176,177],{"align":152},"0.9759",[169,179,180],{"align":152},"0.8669",[169,182,183],{"align":152},"56,000",[143,185,186,189,192,195,198],{},[169,187,188],{"align":148},"Attack",[169,190,191],{"align":152},"0.9872",[169,193,194],{"align":152},"0.8707",[169,196,197],{"align":152},"0.9253",[169,199,200],{"align":152},"119,341",[11,202,203,204,208,209,212,213,23],{},"Precision here is: of the sessions the model flagged as attack, how many really were. Recall is the opposite: of the real attacks, how many the model caught. F1 folds both into one score. Overall accuracy is 90.43%, but the number I actually trust is the attack class's recall: ",[205,206,207],"strong",{},"87.07%",", meaning the model catches 103,912 of the 119,341 real attacks in the test set, and lets 15,429 slip through. In security, this kind of false negative (an attack classified as normal) is the expensive mistake, it's literally the intruder who walks past unnoticed. On the other side, 1,349 normal sessions turned into false alarms, annoying for whoever investigates them, but much less serious than letting a real attack through. ",[205,210,211],{},"AUC on the official test set lands at 0.9830",", close to the 1.0 ceiling, a sign the model knows how to rank sessions by risk well even before any decision threshold enters the picture, ",[14,214,215],{"href":39},"the same ranking metric I already saw with credit card fraud",[25,217,219],{"id":218},"dialing-the-cutoff-precision-against-recall","Dialing the cutoff: precision against recall",[11,221,222,223,226,227,230,231,235],{},"All that evaluation used scikit-learn's default cutoff: ",[33,224,225],{},".predict()"," decides \"attack\" whenever the risk score (the fraction of trees that voted attack, the same number you can get from ",[33,228,229],{},".predict_proba()",") crosses 0.5. But that 0.5 is just one arbitrary point on a whole line of options, the same threshold idea ",[14,232,234],{"href":233},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fclassification-threshold","I already saw in the classification-with-regression lesson",", except now you can actually turn the dial.",[11,237,238,239,242,243,246,247,250],{},"For that, I wrote a second notebook (",[33,240,241],{},"datasets\u002Funsw-nb15\u002F02_dados_extras_scatter_e_threshold.ipynb","), only because the original notebook never saved the continuous ",[33,244,245],{},"y_score",", just the ",[33,248,249],{},"y_pred"," already thresholded at 0.5. This notebook's first cells rebuild, cell by cell, exactly the same pipeline from chapters 2 and 3:",[43,252,254],{"className":45,"code":253,"language":47,"meta":48,"style":48},"df_train = pd.read_csv(\"\u002Fcontent\u002Fdata\u002FUNSW_NB15_training-set.csv\")\ndf_test = pd.read_csv(\"\u002Fcontent\u002Fdata\u002FUNSW_NB15_testing-set.csv\")\n\nDESCARTAR = [\"id\", \"attack_cat\", \"label\"]\nX_train = df_train.drop(columns=[c for c in DESCARTAR if c in df_train.columns])\ny_train = df_train[\"label\"]\nX_test = df_test.drop(columns=[c for c in DESCARTAR if c in df_test.columns])\ny_test = df_test[\"label\"]\n\ncat_cols = X_train.select_dtypes(include=[\"object\"]).columns.tolist()\nnum_cols = X_train.select_dtypes(include=[\"number\"]).columns.tolist()\n\ntransformadores = [\n    (\"num\", StandardScaler(), num_cols),\n    (\"cat\", OneHotEncoder(handle_unknown=\"ignore\", sparse_output=False), cat_cols),\n]\npreprocessador = ColumnTransformer(transformers=transformadores)\n\nmodelo_final = Pipeline([\n    (\"prep\", preprocessador),\n    (\"clf\", RandomForestClassifier(n_estimators=200, random_state=42, class_weight=\"balanced\", n_jobs=-1)),\n])\nmodelo_final.fit(X_train, y_train)\n\ny_pred = modelo_final.predict(X_test)\ny_score = modelo_final.predict_proba(X_test)[:, 1]\n",[33,255,256,261,266,270,275,280,285,290,295,300,306,312,317,323,329,335,341,347,352,357,362,367,372,377,382,387],{"__ignoreMap":48},[52,257,258],{"class":54,"line":55},[52,259,260],{},"df_train = pd.read_csv(\"\u002Fcontent\u002Fdata\u002FUNSW_NB15_training-set.csv\")\n",[52,262,263],{"class":54,"line":61},[52,264,265],{},"df_test = pd.read_csv(\"\u002Fcontent\u002Fdata\u002FUNSW_NB15_testing-set.csv\")\n",[52,267,268],{"class":54,"line":67},[52,269,95],{"emptyLinePlaceholder":94},[52,271,272],{"class":54,"line":73},[52,273,274],{},"DESCARTAR = [\"id\", \"attack_cat\", \"label\"]\n",[52,276,277],{"class":54,"line":79},[52,278,279],{},"X_train = df_train.drop(columns=[c for c in DESCARTAR if c in df_train.columns])\n",[52,281,282],{"class":54,"line":85},[52,283,284],{},"y_train = df_train[\"label\"]\n",[52,286,287],{"class":54,"line":91},[52,288,289],{},"X_test = df_test.drop(columns=[c for c in DESCARTAR if c in df_test.columns])\n",[52,291,292],{"class":54,"line":98},[52,293,294],{},"y_test = df_test[\"label\"]\n",[52,296,298],{"class":54,"line":297},9,[52,299,95],{"emptyLinePlaceholder":94},[52,301,303],{"class":54,"line":302},10,[52,304,305],{},"cat_cols = X_train.select_dtypes(include=[\"object\"]).columns.tolist()\n",[52,307,309],{"class":54,"line":308},11,[52,310,311],{},"num_cols = X_train.select_dtypes(include=[\"number\"]).columns.tolist()\n",[52,313,315],{"class":54,"line":314},12,[52,316,95],{"emptyLinePlaceholder":94},[52,318,320],{"class":54,"line":319},13,[52,321,322],{},"transformadores = [\n",[52,324,326],{"class":54,"line":325},14,[52,327,328],{},"    (\"num\", StandardScaler(), num_cols),\n",[52,330,332],{"class":54,"line":331},15,[52,333,334],{},"    (\"cat\", OneHotEncoder(handle_unknown=\"ignore\", sparse_output=False), cat_cols),\n",[52,336,338],{"class":54,"line":337},16,[52,339,340],{},"]\n",[52,342,344],{"class":54,"line":343},17,[52,345,346],{},"preprocessador = ColumnTransformer(transformers=transformadores)\n",[52,348,350],{"class":54,"line":349},18,[52,351,95],{"emptyLinePlaceholder":94},[52,353,355],{"class":54,"line":354},19,[52,356,58],{},[52,358,360],{"class":54,"line":359},20,[52,361,64],{},[52,363,365],{"class":54,"line":364},21,[52,366,70],{},[52,368,370],{"class":54,"line":369},22,[52,371,76],{},[52,373,375],{"class":54,"line":374},23,[52,376,82],{},[52,378,380],{"class":54,"line":379},24,[52,381,95],{"emptyLinePlaceholder":94},[52,383,385],{"class":54,"line":384},25,[52,386,88],{},[52,388,390],{"class":54,"line":389},26,[52,391,392],{},"y_score = modelo_final.predict_proba(X_test)[:, 1]\n",[11,394,395,396,399,400,402,403,405,406,408],{},"Cell by cell, this is the same code from chapter 2 (the official partition, the ",[33,397,398],{},"ColumnTransformer",") and from the start of this chapter (the ",[33,401,107],{}," with the 200-tree Random Forest), no new line anywhere, just rebuilt here because this notebook runs isolated, in a Colab runtime that doesn't carry over the other two notebooks' state. The only difference is the last line: besides ",[33,404,249],{},", I also keep ",[33,407,245],{},", the continuous probability, so I can test cutoffs other than 0.5 in the next block. With the pipeline retrained, the new block sweeps 19 different cutoffs:",[43,410,412],{"className":45,"code":411,"language":47,"meta":48,"style":48},"thresholds = np.round(np.arange(0.05, 1.0, 0.05), 2)\nthreshold_sweep = []\nfor t in thresholds:\n    y_pred_t = (y_score >= t).astype(int)\n    threshold_sweep.append({\n        \"threshold\": float(t),\n        \"precision\": round(float(precision_score(y_test, y_pred_t, zero_division=0)), 4),\n        \"recall\": round(float(recall_score(y_test, y_pred_t, zero_division=0)), 4),\n        \"f1\": round(float(f1_score(y_test, y_pred_t, zero_division=0)), 4),\n    })\n",[33,413,414,419,424,429,434,439,444,449,454,459],{"__ignoreMap":48},[52,415,416],{"class":54,"line":55},[52,417,418],{},"thresholds = np.round(np.arange(0.05, 1.0, 0.05), 2)\n",[52,420,421],{"class":54,"line":61},[52,422,423],{},"threshold_sweep = []\n",[52,425,426],{"class":54,"line":67},[52,427,428],{},"for t in thresholds:\n",[52,430,431],{"class":54,"line":73},[52,432,433],{},"    y_pred_t = (y_score >= t).astype(int)\n",[52,435,436],{"class":54,"line":79},[52,437,438],{},"    threshold_sweep.append({\n",[52,440,441],{"class":54,"line":85},[52,442,443],{},"        \"threshold\": float(t),\n",[52,445,446],{"class":54,"line":91},[52,447,448],{},"        \"precision\": round(float(precision_score(y_test, y_pred_t, zero_division=0)), 4),\n",[52,450,451],{"class":54,"line":98},[52,452,453],{},"        \"recall\": round(float(recall_score(y_test, y_pred_t, zero_division=0)), 4),\n",[52,455,456],{"class":54,"line":297},[52,457,458],{},"        \"f1\": round(float(f1_score(y_test, y_pred_t, zero_division=0)), 4),\n",[52,460,461],{"class":54,"line":302},[52,462,463],{},"    })\n",[11,465,466,469,470,473,474,476,477,480,481,484],{},[33,467,468],{},"np.arange(0.05, 1.0, 0.05)"," generates the 19 cutoffs (0.05, 0.10, ..., 0.95), and the loop tests each one: ",[33,471,472],{},"y_score >= t"," turns the continuous score into a binary label at that specific cutoff (the same comparison ",[33,475,225],{}," does internally, except fixing ",[33,478,479],{},"t"," instead of always using 0.5), and all three metrics get recomputed from scratch on every pass. ",[33,482,483],{},"zero_division=0"," avoids an error in case some extreme cutoff zeroes out an entire class. The result, 19 rows of precision\u002Frecall\u002FF1, is what feeds the interactive chart below:",[486,487],"intrusion-threshold-explorer",{"precision-label":488,"recall-label":489,"threshold-label":490},"precision","recall","model cutoff",[11,492,493],{},"Drag the slider and watch the trade. Dropping the cutoff to 0.05 (flag \"attack\" on any suspicion at all, however small) pushes recall to 0.9781, catching almost everyone, but drags precision down to 0.9234, more false alarms for the security team to chase for nothing. Pushing the cutoff up to 0.95 (only flag \"attack\" when the model is nearly certain) flips it: precision of 0.999, almost no false alarms, but recall collapses to 0.6951, letting nearly a third of real attacks through. There's no single right cutoff, separate from the business question. If chasing a false alarm is cheap, it's worth pulling the cutoff down and eating more false alarms in exchange for catching more real attacks. If every investigation is expensive, the cutoff goes up.",[11,495,496,497,499],{},"At the cutoff I used through the whole post (0.5), pairing ",[33,498,35],{}," with that default cutoff already delivers 0.8715 recall without me touching anything by hand (the numbers match, off by a thousandth, the table further up, since I rebuilt the pipeline from scratch in this separate notebook). That's why I never went chasing a better cutoff for the final evaluation: the class imbalance was already being compensated for during training itself, not just at decision time.",[25,501,503],{"id":502},"the-roc-curve-the-same-cutoff-seen-from-another-angle","The ROC curve: the same cutoff, seen from another angle",[11,505,506,507,510],{},"The confusion matrix and the precision\u002Frecall table above evaluate the model at a ",[205,508,509],{},"single"," decision cutoff (the default, 0.5). But the slider I just showed already gives away the rest of the picture: the Random Forest doesn't just return a label, it estimates a probability P(attack | x) for every session, and the cutoff is an operational security decision, not a fixed mathematical truth.",[11,512,513,514,517,518,521],{},"The ",[205,515,516],{},"ROC curve"," (Receiver Operating Characteristic) turns that trade into a picture: sweeping across every possible cutoff, it plots the true positive rate (the same recall as always) against the false positive rate (the fraction of normal sessions that turned into false alarms). ",[205,519,520],{},"AUC"," (Area Under the Curve, the area under that plot) condenses the whole chart into a single number, with a direct probabilistic reading: it's the chance the model gives a higher score to a randomly picked attack than to a randomly picked normal session. AUC = 1.0 is the perfect classifier, AUC = 0.5 is the same as a coin flip (the chart's dotted diagonal). In the original notebook, that computation is just two lines:",[43,523,525],{"className":45,"code":524,"language":47,"meta":48,"style":48},"y_score = modelo_final.predict_proba(X_test)[:, 1]\nauc = roc_auc_score(y_test, y_score)\n",[33,526,527,531],{"__ignoreMap":48},[52,528,529],{"class":54,"line":55},[52,530,392],{},[52,532,533],{"class":54,"line":61},[52,534,535],{},"auc = roc_auc_score(y_test, y_score)\n",[11,537,538,541,542,545,546,549,550,553],{},[33,539,540],{},"predict_proba"," returns two columns, the probability of each class, ",[33,543,544],{},"[:, 1]"," grabs just class 1's column (attack). ",[33,547,548],{},"roc_auc_score"," takes the true label and that continuous score, no fixed cutoff involved at all, and computes AUC directly. The original notebook stopped there and only drew the chart with scikit-learn's own ",[33,551,552],{},"RocCurveDisplay.from_predictions(...)",", a static image, never saving the curve's points anywhere.",[11,555,556,557,560],{},"To get a real, interactive chart here on the blog, I wrote a third notebook (",[33,558,559],{},"datasets\u002Funsw-nb15\u002F03_curva_roc.ipynb","), with the same rebuild as always at the start (third time this block shows up in this project by now, so I'll just repeat it, no re-explaining):",[43,562,564],{"className":45,"code":563,"language":47,"meta":48,"style":48},"df_train = pd.read_csv(\"\u002Fcontent\u002Fdata\u002FUNSW_NB15_training-set.csv\")\ndf_test = pd.read_csv(\"\u002Fcontent\u002Fdata\u002FUNSW_NB15_testing-set.csv\")\n\nDESCARTAR = [\"id\", \"attack_cat\", \"label\"]\nX_train = df_train.drop(columns=[c for c in DESCARTAR if c in df_train.columns])\ny_train = df_train[\"label\"]\nX_test = df_test.drop(columns=[c for c in DESCARTAR if c in df_test.columns])\ny_test = df_test[\"label\"]\n\ncat_cols = X_train.select_dtypes(include=[\"object\"]).columns.tolist()\nnum_cols = X_train.select_dtypes(include=[\"number\"]).columns.tolist()\n\ntransformadores = [\n    (\"num\", StandardScaler(), num_cols),\n    (\"cat\", OneHotEncoder(handle_unknown=\"ignore\", sparse_output=False), cat_cols),\n]\npreprocessador = ColumnTransformer(transformers=transformadores)\n\nmodelo_final = Pipeline([\n    (\"prep\", preprocessador),\n    (\"clf\", RandomForestClassifier(n_estimators=200, random_state=42, class_weight=\"balanced\", n_jobs=-1)),\n])\nmodelo_final.fit(X_train, y_train)\n\ny_score = modelo_final.predict_proba(X_test)[:, 1]\n",[33,565,566,570,574,578,582,586,590,594,598,602,606,610,614,618,622,626,630,634,638,642,646,650,654,658,662],{"__ignoreMap":48},[52,567,568],{"class":54,"line":55},[52,569,260],{},[52,571,572],{"class":54,"line":61},[52,573,265],{},[52,575,576],{"class":54,"line":67},[52,577,95],{"emptyLinePlaceholder":94},[52,579,580],{"class":54,"line":73},[52,581,274],{},[52,583,584],{"class":54,"line":79},[52,585,279],{},[52,587,588],{"class":54,"line":85},[52,589,284],{},[52,591,592],{"class":54,"line":91},[52,593,289],{},[52,595,596],{"class":54,"line":98},[52,597,294],{},[52,599,600],{"class":54,"line":297},[52,601,95],{"emptyLinePlaceholder":94},[52,603,604],{"class":54,"line":302},[52,605,305],{},[52,607,608],{"class":54,"line":308},[52,609,311],{},[52,611,612],{"class":54,"line":314},[52,613,95],{"emptyLinePlaceholder":94},[52,615,616],{"class":54,"line":319},[52,617,322],{},[52,619,620],{"class":54,"line":325},[52,621,328],{},[52,623,624],{"class":54,"line":331},[52,625,334],{},[52,627,628],{"class":54,"line":337},[52,629,340],{},[52,631,632],{"class":54,"line":343},[52,633,346],{},[52,635,636],{"class":54,"line":349},[52,637,95],{"emptyLinePlaceholder":94},[52,639,640],{"class":54,"line":354},[52,641,58],{},[52,643,644],{"class":54,"line":359},[52,645,64],{},[52,647,648],{"class":54,"line":364},[52,649,70],{},[52,651,652],{"class":54,"line":369},[52,653,76],{},[52,655,656],{"class":54,"line":374},[52,657,82],{},[52,659,660],{"class":54,"line":379},[52,661,95],{"emptyLinePlaceholder":94},[52,663,664],{"class":54,"line":384},[52,665,392],{},[11,667,668,669,671],{},"And, on top of that ",[33,670,245],{},", extracts the curve point by point:",[43,673,675],{"className":45,"code":674,"language":47,"meta":48,"style":48},"fpr, tpr, thresholds = roc_curve(y_test, y_score)\nauc = roc_auc_score(y_test, y_score)\nprint(f\"AUC (conferencia, deve bater com 0.9830 ja publicado): {auc:.4f}\")\nprint(f\"Curva bruta tem {len(fpr)} pontos, vamos reduzir pra 50 pra plotar no blog\")\n\nfpr_grid = np.linspace(0, 1, 50)\ntpr_grid = np.interp(fpr_grid, fpr, tpr)\n\nroc_curve_points = [\n    {\"fpr\": round(float(f), 4), \"tpr\": round(float(t), 4)}\n    for f, t in zip(fpr_grid, tpr_grid)\n]\n",[33,676,677,682,686,691,696,700,705,710,714,719,724,729],{"__ignoreMap":48},[52,678,679],{"class":54,"line":55},[52,680,681],{},"fpr, tpr, thresholds = roc_curve(y_test, y_score)\n",[52,683,684],{"class":54,"line":61},[52,685,535],{},[52,687,688],{"class":54,"line":67},[52,689,690],{},"print(f\"AUC (conferencia, deve bater com 0.9830 ja publicado): {auc:.4f}\")\n",[52,692,693],{"class":54,"line":73},[52,694,695],{},"print(f\"Curva bruta tem {len(fpr)} pontos, vamos reduzir pra 50 pra plotar no blog\")\n",[52,697,698],{"class":54,"line":79},[52,699,95],{"emptyLinePlaceholder":94},[52,701,702],{"class":54,"line":85},[52,703,704],{},"fpr_grid = np.linspace(0, 1, 50)\n",[52,706,707],{"class":54,"line":91},[52,708,709],{},"tpr_grid = np.interp(fpr_grid, fpr, tpr)\n",[52,711,712],{"class":54,"line":98},[52,713,95],{"emptyLinePlaceholder":94},[52,715,716],{"class":54,"line":297},[52,717,718],{},"roc_curve_points = [\n",[52,720,721],{"class":54,"line":302},[52,722,723],{},"    {\"fpr\": round(float(f), 4), \"tpr\": round(float(t), 4)}\n",[52,725,726],{"class":54,"line":308},[52,727,728],{},"    for f, t in zip(fpr_grid, tpr_grid)\n",[52,730,731],{"class":54,"line":314},[52,732,340],{},[11,734,735,738,739,741,742,745,746,749,750,753,754,757,758,761],{},[33,736,737],{},"roc_curve(y_test, y_score)"," internally sweeps every unique score present in ",[33,740,245],{}," as a candidate cutoff, and returns three same-length arrays: ",[33,743,744],{},"fpr",", ",[33,747,748],{},"tpr",", and the ",[33,751,752],{},"thresholds"," that produced each pair. On this dataset the raw curve came out with 282 points, more granularity than a blog chart needs and a bigger JSON than necessary. ",[33,755,756],{},"np.linspace(0, 1, 50)"," creates 50 evenly spaced positions on the false-positive axis, and ",[33,759,760],{},"np.interp(fpr_grid, fpr, tpr)"," estimates the real true-positive rate at each of those 50 positions, interpolating between the nearest real points, nothing invented: it's the same curve, just sampled on a grid that's lighter to draw. The confirmation print lands on AUC = 0.9830, matching the number computed in the original notebook exactly.",[763,764],"intrusion-roc-chart",{"model-label":765,"random-label":766,"x-label":767,"y-label":768},"Random Forest (AUC = 0.983)","random (AUC = 0.5)","false positive rate","true positive rate",[11,770,771,772,775],{},"The curve hugs the top-left corner, far from the dotted diagonal of chance: at a false-positive cutoff of just 2%, recall already jumps to 86%, and the curve stays nearly glued to the ceiling from there on. That shows the model separates the two classes well ",[205,773,774],{},"independent of any specific cutoff",", and that's exactly the slack that lets me pull the cutoff down, like I did in the previous section, without precision collapsing along with it.",[25,777,779],{"id":778},"who-decides-the-15-variables-that-matter-most","Who decides: the 15 variables that matter most",[11,781,782,783,787],{},"Random Forest lets you peek at which variables most reduced impurity across the trees during training, ",[14,784,786],{"href":785},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fdecision-trees","the same Gini yardstick I already saw with decision trees",", just now summed across 200 trees instead of just one:",[43,789,791],{"className":45,"code":790,"language":47,"meta":48,"style":48},"ohe = modelo_final.named_steps[\"prep\"].named_transformers_[\"cat\"]\nnomes_cat = ohe.get_feature_names_out(cat_cols)\nnomes_features = np.concatenate([num_cols, nomes_cat])\n\nimportancias = modelo_final.named_steps[\"clf\"].feature_importances_\nimp = pd.Series(importancias, index=nomes_features).sort_values(ascending=False)\n",[33,792,793,798,803,808,812,817],{"__ignoreMap":48},[52,794,795],{"class":54,"line":55},[52,796,797],{},"ohe = modelo_final.named_steps[\"prep\"].named_transformers_[\"cat\"]\n",[52,799,800],{"class":54,"line":61},[52,801,802],{},"nomes_cat = ohe.get_feature_names_out(cat_cols)\n",[52,804,805],{"class":54,"line":67},[52,806,807],{},"nomes_features = np.concatenate([num_cols, nomes_cat])\n",[52,809,810],{"class":54,"line":73},[52,811,95],{"emptyLinePlaceholder":94},[52,813,814],{"class":54,"line":79},[52,815,816],{},"importancias = modelo_final.named_steps[\"clf\"].feature_importances_\n",[52,818,819],{"class":54,"line":85},[52,820,821],{},"imp = pd.Series(importancias, index=nomes_features).sort_values(ascending=False)\n",[11,823,824,825,828,829,831,832,745,835,745,838,841,842,745,845,745,848,851,852,855,856,859,860,862,863,866,867,870,871,873,874,876,877,880],{},"This block exists to solve an annoying problem: ",[33,826,827],{},"feature_importances_"," returns just a raw array of numbers, no name attached, and after the ",[33,830,398],{}," the three original categorical columns (",[33,833,834],{},"proto",[33,836,837],{},"service",[33,839,840],{},"state",") don't even exist anymore, they turned into dozens of new binary columns, like ",[33,843,844],{},"proto_tcp",[33,846,847],{},"proto_udp",[33,849,850],{},"service_dns",". To reconstruct \"which number in the list belongs to which variable,\" I need to go fetch those names back from inside the already-trained pipeline itself: ",[33,853,854],{},"modelo_final.named_steps[\"prep\"]"," grabs the preprocessing stage, ",[33,857,858],{},".named_transformers_[\"cat\"]"," grabs specifically the ",[33,861,119],{}," that ran on the categorical columns, and ",[33,864,865],{},".get_feature_names_out(cat_cols)"," asks it \"what were the names of the new columns you created?\" ",[33,868,869],{},"np.concatenate([num_cols, nomes_cat])"," glues that list back together with the 39 numeric ones up front (which kept their original names, ",[33,872,115],{}," never creates a new column), respecting the same order ",[33,875,398],{}," uses internally. Only after that does ",[33,878,879],{},"pd.Series(importancias, index=nomes_features)"," manage to pair every importance number with the right name.",[882,883],"category-bar-chart",{":points":884,"color":885,"x-label":886,"y-label":887},"[{\"label\":\"ct_dst_src_ltm\",\"value\":0.074},{\"label\":\"sttl\",\"value\":0.059},{\"label\":\"sbytes\",\"value\":0.054},{\"label\":\"sload\",\"value\":0.052},{\"label\":\"ct_dst_sport_ltm\",\"value\":0.051},{\"label\":\"smean\",\"value\":0.047},{\"label\":\"ct_srv_dst\",\"value\":0.044},{\"label\":\"ct_state_ttl\",\"value\":0.042},{\"label\":\"rate\",\"value\":0.033},{\"label\":\"dbytes\",\"value\":0.033},{\"label\":\"dload\",\"value\":0.031},{\"label\":\"dur\",\"value\":0.029},{\"label\":\"synack\",\"value\":0.028},{\"label\":\"ct_srv_src\",\"value\":0.027},{\"label\":\"dmean\",\"value\":0.026}]","#33aa55","variable","importance (impurity reduction)",[11,889,890,891,894,895,898,899,745,902,745,905,745,908,911],{},"The winner, ",[33,892,893],{},"ct_dst_src_ltm",", counts how many of the last 100 connections share the same source-destination pair, a direct way to flag scanning or flooding behavior: normal traffic rarely hits the exact same source-destination pair repeatedly, hundreds of times in a row, automated attacks do. Notice ",[33,896,897],{},"sttl"," (the TTL that already showed up in the correlation table) appears again, in second place, confirming that 0.91 correlation with the label wasn't empty statistical coincidence, it's a real signal. And a good chunk of the rest of the list (",[33,900,901],{},"ct_dst_sport_ltm",[33,903,904],{},"ct_srv_dst",[33,906,907],{},"ct_state_ttl",[33,909,910],{},"ct_srv_src",") belongs to the same \"how many recent connections look like this one\" family, the kind of variable specifically designed to catch a burst of repeated connections, the most common signature of automated attack versus human traffic.",[25,913,915],{"id":914},"where-the-model-gets-it-wrong-400-real-sessions-across-the-3-most-important-variables","Where the model gets it wrong: 400 real sessions across the 3 most important variables",[11,917,918,919,921,922,924,925,928],{},"The top two spots in the importance ranking, ",[33,920,893],{}," and ",[33,923,897],{},", plus the third, ",[33,926,927],{},"sbytes",", fit together in one space. It's the same notebook 02 that generated the threshold sweep, with the same retrained pipeline, that also extracts this sample:",[43,930,932],{"className":45,"code":931,"language":47,"meta":48,"style":48},"FEATURES_3D = [\"sttl\", \"ct_dst_src_ltm\", \"sbytes\"]\nN_SAMPLE = 400\n\namostra_idx = X_test.sample(n=N_SAMPLE, random_state=42).index\ny_pred_series = pd.Series(y_pred, index=X_test.index)\n\nsample_points = []\nfor i in amostra_idx:\n    sample_points.append({\n        \"x\": float(X_test.loc[i, FEATURES_3D[0]]),\n        \"y\": float(X_test.loc[i, FEATURES_3D[1]]),\n        \"z\": float(X_test.loc[i, FEATURES_3D[2]]),\n        \"label_real\": int(y_test.loc[i]),\n        \"label_previsto\": int(y_pred_series.loc[i]),\n    })\n",[33,933,934,939,944,948,953,958,962,967,972,977,982,987,992,997,1002],{"__ignoreMap":48},[52,935,936],{"class":54,"line":55},[52,937,938],{},"FEATURES_3D = [\"sttl\", \"ct_dst_src_ltm\", \"sbytes\"]\n",[52,940,941],{"class":54,"line":61},[52,942,943],{},"N_SAMPLE = 400\n",[52,945,946],{"class":54,"line":67},[52,947,95],{"emptyLinePlaceholder":94},[52,949,950],{"class":54,"line":73},[52,951,952],{},"amostra_idx = X_test.sample(n=N_SAMPLE, random_state=42).index\n",[52,954,955],{"class":54,"line":79},[52,956,957],{},"y_pred_series = pd.Series(y_pred, index=X_test.index)\n",[52,959,960],{"class":54,"line":85},[52,961,95],{"emptyLinePlaceholder":94},[52,963,964],{"class":54,"line":91},[52,965,966],{},"sample_points = []\n",[52,968,969],{"class":54,"line":98},[52,970,971],{},"for i in amostra_idx:\n",[52,973,974],{"class":54,"line":297},[52,975,976],{},"    sample_points.append({\n",[52,978,979],{"class":54,"line":302},[52,980,981],{},"        \"x\": float(X_test.loc[i, FEATURES_3D[0]]),\n",[52,983,984],{"class":54,"line":308},[52,985,986],{},"        \"y\": float(X_test.loc[i, FEATURES_3D[1]]),\n",[52,988,989],{"class":54,"line":314},[52,990,991],{},"        \"z\": float(X_test.loc[i, FEATURES_3D[2]]),\n",[52,993,994],{"class":54,"line":319},[52,995,996],{},"        \"label_real\": int(y_test.loc[i]),\n",[52,998,999],{"class":54,"line":325},[52,1000,1001],{},"        \"label_previsto\": int(y_pred_series.loc[i]),\n",[52,1003,1004],{"class":54,"line":331},[52,1005,463],{},[11,1007,1008,1011,1012,1015,1016,1018,1019,1022,1023,1026,1027,1030,1031,745,1033,745,1035,1037,1038,1040,1041,1044],{},[33,1009,1010],{},"X_test.sample(n=400, random_state=42)"," draws 400 indices from the official test set, the fixed seed is just so I can reproduce the same sample later. ",[33,1013,1014],{},"y_pred_series"," repackages the raw ",[33,1017,249],{}," array (which only has position, no index) into a ",[33,1020,1021],{},"Series"," sharing ",[33,1024,1025],{},"X_test","'s index, so a specific session's prediction can be looked up by ",[33,1028,1029],{},".loc[i]"," inside the loop. For each of the 400 sampled sessions, the loop keeps the three features of interest (",[33,1032,897],{},[33,1034,893],{},[33,1036,927],{},") plus the true label and the predicted one, nothing invented, it's exactly the official ",[33,1039,1025],{},"\u002F",[33,1042,1043],{},"y_test",", just narrowed down to a sample small enough to plot. I plotted them on those 3 axes:",[1046,1047],"intrusion-sample-scatter",{"false-negative-label":1048,"false-positive-label":1049,"true-negative-label":1050,"true-positive-label":1051,"x-label":1052,"y-label":893,"z-label":927},"false negative (missed attack)","false positive (false alarm)","correct: normal","correct: attack","sttl (source TTL)",[11,1054,1055,1056,1058,1059,1062,1063,1066],{},"In this sample of 400: 247 correct attack calls, 112 correct normal calls, 36 false negatives, and only 5 false positives, a ratio close to the full test set's. One thing that jumps out while spinning the cloud around: ",[33,1057,897],{}," isn't really continuous, it clusters into a handful of round values (31, 62, 254...), because TTL is literally a router-hop counter each operating system initializes at a default value (64, 128, or 255) and decrements by one at every hop. Among the 283 attack sessions in this sample, ",[33,1060,1061],{},"sttl=254"," shows up in 238 of them (84%), but it also shows up in 37 of the 117 normal sessions (32%), the most common value on both sides, far from a clean separator on its own. ",[33,1064,1065],{},"sttl=31",", on the other hand, is almost exclusively normal traffic: 74 of the 117 normal sessions, zero attacks. That's exactly why no single variable decides anything here. You can see the 36 red points (false negatives) scattered through the middle of the cloud, never forming an isolated island, and that's where the other 39 variables the chart doesn't show come in, working together to close the 87% recall the final model delivers.",[25,1068,1070],{"id":1069},"trying-to-improve-with-gridsearch-and-hitting-my-own-ceiling","Trying to improve with GridSearch, and hitting my own ceiling",[11,1072,1073,1074,1077],{},"I ran ",[33,1075,1076],{},"GridSearchCV"," varying the number of trees (200 or 400), max depth (unlimited or capped at 30), and minimum leaf size (1 or 2), 8 combinations, 3-fold cross-validation, on a sample of 30 thousand sessions to keep the cost manageable:",[43,1079,1081],{"className":45,"code":1080,"language":47,"meta":48,"style":48},"param_grid = {\n    \"clf__n_estimators\": [200, 400],\n    \"clf__max_depth\": [None, 30],\n    \"clf__min_samples_leaf\": [1, 2],\n}\n\ngrid = GridSearchCV(pipe_rf, param_grid, scoring=\"f1\", cv=cv, n_jobs=-1, verbose=1)\ngrid.fit(Xg, yg)\n",[33,1082,1083,1088,1093,1098,1103,1108,1112,1117],{"__ignoreMap":48},[52,1084,1085],{"class":54,"line":55},[52,1086,1087],{},"param_grid = {\n",[52,1089,1090],{"class":54,"line":61},[52,1091,1092],{},"    \"clf__n_estimators\": [200, 400],\n",[52,1094,1095],{"class":54,"line":67},[52,1096,1097],{},"    \"clf__max_depth\": [None, 30],\n",[52,1099,1100],{"class":54,"line":73},[52,1101,1102],{},"    \"clf__min_samples_leaf\": [1, 2],\n",[52,1104,1105],{"class":54,"line":79},[52,1106,1107],{},"}\n",[52,1109,1110],{"class":54,"line":85},[52,1111,95],{"emptyLinePlaceholder":94},[52,1113,1114],{"class":54,"line":91},[52,1115,1116],{},"grid = GridSearchCV(pipe_rf, param_grid, scoring=\"f1\", cv=cv, n_jobs=-1, verbose=1)\n",[52,1118,1119],{"class":54,"line":98},[52,1120,1121],{},"grid.fit(Xg, yg)\n",[11,1123,513,1124,1127,1128,1131,1132,1134,1135,1138,1139,1142,1143,1146,1147,1150,1151,1159,1160,1162],{},[33,1125,1126],{},"clf__"," prefix in front of each key isn't decoration: ",[33,1129,1130],{},"pipe_rf"," is a ",[33,1133,107],{}," with a stage named ",[33,1136,1137],{},"\"clf\""," (the Random Forest), and the ",[33,1140,1141],{},"__"," is scikit-learn's way of saying \"the ",[33,1144,1145],{},"n_estimators"," parameter belongs to the ",[33,1148,1149],{},"clf"," stage, not any other one in the pipeline,\" ",[14,1152,1154,1155,1158],{"href":1153},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fpipeline-cross-validation","the same ",[33,1156,1157],{},"model__"," syntax that already showed up in the pipeline and GridSearch post",". Without that prefix, ",[33,1161,1076],{}," wouldn't know which stage to touch.",[1164,1165,1166],"blockquote",{},[11,1167,1168,1171,1172,1175,1176,1179],{},[205,1169,1170],{},"Output:"," best combination found, ",[33,1173,1174],{},"n_estimators=200, max_depth=None, min_samples_leaf=1",", with a validation F1 of 0.9716. Exactly the ",[205,1177,1178],{},"default"," parameters of the Random Forest I'd already trained before.",[11,1181,1182],{},"To avoid blindly trusting that validation F1 (computed on a sample of 30 thousand, remember), I retrained the winning configuration on the full official training set and evaluated it on the official test set, the same rigorous way as the final evaluation:",[43,1184,1186],{"className":45,"code":1185,"language":47,"meta":48,"style":48},"melhores = {k.replace(\"clf__\", \"\"): v for k, v in grid.best_params_.items()}\nmodelo_tunado = Pipeline([(\"prep\", preprocessador),\n                          (\"clf\", RandomForestClassifier(random_state=42, class_weight=\"balanced\",\n                                                         n_jobs=-1, **melhores))])\nmodelo_tunado.fit(X_train, y_train)\ny_pred_t = modelo_tunado.predict(X_test)\ny_score_t = modelo_tunado.predict_proba(X_test)[:, 1]\n\nprint(classification_report(y_test, y_pred_t, digits=4))\nprint(f\"AUC-ROC (ajustada): {roc_auc_score(y_test, y_score_t):.4f}\")\n",[33,1187,1188,1193,1198,1203,1208,1213,1218,1223,1227,1232],{"__ignoreMap":48},[52,1189,1190],{"class":54,"line":55},[52,1191,1192],{},"melhores = {k.replace(\"clf__\", \"\"): v for k, v in grid.best_params_.items()}\n",[52,1194,1195],{"class":54,"line":61},[52,1196,1197],{},"modelo_tunado = Pipeline([(\"prep\", preprocessador),\n",[52,1199,1200],{"class":54,"line":67},[52,1201,1202],{},"                          (\"clf\", RandomForestClassifier(random_state=42, class_weight=\"balanced\",\n",[52,1204,1205],{"class":54,"line":73},[52,1206,1207],{},"                                                         n_jobs=-1, **melhores))])\n",[52,1209,1210],{"class":54,"line":79},[52,1211,1212],{},"modelo_tunado.fit(X_train, y_train)\n",[52,1214,1215],{"class":54,"line":85},[52,1216,1217],{},"y_pred_t = modelo_tunado.predict(X_test)\n",[52,1219,1220],{"class":54,"line":91},[52,1221,1222],{},"y_score_t = modelo_tunado.predict_proba(X_test)[:, 1]\n",[52,1224,1225],{"class":54,"line":98},[52,1226,95],{"emptyLinePlaceholder":94},[52,1228,1229],{"class":54,"line":297},[52,1230,1231],{},"print(classification_report(y_test, y_pred_t, digits=4))\n",[52,1233,1234],{"class":54,"line":302},[52,1235,1236],{},"print(f\"AUC-ROC (ajustada): {roc_auc_score(y_test, y_score_t):.4f}\")\n",[11,1238,1239,1242,1243,1245,1246,1249,1250,1253,1254,1256],{},[33,1240,1241],{},"grid.best_params_"," keeps its keys with the ",[33,1244,1126],{}," prefix (the same prefix explained above), so the first line strips that prefix so the values can be passed straight into ",[33,1247,1248],{},"RandomForestClassifier","'s constructor via ",[33,1251,1252],{},"**melhores",". The rest is the same ",[33,1255,107],{}," as always, just with the winning hyperparameters in place of the defaults, trained on the entire official training set (not the 30-thousand search sample anymore) and evaluated on the official test set.",[1164,1258,1259],{},[11,1260,1261,1263,1264,23],{},[205,1262,1170],{}," precision, recall, and F1 per class identical to the original Random Forest, decimal place for decimal place, and ",[33,1265,1266],{},"AUC-ROC (ajustada): 0.9830",[11,1268,1269,1270,1274],{},"The result matches, decimal place for decimal place, the original Random Forest: same precision, recall, and F1 per class, same 0.9830 AUC. It's not a search failure, it's the search confirming something worth saying plainly: for this dataset, with this set of candidates tested, scikit-learn's default guess (100 or 200 trees, unlimited depth) was already at the ceiling this hyperparameter family reaches. Not every optimization run finds something better to optimize toward, ",[14,1271,1273],{"href":1272},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Fensembles","the same kind of \"boring\" result I already saw in the ensembles post",", when plain logistic regression beat every ensemble no amount of tuning would have changed.",[25,1276,1278],{"id":1277},"wrapping-up-the-project-and-the-whole-playlist","Wrapping up the project (and the whole playlist)",[137,1280,1281,1291],{},[140,1282,1283],{},[143,1284,1285,1288],{},[146,1286,1287],{"align":148},"What I already knew, from an earlier post",[146,1289,1290],{"align":148},"What this final project tied together",[164,1292,1293,1301,1316,1324],{},[143,1294,1295,1298],{},[169,1296,1297],{"align":148},"Imbalanced data fools accuracy",[169,1299,1300],{"align":148},"Here, with 12.6% attack, the minority class's recall matters more than the overall number",[143,1302,1303,1306],{},[169,1304,1305],{"align":148},"High correlation signals redundancy",[169,1307,1308,1309,1311,1312,1315],{"align":148},"Correlation between a feature and the label (",[33,1310,897],{}," × ",[33,1313,1314],{},"Label",", 0.91) can signal real signal, not just redundancy between two variables",[143,1317,1318,1321],{},[169,1319,1320],{"align":148},"Random Forest handles nonlinear patterns well",[169,1322,1323],{"align":148},"Here it beat Logistic Regression, a lone Decision Tree, and Naive Bayes on both axes (F1 and AUC)",[143,1325,1326,1329],{},[169,1327,1328],{"align":148},"GridSearch finds the best point in the search space",[169,1330,1331],{"align":148},"Sometimes the best point already is the default, and it's worth finding that out by measuring, not assuming",[11,1333,1334,1335,1339],{},"This project closes a loop for me: I started this playlist ",[14,1336,1338],{"href":1337},"\u002Fen\u002Fplaylists\u002Fpattern-recognition\u002Flinear-regression-estimator","fitting a line with the normal equation",", went through trees, clustering, dimensionality reduction, text turning into vectors, and ended up applying all of it on my own, no lecture to follow, on a real network security problem, with a real result (0.98 AUC on the official test set) I can defend line by line, from the raw CSV all the way to the final decision tree. Thanks, Professor Boldt, I've got the rest from here.",[1341,1342,1343],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":48,"searchDepth":61,"depth":61,"links":1345},[1346,1347,1348,1349,1350,1351,1352],{"id":27,"depth":61,"text":28},{"id":218,"depth":61,"text":219},{"id":502,"depth":61,"text":503},{"id":778,"depth":61,"text":779},{"id":914,"depth":61,"text":915},{"id":1069,"depth":61,"text":1070},{"id":1277,"depth":61,"text":1278},"2026-08-26","Chapter 3, the closing chapter of my final project for Pattern Recognition: the Random Forest chosen in chapter 2 evaluated on the official test set (98.3% AUC), the decision cutoff explored end to end, the ROC curve, which variables weighed most, where the model gets it wrong, and finding out that hyperparameter tuning doesn't always beat the default guess.","md",{},"\u002Fen\u002Fprojects\u002Fnetwork-intrusion-detection\u002F03-avaliacao-final-e-ajuste","network-intrusion-detection",{"title":6,"description":1354},"published","en\u002Fprojects\u002Fnetwork-intrusion-detection\u002F03-avaliacao-final-e-ajuste",[1363,1364,1365,1366],"security","random-forest","unsw-nb15","anomaly-detection","cM7iP6Lojuo65771MsH9v17M-W_ORqpIi09OM6z00tw",1788965102334]