Intrusion Detection: Hunting Attacks in 2.5 Million Network Sessions

This project is different even from Olist. I built Olist chapter by chapter, with Professor Boldt around only in the shape of the course he taught. This one is literally my final project for the Pattern Recognition course, the closing exam for everything the rest of this playlist taught: EDA on messy, imbalanced data, preprocessing with a ColumnTransformer, honest model comparison, evaluation beyond accuracy, GridSearch. Except this time there's no professor showing the way, it was me alone with the dataset and Bishop by my side.

The dataset is UNSW-NB15, created by the Australian Centre for Cyber Security (ACCS) by simulating real network traffic mixed with nine different attack families (from brute force to worms), captured from real packets in a lab environment. The task is the most classic one in network security: looking at a network session's characteristics (how long it lasted, how many bytes it exchanged, which protocol it used, and so on), decide whether it's normal traffic or an attack.

Unlike Olist, this isn't independent angles over the same master dataframe, it's an end-to-end flow across three chapters, each one assuming what the last already showed: chapter 1 is the full exploratory analysis on the raw files, chapter 2 switches datasets and picks the model, chapter 3 closes with the final evaluation, the ROC curve, and hyperparameter tuning. Start with chapter 1 here.