Spark-based port and net scan detection

Network security is more and more important today. Port and net scan are the typical preliminary steps an attacker makes to find victims in a certain network, and are currently the most spread network anomalies. In this work, we focus on traditional approaches for port and net scan detection, previously abandoned due to their limited speed, and we use Big Data Analytics to speed them up and cope with current high-speed networks. The paper describes our approach and presents an experimental analysis in terms of detection performance and execution time of a threshold-based algorithm on Apache Spark. We use real traffic traces from MAWI archive and MAWILab anomaly detectors to compare with our results. The analysis shows that i) the threshold-based algorithm is already able to achieve detection performance higher than MAWILab (in 95% of the considered cases with the best threshold value), currently considered the gold standard in the field; ii) the execution time can be as low as 25 seconds for a 24h traffic trace collected over a 10Gbps link, which makes it usable also in real time. Moreover, we bridge an important gap in literature providing the research community with a new labeled dataset, validated using MAWILab and extended with other anomalies not detected by it.