Automating Intuition: Digging for Gold in Network Data with Machine Learning

Virtual October 22, 2020 1:00 pm - 1:40 pm Feedback     

Bookmark and Share

Serge-Olivier Paquette

Intuition, acquired through years of experience, is what sets experts apart from novices. Intuition is the ability to look at a large amount of information, quickly spot interesting items, and dismiss the rest. In the case of security audits, intrusion testers typically face hundreds, or even thousands, of assets early in an engagement. Their ability to focus on priority targets can save dozens of valuable hours. Yet only the most experienced pentesters can do this confidently and effectively…those with intuition developed over years of accumulated experience.

In this talk I will demonstrate how to use effective and modern machine learning methods to sift through mountains of network data to quickly narrow down the scope to interesting, valuable, and sometimes odd targets: the gold nuggets. In short, I will be demonstrating a substitution of machine learning for human intuition! I will present Batea, an open-source tool using the Isolation Forest family of algorithms and a clever set of expert-engineered features to rank network devices in order of interest for an attacker. From understanding data sources (nmap scans) and their transformation into efficient representations for machine learning to the interpretation of algorithms and results, I will cover the whole life cycle of information from raw data to actionable insights. Attendees will gain an intuitive understanding of how some simple-yet-clever ideas and interesting techniques from the field of machine learning allow for actionable, day-to-day security improvements using only open source tools. The technique and tools presented can be readily implemented in any network security assessment or threat modelling program. All this without the need for advanced expertise and expensive tools.