BlackBerry, like many other companies, is on the move to containerized production environments. As containers become more ubiquitous, container security becomes crucial as well. Scanning container images for known vulnerabilities caused by vulnerable software is a critical security activity of the CI/CD process. Both commercial and open-source tools exist for container image scanning, however, not everything is peachy in the field. During Security Research Group’s journey into container security research to support BlackBerry’s containerization push we have encountered many cases of scanners’ unexplained behavior, unexpected errors, and results inconsistencies. We then dug deeper into scanners’ architecture and workflows to understand this behavior. To make the DevSecOps community aware of the possible pitfalls and intricacies I will share the results of our investigations and provide practical recommendations for choosing the best scanner for a specific environment. I will also present the Ultimate Benchmark for Container Image Scanning (UBCIS) – our open source tool for scanner evaluation. UBCIS, the tool used to choose the best open source scanner, is being presented at Usenix CSET 2020. The investigations into unexplained behaviour, unexpected errors, results inconsistencies and some of the practical recommendations are not covered by our Usenix CSET work and will be novel to SecTor.
