I was looking over some automated static analysis scan results at a client site a couple of weeks ago and was amazed to see that – in a couple-ten-thousand lines of applications code – they had managed not to introduce any vulnerabilities that the scanner could find. Impressive! Or was it?
Upon a little further inspection we noticed that the scanner had not been provided with any of the front-end web pages that pretty much make up all of the application’s significant attack surface. No attack surface means the scanner is not going to find any source functions. No source functions means it is pretty tough to find links between sources and sinks. The end result: no significant checking was being done for SQL injection or Cross-Site Scripting (XSS) and those are two of the bigger issues you would expect your automatic static analysis tools to find for you.
Automated scanning tools are great but they can only scan what you give them. Any scanning process must include sanity checks to make sure the scanners are getting proper coverage. Otherwise scan results will likely present a falsely rosy view of the security state of the application.
dan _at_ denimgroup.com