New Web Application Scanner Technology Guarantees NO False Positives

Recently I’ve been working with some other Denim Group folks to do our regular internal benchmarking of various application security scanners.  Last week we got into a deep discussion of false positives, how many scanners claim to reduce or eliminate them, and different techniques to make this happen.  During the talk we came across an idea for a surefire way to completely eliminate false positives and this weekend I had enough free time to put together a proof of concept.  I’ve included the proof of concept code at the bottom of this post and you can consider it to be released under the Apache 2.0 license.

This is revolutionary enough that we are currently trying to make testing with this technology required for Scanless PCI Certification and we want to make sure that all Certified Application Security Specialists are required to be conversant with it.  We’re considering shifting all our use of scaning tools to use our new technology.

Below is the proof of concept code:

package com.denimgroup.nofalsepositives;

 

import java.util.ArrayList;

import java.net.MalformedURLException;

import java.net.URL;

 

publicclass DynamicAnalyzer

{

  publicstaticvoid main(String[] args)

  {

    ArrayList vulnerabilities = new ArrayList();

          

    String sUrlToScan;

    URL urlToScan = null;

          

    long scanStart;

    long scanEnd;

          

    // Must at least enter a URL for the site to scan

    if(args.length < 1) {

      usage();

      System.exit(1);

    }

          

    // Make sure the URL is valid

          

    sUrlToScan = args[0];

    try {

      urlToScan = new URL(sUrlToScan);

    } catch (MalformedURLException e) {

      System.out.println(“Provided URL was invalid.  Unable to scan.”);

      System.exit(2);

    }

          

    // Kick off the scan

          

    scanStart = System.currentTimeMillis();

    System.out.println(String.format(“Starting scan of %s at %d”, urlToScan.toString(), scanStart));

          

    // Finalize scan and report findings

          

    scanEnd = System.currentTimeMillis();

    System.out.println(String.format(“Finished scan of %s at %d”, urlToScan.toString(), scanEnd));

 

    System.out.println(String.format(“Found %d vulnerabilities with NO false positives”, vulnerabilities.size()));

  }

    

  publicstaticvoid usage()

  {

    System.out.println(“usage: java com.denimgroup.nofalsepositives.DynamicAnalyzer <SITE_URL>”);

  }

}

In case anyone hasn’t figured it out by now this whole post is completely facetious.  Automated scanning has its place in any credible application security program, but no credible application security program consists only of automated scanning.  And you will always get false positives as well as results that need to be re-prioritized based on the business context of the vulnerability.  Automation is great, but you can’t automate everything.

Contact us if you would like more info on constructing your application security program.

dan _at_ denimgroup.com

@danielcornell

Posted via email from Denim Group’s Posterous

About Dan Cornell

A globally recognized application security expert, Dan Cornell holds over 15 years of experience architecting, developing and securing web-based software systems. As the Chief Technology Officer and a Principal at Denim Group, Ltd., he leads the technology team to help Fortune 500 companies and government organizations integrate security throughout the development process. He is also the original creator of ThreadFix, Denim Group's industry leading application vulnerability management platform.
More Posts by Dan Cornell

Categories: Uncategorized

3 Responses to “New Web Application Scanner Technology Guarantees NO False Positives”

  1. Robert Sullivan

    Good bit, and entertaining. Is Dan nibbling at the edges of a serious hypothesis that deserves consideration? “When the ratio of false-positives to positives goes below 50% it is time to get new tools.”

  2. mls

    Dan, I fear you may have been scooped:

    From http://www.mavitunasecurity.com/

    “So we developed a new technology which can confirm vulnerabilities on demand, which allowed us to develop the first false positive free web application security scanner.”

    However their claims are neither ironic nor spoofed. While there is enough wiggle room and caveats in the surrounding paragraphs for the Mavituna guys to make their case, the phrase “false positive free web application security scanner” should cause some consternation.

  3. Dan Cornell

    Robert:

    I suppose it depends what you are looking to use the automated scanner for. If you want it to be point-and-click then looking at the ratio of false positives to actual results makes sense. However if you are using it as an information gathering tool to focus follow-on manual efforts the ratio is less important than the time required to validate results and cull out the false positives.

    We are getting ready to release some technology we put together that lets you import results from multiple scanning tools and then merge them together. One item of interest is what sort of metrics we can track, and looking at false positive ratios per tool might be interesting.

    –Dan

Leave a Reply

Your email address will not be published. Required fields are marked *