Integrating vulnerability scanners with defect tracking systems is how Vulnerability Manager got its start. A HUGE problem we see with so many groups doing security testing (in-house or external) is that assessments/penetraiton tests/code scans are run resulting in the identification of a number of vulnerabilities. These vulnerabilities are then stuffed into a 200+ page PDF report (complete with color graphs!) and that report is handed off to the development team. If the secuity folks are particularly conscientious they put sticky notes on the pages with the vulnerabilities they really want to see fixed.
Is it any wonder developers react in a defensive way?
Our first solution to this problem was to make each security vulnerability a software defect. That way we can communicate with developers “on their level” Nearly everyone at Denim Group is a software developer by background, so, regardless of whether they are doing software development, penetration tests, or code reviews, they think in terms of software bugs (defects).
So a naïve approach would be to create a software defect for each vulnerability. That way you could track the defects as they were addressed and make sure all of them had been resolved. However, let’s look at an example of the 1:1 approach to translating security vulnerabilities into software defects… If you find 100 XSS vulnerabilities in an application and translate those into 100 defects in your software defect tracking system, one of three things will happen:
1.The quality assurance (QA) manager will kill you
2.The development lead will kill you
3.The developers will simply ignore you
This highlights the challenge we ran into. When we did application assessments we would find a ton of security vulnerabilities, but those vulnerabilities needed to be communicated to development teams as a more concise and sensible set of software defects. Otherwise you will just aggravate them. Sending developers 200+ page PDFs is a non-starter. Sending them hundreds of new vulnerability/defects is also a non-starter. However, packaging multiple vulnerabilities into a single defect seems to work for both security folks and developers.
In the Vulnerability Manager, now that we have all of our vulnerability data centralized and consolidated, we can do just that. We support two operations with the defect tracking systems:
1.Create a new defect
2.Check the status of a defect
That way we can bundle vulnerabilities together to create a defect that appears directly in the defect tracking system. And over time we can watch to see which defects (and hence vulnerabilities) have ostensibly been addressed by the development teams so they can be re-tested to validate the fix.
Currently the bundling is done manually – basically you pick the vulnerabilities you want to use to create the defect and check checkboxes. This is all right, but can get cumbersome. We are going to be adding the capability to group vulnerabilities for an application by combinations of type, severity, directory and file because those are the ways we have found to be most useful. Check out the slide deck below for the pros and cons of different grouping approaches.
Also right now we are just providing information about the location of the specific vulnerabilities in the text of the defect – one or more AttackSurfaceLocations or CodeLocations, depending on how much information we have about the vulnerability. The goal is to provide as much information as possible to the developer so they can track down the vulnerability quickly and fix it. In the future we will also be including type-specific remediation recommendations in the defect text and this should also be customizable to the organization using Vulnerability Manager so it will communicate your standards for fixes to the developers.
For more information on where we are coming from, you can check out the slides from a presentation I did at the 2010 SANS Application Security Summit titled “Treating Security Vulnerabilities as Software Defects”
Contact us for more information about treating security vulnerabilities as software defects.
dan _at_ denimgroup.com