Many organizations use ThreadFix as the platform for running application security program – tracking their application portfolio and getting their applications under a cycle of regular security testing. But before you can start getting applications under security management, you have to know about them and get them installed in the system. In this post, we look at some technical means to discover web applications running in your environment and get them loaded into ThreadFix.
As you look to structure and scale your application security program, a critical thing to understand is your organization’s software attack surface. This refers to the set of all software you expose to the world that could be attacked. That’s potentially a lot of software – check out slides 8 – 26 for more background – so to keep yourself sane (and to have any chance of success) you need to draw some lines around what you consider to be in scope and out of scope for your application security program. In addition, you can’t defend attack surface that you don’t know about, so discovery of these application assets is a critical part of the process of standing up a successful application security program. In this blog post, we will look at some technical means for identifying web applications on your network, and how to get those applications under management in ThreadFix.
How Do I Find My Web Applications?
Web applications are hosted on – wait for it – web servers. And web servers tend to listen on a common set of ports. So – if you know the IP ranges where you are likely to be hosting applications, you can scan those networks and ports to identify likely web applications. Now – just because you’ve identified a web server doesn’t mean you’ve found an individual web application – a web server can host multiple applications or a web app may be hosted by a series of web servers. So once you’ve discovered a web server you will need to do some investigation to get to a final list of actual web applications. But a list of web servers is a great starting point. Once you know about web servers, you can also get them loaded into ThreadFix so that you can start tracking the results of application security testing.
Cool – Let’s Do It!
First – I want to introduce a new GitHub repository we’ve turned on: threadfix-examples. This is basically just a home for various utilities and examples of how you can do different things with ThreadFix. The first thing we’ve pushed up there is an initial version of a Python script to identify potential web servers and, optionally, load that data into ThreadFix.
To use the script, you have to install a couple of dependencies. The first of those is the nmap network discovery tool. You can find instructions on downloading and installing nmap for your environment here.
Also, on my OS X laptop where the script was originally developed, I’m running Python 2.7.10 and I needed to install a couple of dependencies:
- The python-nmap This is a handy wrapper for nmap that lets you run nmap and manipulate the results of scans.
- The threadfix_api This is a wrapper for the ThreadFix REST API that makes it easy to interact with ThreadFix servers. Also I want to send out a big “thanks!” to Adam Parsons who developed this Python package. It is fantastic and makes it really easy to do cool automation stuff with ThreadFix.
These Python module dependencies can be installed by running:
pip install python-nmap pip install threadfix_api
So what does the script do? It:
- Runs a scan of a network and a set of ports to identify possible webservers
- (Optionally) creates new applications in a ThreadFix server for those webservers
Here’s the script:
And here’s the output when run on a test network:
And here’s what shows up in ThreadFix:
A couple of things to note:
- If the script is run without the –create option, then it will scan the network with nmap and dump results to standard output, but it will not put the results in a ThreadFix server. So – you can use this script even if you’re not running ThreadFix.
- You can use the –ports option to specify a custom list of ports. By default, the script uses some common web server ports (80, 443, 8000, 8008, 8080, 8443)
So How Would You Actually Use This?
First – identify the data centers where you would expect to be hosting web applications. And while you’re at it make sure it is cool for you to be running nmap on those networks. Then you can run this script on each network segment – probably with a different ThreadFix “team” name for each. That will get you your list of web servers broken down by network. Once you have all of the data loaded in, you will need to do some discovery to figure out what the actual list of applications looks like based on the list of web servers and clean up the list in ThreadFix. This will probably entail renaming applications to something more sensible – versus the IP and port – and probably renaming the ThreadFix “teams” and moving applications between them. So you aren’t done when you run the script, but it should give you a solid base from which to work.
- The script provides a starting point for discovering web applications running on your networks and automatically loading them into ThreadFix. This gets you underway in the process of getting these applications under security management.
- This only works for web applications and your organization likely has lots of different kinds of applications you should be concerned about (more posts to come to look at discovering mobile and other types of applications)
- This doesn’t provide a lot of context about the “applications” it discovers so you’ll still need to clean up the data and rename stuff to get a true, business-focused list of your web applications (another post will follow that tries to help a bit with this)
- This will only find web servers on networks that you scan and doesn’t help with web applications that may be hosted with external IaaS or PaaS providers. So you’ve found some of your web applications, but almost certainly not all of them.