6 Critical Questions to Ask Vendors to Ensure IT Project Success

An Article by Denim Group Principal, John Dickson

Historically, a large portion of IT projects end with the client feeling unsatisfied. Denim Group feels that lack of proper methodology and project management are the leading cause of this failure. Hold your vendors to a higher standard. Consider the following when selecting a vendor for your IT project:

Does the vendor provide a detailed statement of work with well-articulated descriptions and mutually agreed upon milestones? The statement of work should protect both the client and the vendor. One of the easiest ways to hold a vendor accountable for delivery is to tie contractual payments to project milestones.

Does the vendor use an industry standard methodology that is considered “best practice”? There are many well-established methodologies that outline all of the processes from project management to execution including, but not limited to, Microsoft Solution Framework (MSF), Rational Unified Process, Iconix and Feature Driven Development. Different methodologies are appropriate for different projects and organizations. Beware of firms who have no methodology or cannot articulate how they will execute on the methodology selected for your project. Ask the vendor to elaborate on the execution of the project methodology and ask to see sample deliverables.

How detailed is the vendor’s requirement gathering phase and what is the deliverable at the end of it?Look for a vendor that will develop a very detailed Functional Specification before any development begins. This document should become the blueprint against which the vendor will deliver and be contractually bound. When budgets are tight, this is often the first thing that gets scaled back, but it is the most critical step in ensuring that you will be satisfied with the outcome. Uncertainty that is allowed to persist during a project hurts both the client and the vendor by reducing project predictability, damaging the client/vendor relationship and taking the focus off delivering business value to belatedly allocating responsibility and ultimately blame. Also be wary of a firm that offers to get started on your project right away and skip these steps. Getting started immediately is tempting, but means that the vendor is not taking the time to truly understand your business and the bounds of the project.

How will change orders affect your specific project and what landmarks will there be along the way?Business requirements inevitably change during the course of every project. It is important to understand that these changes will affect the overall budget and deadline differently at various stages of the project. Consider the difference in adding a room to the blueprint of a house before building versus adding a room once the foundation has been poured and the framing has begun. It helps to get major changes out of the way early and to have short phases that give you the ability to re-prioritize at scheduled checkpoints. While it is impossible to plan for everything in advance, look for a vendor who can clearly articulate the landmarks in your project.

Is there a clearly defined division of labor on the project? At the very minimum, there should be three separate people involved in a project: a project manager, a developer and someone to handle quality assurance and testing. There are inherent problems with a developer testing her own code, and acting as her own project manager, and yet it happens regularly at less mature vendors. Visit the vendor’s premises and ask open-ended questions about how a typical team is structured to gain insight into their methods. Beware if the division of labor is not clearly defined.

How does the vendor perform Quality Assurance and is it included in the estimate? Ensure that the project has a realistic amount of time allocated to testing and stabilizing the solution and that the developers will not be testing their own code. Find out whether they will be using automated regression testing to prevent reintroduction of bugs once they have been fixed. You will be responsible for testing the entire system and signing acceptance. Make sure you have resources allocated for this phase and make sure the vendor has a mechanism for tracking your issues, fixes and verifications.

If a potential vendor company can satisfy your expectations regarding these criteria, you’re off to a solid start. You should not make exceptions to work with anyone who may not be able to meet your standards. If the client and vendor are on the same page and fully understand each other’s business, processes, limitations and expectations from the beginning, you are more likely to have a successful final product that satisfies everyone, and a good relationship that you can count on for future projects.

About John Dickson

John Dickson is an internationally recognized security leader, entrepreneur and Principal at Denim Group, Ltd. He has nearly 20 years' hands-on experience in intrusion detection, network security and application security in the commercial, public and military sectors. As a Denim Group Principal, he helps executives and Chief Security Officers (CSO's) of Fortune 500 companies, including major financial institutions, launch and expand their critical application security initiatives.