Why the Waterfall RFP Kills Most IT Projects

Based on what the customers said they wanted, software engineers would spend 4-6 months writing up a requirements document and lists of product features, attempting to capture what the customer asked for.  This of course assumes that the customer actually knows what they need and what to ask for…which in reality, they do not know.

Then the software engineers would spend a year building the software in a workshop somewhere. Over the course of that entire process, the software engineer maybe talked to the customer a couple times, if that.

Designing software like this is following a waterfall approach. Like an actual waterfall, this method flows downward through its specific phases. It places an emphasis on tools and detailed design documents that anticipate everything the software will need to do 12 months from now.

The problem with this method is that it fails. Almost every time. Remember HealthCare.gov? You get the idea.

Waterfall development methodologies, which form the basis of most RFP’s in the marketplace today, explain why only 20% of projects actually succeed. And why

17% of 5,400 IT projects went so badly that their company’s reputation was put at stake. It can also explain why on average, IT projects run 45% over budget.

Once a product is finished and delivered to customers, a typical response from customers is “oh wait hold on that’s not what I meant…what I needed was….” In other words, by the time the customers actually know what works and what doesn’t work, or by the time they start to understand what they actually need or want, it is too late.

So instead of building what people actually need by measuring how it’s used and how it works and refining it till the customer loves it, software engineers build what they heard customers tell them they needed… in legal circles they call this hearsay and there’s a reason you hear them call it inadmissible as evidence!

A big design-up-front project results in software that does not meet the needs of the customer.

And then the software is never used.

At a conference we attended recently, 11 of the 12 presentations given by participants were about IT projects for mobility solutions, which failed.

The script was the same in every case:

1. Gather requirements
2. Write down requirements in a specification or RFP
3. Hire a team or engage consultants to build it
4. Start testing with users when you have a working version

The problem really lies in writing down the requirements months in advance. Anything you write down is bound to be wrong by the time you get actual feedback from actual users.

The time to get feedback from users and customers is at the time you’re building a feature. This is how almost all successful software today (short of DOD projects and NASA) is developed.  The timeframe from starting work on a feature to getting user feedback is days…and this is the way it should be!!

If you wait until you’ve already created the software, what happens when customers don’t like it?? We’ll tell you what happens… the software is either thrown out all together or best-case scenario, the customer can hire the engineers to find and then fix all of the problems with the software but at what cost?

Studies show that software engineers spend about 40-50% of their time on avoidable rework rather than on what they called “value-added work”. And if for some reason software with errors makes it into the field, the cost of fixing an error can be 100 times as high as it would have been during the development stage.

Best in class defect rates in software are 2.5 defects per function point in the software (Clark, 2010). Average software development defect rates are 8.0 defects per function.

This rework can start to take over your project and can ruin your company’s reputation. So don’t let it ruin yours!

While we don’t deny that this method may work well in a highly structured and predicable environment, like a car manufacturing plant or Space Shuttle development (and we’re not building Space Shuttles here), it’s not the right fit for an environment that is constantly changing nor is it the right fit for customers that have real needs and problems worth solving but who aren’t engineers.

Software development is not like building a car.  It’s more like creating art. While artists start out with an end goal in mind, they don’t follow a specific recipe or procedure in their artwork. They iterate and remove a bit here, a bit there, add some here… until the finished piece comes out. That’s how software works in real life. Facebook, Google, LinkedIn… all the common names iterate over their software, sometimes ending up with something very different to what they started with.

Because 60% of initial requirements change throughout an average development project, being flexible to change is key to making sure your product is successful in meeting your customer’s needs. After all, you want to create a product that your customers will actually use and that will benefit your business, right?

Unlike the Waterfall methodology where you are required to go through each phase only after finishing the prior phase, the iterative model allows you to come back to previous phase(s) whenever you need too, revamp anything that needs revamping and it allows you to catch and fix mistakes early on.

Under this method, you only build the feature that you’re working on right now, preferably the one with the highest business value. And the second that you have it ready, you push it to the field and get real feedback from real people. And by real people I mean your customers! Your customers are the people that matter in the end because they are the ones who will be using the software. And they are the people that need to be involved throughout the entire process.

This model helps you respond to change, unpredictability and constant feedback with ease. In fact, this model places an emphasis on people communicating with each other. It measures success by working software, not perfect software.

So how is it that this model is able to meet customer needs and produce working software within a much shorter time frame than the traditional waterfall approach?

If you still aren’t sure how an iterative model can benefit your business, listen up.

The Iterative Model (we call this Agile software development) does not start with a list of specific requirements and goals that need to be met by certain dates.  Instead, it works to minimize waste by meeting customer’s needs and solving their problems through early and continuous delivery.

You don’t spend 1-2 years developing software that may or may not meet your needs as a customer; it begins with the implementation of a small part of the software, which is then used by the customer – in the wild, so to speak.

The whole idea here is to get software in front of end-users as early as possible in order to get their feedback as early as possible.  New features are planned in a project plan as high level “user stories” that describe from the point of view of the user what it needs to achieve (never how it actually works).  You prioritize these and focus on only the ones with the highest business value and you build only those.

When the development team is ready to work on that feature, they do the design work just before they start work – not months earlier.   They may even want to talk to some actual users right as they’re starting work.  Working in 1 or 2-week time blocks (called “sprints” or “iterations”) they build something and get it to the customer for review ASAP.  The customer then can say “hey, that’s not what I meant… I meant”… and you can go back and rework and iterate until you get something that meets their actual needs… but involving them in the process.

Features that don’t meet their needs are removed. The cycle continues until you have a product that’s good enough to use in the field. And most Agile projects are useable far earlier than their waterfall counterparts.

Most Agile projects use less customer time overall in the development process, because instead of sitting through mindless requirements sessions that last for weeks usually, you’re reviewing actual working software and providing feedback in small chunks on a weekly basis. Most software designers will also make sure that the whole thing works together as a whole as well, because they take pride in what they do and that’s important to them.

The idea here is not to build the Titanic.  It’s to build a lifeboat. And then be able to steer the wheel. And maybe a hull and a compartment or two.  And get feedback from the real world on each one. Because if you build the Titanic, it’s not invincible.  You just can’t predict everything that can and will happen in the real world up front..it just doesn’t work that way..Sorry to disappoint you but better you face the facts now, right?!

Today, most software in the world today is built using this methodology. Because it works. Simply put, it makes sense.

By building small parts and getting user feedback quickly and often, you can end up with a piece of software that’s going to meet your customers actual needs rather than what you thought your customer’s needs were going to be 2 years earlier when the engineer asked you what you needed…  In most cases until you’ve seen it, how do you know?

If you are going to go down the route of developing a software project, we don’t want you to repeat the mistakes of those in the past. We don’t want you to fall into the trap of writing a requirements document and doing an old school waterfall project thinking that it has to take 2 years. There are platforms out there that can help you build a custom configured app within a couple months that actually works. Crazy, huh?!

The way we see it is software development failure is almost always avoidable and you do not have to fail! If you know for a fact that a custom-configured software, like Field Squared, isn’t the right fit for your business, we hope that you will still use–or at the very least refer–to an iterative model when developing your own app.