Best Practice Requirements

How effective is today's requirements elicitation? What do the statistics say? How long is your change request log? Did you ever wonder why there are so many changes during requirements definition? And why can so many project failures be traced back to the requirements?

I'd like to apologize because it's partly my fault. I don't mean just myself personally, but rather the generation of business analysts who crafted the foundation for today's requirements elicitation best practices. That said, the blame shouldn't be placed entirely on the shoulders of my generation. You see, part of the problem is our inability to learn, organizationally. Organizations have adopted a best practice approach to learning. And therein lies the problem. Let's examine the serious drawbacks of this learning approach.

The best practice approach is based on evolution. It is slow, agonizingly slow. I don't know what nature's purpose is, but I can say with some certainty that in nature it is the strong and the prolific (great in numbers) that survive. Nature is not necessarily interested in keeping the "best". So it is with best practices. How do best practices emerge?

It all starts with personal experience. We all have it, and we all develop personal practices around it. At some point, an organization may decide that it doesn't want that many personal practices and it prefers a single organizational practice. So it sets about establishing one. A team is assembled, and based on their past experience, usually 5 - 10 years, they develop an organizational practice. But the one that ends up the winner, isn't necessarily the best, but rather the one that gets consensus. Often this is a compromise close to the majority practice. Of course, we all know that the best is rarely in the majority. Practices are rarely subjected to verification. And so the strong survive, and we get an organizational practice. Now this practice needs to be propagated to the organization. The time-frame for this can easily be 5 - 10 years or even longer. So at the end of the day we have a practice based on experience that is 5 - 10 years old and which takes 5 - 10 years to spread through the organization. So we have a timeline of 10 - 20 years or more to develop an organizational practice.

Then along come consultants who notice these practices. In some they see an opportunity for business. And so, they get behind these practices. Of course, of the many practices out there, the ones with the greatest opportunity for them, are the ones that will get the most support. And so, these practices start to show up at conferences, in articles and in books. As more and more organizations are convinced to try them, we reach a critical point where hundreds of organizations have tried the emerging "best practice". Among the hundreds, we search for three to five organizations that are all well-known, respected and successful. These become the poster boys for the marketing campaign to follow. Their success serves as the fuel for pushing the practice forward. The fact that failure rates for the practice exceed the 90% mark, are never revealed. All we need to show is that it can work, and that it has worked. Eventually, after 10 to 20 years or more, enough organizations are convinced, so we get a dramatic increase in adoption. Now it's out of the hands of the proponents, the pushers, and into the hands of the adopters. The adopters now begin to talk to each other and stories of failures begin to increase. The true effectiveness begins to emerge. And within five to seven years, another best practice hits the dust.

And how long does all this take? It takes 10 to 50 years to discover that a best practice may not be that good. Can you afford to wait that long to try a practice developed by people based on experiences from decades ago? Six Sigma was branded in 1986 by Motorola based on principles that date back to 1920. It is just now beginning to achieve the level of adoption that immediately precedes abandonment. Lean was primarily developed following World War II by Toyota. The majority of Lean principles taught today were developed between 1945 and 1965. That's practically ancient history. Again, it is just now beginning to achieve the level of adoption that immediately precedes abandonment. Now abandonment doesn't mean it'll completely disappear.