The article first appeared in CrossTalk, Aug 2008
To mature software engineering, we must learn from history, and to learn from history, we must define what we do when we develop software and how we do it. When we produce a process specification and define measures for the key process parameters, we are beginning to build the basis for reliably and consistently recording and learning from history. That is why the process revolution is so important.
Judging by the history of other fields, software engineering and computer science are still in their infancy. We have yet to build an accepted foundation of proven and generally followed principles, and there is little in the way of defined and widely taught practices to guide working professionals. When new problems arise in the software field, the tendency is to invent some new language, tool, or method to solve them.
The true measure of maturity for our field will be when we stop dismissing the lessons of history and start building upon them. While there are many proven truths in the practice of software engineering, few software people accept them until they have personally suffered through painful and expensive failures. Unfortunately, as our field has grown and expanded, the number of ways to fail has become so large that learning through failure is no longer a viable way to advance the field.
To have a consistently effective software industrial capability, we must use defined and precise methods and well-understood procedures. Learning from history is the only way to develop such methods. While I was not the first or only person to start introducing process methods into industrial software engineering practice, I was fortunate enough to be involved in some of the early key events. This brief history provides insight into what happened, why it happened, and what is likely to happen next.
Software Factories
Since the beginning of the computer age, software projects have been troubled. Schedules were rarely met, costs were unpredictable, and total project failures were common. Of the many attempts to address this situation, one particularly ill-conceived idea was an attempt approximately 20 years ago to solve software’s problems by establishing what were called software factories. The rationale for this idea, as I understood it, was that since typical manufacturing factories regularly produced products with predictable costs and schedules, the software community should emulate factory practices. The hope was that this would somehow enable them to achieve factory-like results.
While at IBM in the early ’80s, I visited Japan and met with several software groups. One proudly took me on a tour of their software factory. It was comical. Before entering the factory’s cleanroom, we had to put on smock coats just like the ones worn in semiconductor cleanrooms. Inside were dozens of software developers lined up at rows of identical work stations. While this did actually look like a factory, I had trouble believing that this factory charade could have anything but a negative impact on software development creativity and productivity. Sure enough, in a relatively short time, the idea of the software factory disappeared into the growing dustbin of software’s simplistic and superficial methods that do not address fundamental problems.
Mike Fagan
A more thoughtful approach to software’s problems had actually been initiated a few years earlier when Mike Fagan published his landmark paper on software inspections [1]. In it, he not only described a way to economically improve software quality, but he provided a coherent description of software process principles and explained how process concepts can be used to improve the performance of software groups. Fagan’s ideas had already been tested on several projects and were found to provide truly extraordinary results. The benefits were so substantial, in fact, that he was given a $100,000 outstanding contribution award by IBM corporate management.
The 1986 Attitude
In my final years at IBM, my views were heavily influenced by Fagan and Al Pietrasanta [2]. I even drafted a short paper for the IEEE Spectrum, outlining a vision of how, by improving the software process, we could greatly improve the performance of software groups [3]. When I joined the Software Engineering Institute (SEI) in 1986, Fagan’s inspection process was widely used at IBM but practically nowhere else. Furthermore, the ideas that he, Leon Osterweil, Bob Balzer, and others were discussing about defining and using processes were not understood by most of the software community or even recognized as important [4]. In fact, when I talked about software process at the time, the typical question I was asked was, “What in the world is a software process?”
This attitude posed a problem. Fortunately the SEI then had a shortage of experienced business managers so I was asked to run the finance and administrative operations. I agreed to do so on two conditions: 1) that the assignment would be temporary, and 2) that I could start a process group of my own. That is how we started the SEI Process Program.
Comments