QA Metrics

Last month in Executive Brief, I defined how to identify and exploit Quality Checkpoints. As mentioned, a simple definition of quality checkpoints is an opportunity within a project sub-process to identify and remove defects or “bugs.” These defects may be found through walkthroughs, reviews or testing.

Quality checkpoints, however, are only the beginning when it comes to quality assurance. Once checkpoints have been identified within the project life cycle, these checkpoints can then be used to extract objective metrics data. A client once dubbed this end product the “Metrics Report Card” – an opportunity to use the Quality Checkpoint sub-process to capture objective metrics that can evaluate quality, scheduling, or productivity.

Why Measure?

What business value can your organization expect from a measurement program?

An effective measurement and analysis program can help evaluate current performance and quality as well as predict future performance and quality. Further, measurement programs can communicate these particular assessments to others, help an organization to focus on areas that require process improvement and these programs can also identify the steps that must be taken to implement these improvements.

Please keep in mind that measurement itself is not a goal. Measurement and the associated metrics are diagnostic tools and therefore they simply are a means to an end. Further, to be most effective, measurement activities must be focused on seeking corrective actions that will build a foundation for continuous process improvement within an organization.

What to Measure?

At its simplest, “metrics” can be defined as a collection of measurements that provide information about a product and/or a process. An overly ambitious metrics program can collapse under its own weight if a metrics program attempts to measure too much. Similar results can be expected if an organization does not initially evaluate why a certain metric will be collected or how a metric will be used.

For a new metrics initiative, I typically recommend a “back to basics” approach that concentrates on collecting the following four base types of metrics:

  1. The size of the project or product
  2. The effort required to make the product or complete a project
  3. The quality of the product
  4. The schedule of the project

Further, it is essential that stakeholders understand how they will use the metrics - and how they will benefit from the metrics. A well-designed metrics program will:

  1. Provide motivational and tangible goals for management and staff
  2. Provide information for managing current projects
  3. Provide baseline information for planning future projects
  4. Identify opportunities for improvement (i.e. continuous process improvement)
  5. Eliminate rework