Step Right Up

Step Right Up David Liss May 16, 2001

PM 101 Advice, Anecdotes and Case Studies

Metrics are measurements, collections of data about activities, resources and deliverables. We all use metrics, informally and formally, every day of our lives to make decisions about everything from relationships--"If she does that freaky laugh again, I will shoot myself"--to how you decide to buy a car, to most any activity in your life. In your company metrics are used (hopefully) to evaluate your performance, your group's performance and your company's performance. Increasingly companies are looking to various metrics and standards to improve efficiency and control costs for software development and for other organizational objectives. YOU--THE PROJECT MANAGER--WILL HAVE TO IMPLEMENT THESE @#$%&@ METRICS. Watts Humphrey, the creator of the Capability Maturity Model® (CMM®)  claims that Microsoft could have saved $4 billion in development costs with proper quality processes and better testing procedures. This article will be one of a continuing series of articles on software metrics and structures. The purpose of these articles is to give you a basic understanding of the methodologies and how they may be implemented. You are on your own to decide how to deal with your date with the freaky laugh.

According to Bill Pollak, public relations manager for the Software Engineering Institute (SEI), at Carnegie Mellon University, where the Capability Maturity Model was developed, "For software development, there are two general kinds of metrics--metrics that focus on management, such as CMM, and metrics which focus on technical practices and performance like lines of code written and function point analysis (a tool for measuring changes in functionality from software development projects). These metric standards do not compete and these performance metrics are in fact part of a later stage of the CMM model."

Enough Already, So What is CMM and Why Should I Care?
SEI Fellow Watts S. Humphrey first created CMM at the SEI in 1987. (Is that enough acronyms in one sentence for you?) The project was originally funded by the U.S. Department of Defense. More than 5,000 government and private companies in more than 42 countries now use this standard. Large portions of the Level 5 assessments that have been made are based in India. Among the primary goals of this process are to identify and reduce errors during the initial coding process to cut the amount of rework required from finding errors during the testing processes. 

According to Pollak, "The CMM for Software (SW-CMM) is a graduated collection of best software engineering practices. Levels 1 through 5 of the SW-CMM define a path toward organizational improvement. The practices are 'graduated' in the sense that the levels build on one another. Note that we use the word 'practice' as opposed to 'theory'--these are proven best practices, based on real-world experience." The Capability Maturity Model for Software describes principles and practices to take organizations on an evolutionary path to move software development from a magical art to an engineering discipline. The CMM is organized into five maturity levels. The steps below are excerpted and adapted from the  SEI website:

Level 1: Initial
The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics and sometimes clubs, whips, chains. With occasional bouts of pleading and begging (quite unbecoming, really).

Level 2: Repeatable
Project management structures and controls begin here! Basic project management processes are established to track cost, schedule and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications. The key components of project management at this level are: Requirements Management, Software Project Planning, Software Project Tracking and Oversight, Software Subcontract Management, Software Quality Assurance and Software Configuration Management.

Level 3: Defined
The software process for both management and engineering activities is documented, standardized and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization's standard software process for developing and maintaining software. The key process areas at Level 3 address both project and organizational issues. At this level the organization establishes an infrastructure within the company to institutionalize software engineering and management processes across all projects. Compliance with these standards for all projects and by all project managers and development teams are REQUIRED!! These activities are Organization Process Focus, Organization Process Definition, Training Program, Integrated Software Management, Software Product Engineering, Intergroup Coordination and Peer Reviews. Frisking by security personnel at entrances is used to better control the use of whips and chains as described in Level 1.

Level 4: Managed
Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood, controlled and evaluated by the CIO and related staff. The key process areas at Level 4 focus on establishing a quantitative understanding of both the software process and the software work products being built. They are Quantitative Process Management and Software Quality Management. It is at this level where analysis tools such as function point analysis and other performance metrics are implemented.

Level 5: Optimized
Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies. The key process areas at Level 5 cover the issues that both the organization and the projects must address to implement continual, measurable software process improvement. They are Defect Prevention, Technology Change Management and Process Change Management. (Time for a big group hug!)

The Downside: 
Organizational integration prioritization and obfuscation. What?!

Getting your company to make an effort to move up to the next CMM level may not be easy. Many companies are struggling to just keep their heads above water with the increasing demands placed on technologists within a company. In the scheme of things it is often easier to do nothing than something. Change in an organization is hard, expensive and takes resources away from implementing newer, "sexier" projects and the operation of mission critical systems. In addition, highlighting the inefficiencies of an organization can be perceived by some executives as highlighting their weaknesses as a manager. This can result in half-hearted implementation efforts by senior executives and a lot of wasted time by project managers and development teams. 

Organizations are often placed in circumstances where organizational crises overwhelm intentions for graduating to higher levels of the CMM methodology. Many companies simply do not have the commitment and follow-through to implement the required ?infrastructure? and never graduate beyond Level 1: "ad hoc, little formalization with few rules." Few organizations in the United States can be identified at Levels 4 or 5.

Stay tuned for further insight into metrics and CMM. Next up: Dealing with the freaky laugh and more on the proper use of whips and chains described in CMM Level 1.

David Liss is a management strategist, consultant and business writer based in Arlington, Virginia.