Use Metrics to Manage
Analytical Metrics
An extensive metrics program with early indicators of evolving problems on ongoing projects and for measuring the effectiveness of management and technical practices is essential to success. Almost all, useful software metrics can be collected at very little additional cost as part of implementing some best management or technical practice. The value from a metrics program comes from the actions taken as a result of metrics analysis. Therefore, collecting metrics solely for the sake of collecting metrics is not a best practice and is, in fact, counterproductive. Of key importance in an effective metrics program is having predefined thresholds for metrics values that trigger predefined actions when measured metrics values are outside their threshold values.
There are two perspectives that must be addressed under this practice. The first and most obvious is the status and health of the programs technical activity: Is the program accomplishing its objectives? The measures to determine this will typically be task-specific. The other perspective is whether metrics management is ingrained into the culture of the organization.
The following table illustrates the Use Metrics to Manage measurement relationships and how they are applied.
| Practice Area | Leading Indicator(s) | Secondary Measures | Contributing Practice Area |
| Use Metrics to Manage | The SPMN Control Panel | Cost of measurement | Project Integrity |
| Adopt Continuous Program Risk Management | |||
| Estimate Cost and Schedule Empirically | |||
| Track Earned Value | |||
| Track Defects against Quality Targets | |||
| Construction Integrity | |||
| Adopt Life Cycle Configuration Management | |||
| Manage and Trace Requirements | |||
| Use System-Based Software Design | |||
| Ensure Data and Database Interoperability | |||
| Design Twice, Code Once | |||
| Assess Reuse Risks and Costs | |||
| Product Integrity | |||
| Inspect Requirements and Design | |||
| Manage Testing As A Continuous Process | |||
| Compile and Smoke Test Frequently | |||
The SPMN Project Control Panel is both a concept and a tool
for visualizing and monitoring the condition of a project and
predicting its future course. The Control Panel facilitates the
entire project team's quick determination of their project's
status and identifies areas for improvement. The Control Panel
was designed to help project managers keep their projects on
course when data for the Control Panel are updated regularly.
When gauges are not in acceptable ranges, they indicate to management
that potential trouble lies ahead. The Control Panel displays
information on progress that includes productivity and completion,
change, staff, risk, and quality. These criteria were chosen
to cover the primary areas that every project manager needs to
track to avoid failure on large-scale software development projects.
(The SPMN Control Panel is being modified to reflect the leading
indicators identified in this document.)
Use Metrics to Manage Secondary Measure
Cost of Measurement
To use metrics as the basis for managing a program, it is necessary to collect information. The cost of data collection is one of the most critical, yet misunderstood, attributes of a software measurement program. Many organizations assume that the cost of measurement is so excessive that they cannot justify establishing a measurement program. Others claim that measurement can be a nonintrusive, no-cost addition to an organization and will have no impact on the organization's overhead. The truth lies somewhere in between.
Measurement is not free, but it can be tailored in size and cost to fit the goals and budgets of any software organization. A measurement program must be undertaken with the expectation that the return will be worth the investment. If the cost is not budgeted, there will be frustrations, attempts at shortcuts, and a failed software measurement program. Planning must incorporate all hidden elements of the proposed effort-elements that are often more expensive during start-up than after the measurement program becomes operational. The higher start-up cost is an additional reason to start small.
Planners often incorrectly assume that the highest cost will be to the software development or maintenance organization. This part of the overhead expense, which includes completing forms, identifying project characteristics, and meeting with analysts, is actually the least expensive of the three major cost elements of the measurement program:
- Cost to the software projects-the source of data
- Cost of technical support
- Cost of analyzing and packaging.
The cost of the measurement program also depends on the following considerations of scope:
- Size of the organization
- Number of projects included in the measurement program
- Extent of the measurement program (parts of the life cycle, number of measures, etc.)
Experience from the Software Engineering Laboratory (SEL) at the NASA Goddard Space Flight Center shows that there is a minimum cost associated with establishing and operating any effective measurement program. The total cost will increase depending on the extent to which the organization wants or can afford to expand the program to address additional projects, more comprehensive studies, and broader measurement applications.
Generally, the SEL experience is that the cost of measurement to the development or maintenance project will not exceed 2 percent of the total project development cost and is rather more likely to be less than 1 percent (which implies that the cost may be too small to be measured).
