"A good cost estimator should never make an estimate that deviates from the actual cost at delivery by more than 20 percent. A good schedule estimate should be within 5 percent of actual schedule at least 95 percent of the time, and should not deviate by more than 12 percent less than the actual schedule." -- Capers Jones

Practice Description

It has been said that experience is the best teacher. When it comes to cost and schedule estimation this is could not be truer. The best indicator of how long a project will take or how much it will cost is how much a similar project cost and how long it took. This is not a reflection of our inability to forecast and analyze a new job. Rather, it is a reflection of the inherent complexity of software development projects and the multitude of ever-present factors that can influence cost and schedule.

Unfortunately, this is often more easily said than done. While few would argue the validity of the above statement, the "view from the trenches" is that comparisons are hard. The rapid evolution of software technology, languages, tools and practices often makes it appear that every software development effort is different. There are, however, some common navigational points that can be exploited. The fundamentals identified below try to capture these.

Practice Essentials

• A cost estimate should be made.
   
• The cost estimate should be based on actual costs from past, comparable projects.
   
• A software size estimate should be an input to the cost estimate. The size estimate should include an estimate of the equivalent new development lines of source code for planned legacy reuse, commercial-off-the-shelf (COTS) and government-off-the-shelf (GOTS) software.
• When COTS application software is involved, an analysis of the fit of the COTS to the current application should be made prior to cost estimation.
   
• A productivity estimate should be made for input to the cost estimate based on measured productivity on past, comparable projects.
   
• The empirical cost estimate should be made with a cost model based on metrics from past, completed projects.
   
• The empirical cost estimate should be compared with a bottom-up engineering estimate made by estimating the labor and other resources for each task in the project's detailed activity network, and differences reconciled.
   
• The schedule should not be unreasonably compressed. If experience shows that similar projects have taken a given length of time, the new project should not be expected to complete in less than 85 percent of that time.
   
• Schedules and cost estimates are influenced by a number of factors, not all of which are under a program manager's control. Do not expect an estimate to be extremely accurate. A good estimation technique will be within 20 percent for cost and 5 percent for schedule. 

Implementation Guidelines

• Estimate the cost, effort, and schedule for a project for planning purposes and as a yardstick for measuring performance (tracking). Software size and cost will be estimated prior to beginning work on any incremental release.
   
• Software cost estimation should include a reconciliation between several different approaches such as a top-down estimate (based on an empirical model; e.g., parametric, cost) and a bottom-up engineering estimate.
   
• Software cost estimation will also be subjected to a "sanity check" by comparing it with industry norms and specifically with the supplier/developer's past performance in areas such as productivity and percentage of total cost in various functions and project phases.
   
• All of the software costs will be identified with the appropriate lower-level software tasks in the project activity network.
   
• Productivity estimate should be a function of software size and type of software.
   
  • Real-time 
  • Engineering systems 
  • Business systems
     
• Productivity estimate should be based on productivity metrics from past projects of the performing organization using the planned processes and technology.
   
• Productivity estimate should be reduced for new people hired into the organization.
  
  
• Estimate should be adjusted for experience base (for example: new application -- no valid comparison available, no experts with experience in application).
   
• Estimate the cost of code reuse by first identifying the specific code modules to be reused and then estimating the percentage of the reuse code in the application that is new or modified.
   
• Cost and schedule estimates at completion should include costs and schedule to overcome problems resulting from risks materializing into problems.
   
  • Risk materialization cost should be the expected value sum (chance x cost impact) for all risks.
     
• Subject cost estimate to sanity-check rules of thumb:
   
  • Percentage of total cost for individual phases and functions
     
  • Productivity computed from size and labor effort estimates
     
  • Cost estimate for the effort to modify and integrate reuse code
     
• Explicitly identify all source code modules to be reused and estimate the amount of code modification and new code to write to integrate each identified module.
   
• Adjust the schedule to avoid large increases over a short time in number of staff in any labor category.
   
• The nominal schedule duration in months is given by nominal calendar months = 2.5 x (staff months)1/3
   
• Allocate the estimated total project labor effort among all the tasks in the activity network. 
  
  
• Detailed activity net will have no more than 20 percent level of effort (LOE).
  
  
• The estimate must be verifiable with documented rationale.

Things to Watch Out For

• Cost estimates are often based on productivity estimates that are better by many factors than the organization has ever accomplished. The rationale supporting this is that a new, unproven technology, never used by the organization before, will support this markedly improved productivity.
   
• Cost estimators and project managers usually do not know the industry norms for productivity as a function of the size and type of the software.
   
• Cost estimates often assume that there is either no or negligible cost in reusing source code.
   
• Most project managers have never heard of the metrics concerning schedule compression.
   
• Cost estimates are frequently based on incomplete and ambiguous system requirements.
   
• When a project begins to show bad earned value metrics, poor cost estimation is usually not considered as a possible reason.
   
• Size estimates that were made at the inception of a project frequently grow considerably over time because system requirements were not adequately defined at that time. 

Figuring Out Where You Are - Project QuickLook

Chapter 6 contains a series of detailed questions that can be applied to any program to determine the health of its cost and schedule estimation approach. These questions should be applied on a regular basis. However, there are some quick indicators that can be used to see if the risk management process requires more in-depth examination. Specific "quicklook" questions are:

• Do you have a database of productivity data? 
• Has a cost and/or schedule estimate been prepared? 
• When was the last time the cost or schedule estimate was updated? 

An inappropriate answer to any of these questions could indicate a more serious problem and merits further investigation.


Additional Considerations


Estimating cost and schedule should be consistent with organizational policy and procedures. Likewise the evaluation of cost and schedule estimates for software acquisition should be consistent with organizational policy and procedures.

• There should be a documented procedure for estimating costs and schedules to ensure consistency.  This procedure should be adapted over time based on the experience from the project.  The procedure should call for estimation of detailed work packages, based on historical data, documenting assumptions and review/approval of estimates.
  
  
• There should be a documented procedure for estimating use of critical assets.  The procedure should call for assets to be identified, the estimates to be based on the software products to be used, traffic and the operations concept.  The estimates should be reviewed and agreed to by all parties at the system level.  
  
  
• Measurements should be made for all tasks to determine cost and schedule status.  This includes support tasks such as SQA and SCM.
  
  
• The cost and schedule estimating evaluation process should be coordinated with organizational processes for consistency and accuracy.
  
  
• Tailoring of the organization cost and schedule estimating evaluation process should be tailored to the project acquisition strategy and documented.
  
  
• Project teams are informed of the tailoring of the organizational cost and schedule estimating evaluation process.

top

---