When an organization executes a certain number of projects over a period of time it needs to compute certain summaries for the purposes of evaluating the performance of the company.
Some of the metrics that need to be computed are net effort variance or the variance of the total effort with respect to the planned effort. During the project planning phase a project manager estimates the effort required to complete a task. A task in a software engineering company can be an analysis task or a programming task. So during the project planning phase the planner states that a specific programming task would take a certain amount of hours to complete.
When the project is executed the actual effort (say) is measured and is recorded against the planned activity there may be a variance or a difference between the two values. The same is the case with project schedule. On a related note Project Schedule should be derived from effort and not independently of effort by using independent models for effort and schedule as schedule is statistically correlated to effort.
When planning the schedule and later on while measuring the actual there may be a difference or a variance. During some review period an organization releases organizational baselines with summary information for effort, schedule variance and also for the number of defects occurred, productivity ratio etc.,
Care should be taken to compute (say) the net effort variance, for example one should not add all the project variances together to obtain the cumulative variance. To explain why this should not be done many of these projects may have been executed simultaneously and so many of them may have a common cause of variation, for example if there was a server crash on a particular date the downtime may affect many projects uniformly and may prolong the time required to complete a task. Adding all these variances without doing a causal analysis will lead to reporting an increased figure. What can be done is to mathematically split the effort /schedule variance between all the projects that are affected by it.
Also an analysis of the variance has to be undertaken and one has to check if there are false positives or false negatives using hypothesis testing. One should also use stratified sampling to analyse the net variance. For example if a project group with lower developer skill is dominating the measurements, corresponding scaling factors have to be applied to each measurement obtained from individual projects so that one sampling group alone does not dominate the others.
In synopsis the variance obtained after comparing the actual in the project with the plan should be subject to standard ANOVA tests. Also the actual value of the variance should be filtered out for repeated measurements of the same deviation being caused in multiple projects.
