software defect and release measures
Written by Edvin Eshagh   

Estimate errors in software system by fault seeding, defect pooling, specify your confidence level in an error free system (i.e. 95% sure), how many hours to test, system reliability, system availibility, defect density, defect removal efficiency, testing effectiveness

Below list provides specific strategies for determining when a software product is ready for release.

 

Errors in System - Fault seeding; explicitly inject number of faults in the systems, and your ability to find the number of faults provides an estimate to total number of errors.
Total_Number_of_Errors_in_System = Total_Seeded_Faults * Number_of_Unseeded_Faults_Found / Number of Seeded Faults Found

Errors in System - Defect pooling; using two independent testers to detect faults
Total_Number_of_Errors_in_System = Number_of_Faults_Found_by_Group1 * Number_of_Faults_Found_by_Group2 / Number_of_Faults_Found_That_IS_Shared_By_Both_Groups

Confidence in error free system - Fault seeding; Here we want to say our confidence for error free system. For example, we are 95% sure that this product is error free; and is ready for release.
Lets define some variables for this strategy.
Let S = Number of SEEDED faults
Let s = Number of SEEDED faults FOUND through testing
Let N = Number of KNOWN faults (which you may think is zero)
Let n = Number of UNSEEDED faults found

If n <= N Then your confidence for error free system is defined as: C(S, S-1) / C(S+N+1, N+s)
Where C(a,b) is combination notation = [a * (a-1) * …(a-b+1)] / [b * (b-1) … 2 * 1 ]

Hours Left to Test - Number of hours to test to reach a reliability goal (aka, “zero failure testing”)
Let C = Target number of failures allowed to be seen by customer (say 1)
Let N = Number of KNOW faults
Let T = Number of hours spent testing

Total Project Testing Hours = [ ln C / (0.5+C) ] / [ ln (0.5+C) / (T+C) ]

If you find a failure in the remaining test hours, you’ll need to retest. For example, see “Practical Software Testing” by Burnstein, p422

System Reliability – How reliable is your system, now that you have gone live.
Let MTTR = Mean Time To Repair
Let MTTF = Mean Time to Failure
Let MTBF = Mean Time Between Failure = MTTF + MTTR
Reliability = MTBF / (1+MTBF)

System Availability (up time) = MTBF / (MTBF + MTTR)

Defect Density (DD) – Measure your DD and compare it to historic DD for a lifetime of an existing system. Unless you’ve undergone significant process change, your defect density ratio will remain in the right order of magnitude.
DD = Total_Defects / Number_Lines_Of_Code
For example:
Let Lines_of_code = 100k lines
Let Defects_Found = 500
Therefore, your DD = 500/100k = 5 per 1000 lines of code.
So, if in your new enhancement update you added 5000 lines of code, then you better find about 5 * 5 = 25 defects

Estimate Defects through software life cycle (analysis, design, coding, developer testing, system testing, operation) using Norden-Rayleigh model (http://www.sceaonline.org/Files/2004confpaper_meth2_Rayleigh%20Curves-A%20Tutorial%20-%20Chelson%20Coleman%20Summerville%20VanDrew.ppt); Software Management 7th ed (Donald J Reifer) p350. With this mode, we can predict the number of defects to be discovered by a certain time within a development lifecycle.

Defect Removal Efficiency - As time passes, we may want to see if our defect removal rate improves before a launch.
Let preDD = Pre deployment Defect Density
Let postDD = Post deployment Defect Density
Defect Removal Efficiency = preDD/postDD

Testing effectiveness
Let N = Number of errors found during testing
Let C = Number of errors found after Deployment
Testing effectiveness = N / (N+D)