| software defect and release measures |
| Written by Edvin Eshagh |
|
Estimate errors in software system by fault seeding, defect pooling, specify your confidence level in an error free system (i.e. 95% sure), how many hours to test, system reliability, system availibility, defect density, defect removal efficiency, testing effectiveness Below list provides specific strategies for determining when a software product is ready for release. Errors in System - Fault seeding; explicitly inject number of faults in the systems, and your ability to find the number of faults provides an estimate to total number of errors. Errors in System - Defect pooling; using two independent testers to detect faults Confidence in error free system - Fault seeding; Here we want to say our confidence for error free system. For example, we are 95% sure that this product is error free; and is ready for release. If n <= N Then your confidence for error free system is defined as: C(S, S-1) / C(S+N+1, N+s) Hours Left to Test - Number of hours to test to reach a reliability goal (aka, “zero failure testing”) Total Project Testing Hours = [ ln C / (0.5+C) ] / [ ln (0.5+C) / (T+C) ] If you find a failure in the remaining test hours, you’ll need to retest. For example, see “Practical Software Testing” by Burnstein, p422 System Reliability – How reliable is your system, now that you have gone live. System Availability (up time) = MTBF / (MTBF + MTTR) Defect Density (DD) – Measure your DD and compare it to historic DD for a lifetime of an existing system. Unless you’ve undergone significant process change, your defect density ratio will remain in the right order of magnitude. Estimate Defects through software life cycle (analysis, design, coding, developer testing, system testing, operation) using Norden-Rayleigh model (http://www.sceaonline.org/Files/2004confpaper_meth2_Rayleigh%20Curves-A%20Tutorial%20-%20Chelson%20Coleman%20Summerville%20VanDrew.ppt); Software Management 7th ed (Donald J Reifer) p350. With this mode, we can predict the number of defects to be discovered by a certain time within a development lifecycle. Defect Removal Efficiency - As time passes, we may want to see if our defect removal rate improves before a launch. Testing effectiveness |