basic

SOFTWARE TESTING EFFORT  ESTIMATION :

This topic is a mixture of practical experiences and estimation theory (estimation science, theoretical knowledge). The purpose of this topic that the Test Leads, Managers or aspiring leads, managers must aware of all the test estimation techniques. This will helps in clearing interviews and in test planning as well. In the upcoming topics we will discuss the Guidelines and Principles for Test Estimations.
  Introduction - Estimation is done often because it is expected to help in predicting how much will the project cost and when will the project get completed. Proper analysis and effort estimation is necessary for successfully planning for a testing project. Any flaw in critical estimation phase, results in missing the project deadlines, reduces ROI and loses of customer’s faith. Remember - Bad estimation can lead to poor distribution of work. 

There are different standard and non standard methods for test estimation. In many product based companies, people utilize non standardized but conventional estimation methods to make things work. These methods might have developed over a continuous period accommodating hidden factors like nature of application under test, environment, and risk factors for that specific product/market. But these methods can’t be adopted as a generalized organization standard for a mature operation model.

Fact: Managers/Leads are not comfortable with software estimation work. But it is a required activity, so based on their past experience on one particular Product, Test Leads/Test Managers estimate the entire testing project (but for that Product only). Because they spent 1-2 or 2-3 or even more years on that particular product. But if they are asked to change company or shift to new product of entire different domain, then it is very hard for them to do estimation.

In this topic, we will discuss following methods for test estimation: 1. FIA (finger in the air) or best guess 2. Ad-hoc method 3. Experience Based - Analogies and experts 4. WBS 5. Delphi technique 6. Three-point estimation (successive calculation) 7. Function points / Test point Analysis 8. Percentage of development effort method 9. Percentage distribution 10. Use case point estimation method

 

Lets discuss one by one:

1. Best Guess – This technique is purely guesswork and based on the some sort of experience.

The method is very common, but since it is based on your gut feeling, its uncertainty contingency is probably around 200% or even higher.

2. Ad-hoc method - The test efforts are based on tentative timeframe. The timeline set by managerial or marketing personnel or by client without any guess/experience. Alternatively, it is done until the budgeted finances run out.

This is very common practise in extremely immature organizations and has error margins of over 100% at times.

3. Experience Based - Analogies and experts:

• Metrics collected from previous tests.

• You already tested similar application in previous project.

• Inputs are taken from Subject Matter experts who know the application (as well as testing) very well.

4. Work Breakdown Structure - It is created by breaking down the test project into small pieces. Modules are divided into sub-modules. Sub modules are further divided into functionalities and functionalities are divided in sub-functionalities.

Review all the requirements from Requirement Document to make sure they are added in WBS. Now you figure out the number of tasks your team needs to complete. Estimate the duration of each task.

5. Delphi technique – Same as above WBS. Here functionalities and each task is allocated to each team member. Then team member gives estimate that they will take this much hours to complete the task.

Averagely, this technique gives good confidence in the estimation. This technique can be combined with other techniques.

6. Three-point estimation – This technique is based on statistical methods In this technique, task is broken down into subtasks (similar to WBS) and then three types on estimation are done on each chunk –

• Optimistic Estimate (Best case scenario in which nothing goes wrong and all conditions are optimal.) = a

• Most Likely Estimate (most likely duration and there may be some problem but most of the things will go right.) = m

• Pessimistic Estimate (worst case scenario which everything goes wrong.) = b

Formula to find Value for Estimate (E) = a + (4*m) + b / 6

Standard Deviation (SD) = = (b - a)/6

More information about this technique can be find at:

http://www.projects.ed.ac.uk/methodologies/Full_Software_Project_Template/EstimationGuidelines.shtml

7. Function Point/Testing Point Analysis: The FP technique is a direct indicator of the functionality of software application from the user's perspective. This is the most accepted technique used to estimate the size of a software project.

This technique is a part of TMap. Base of this technique is function point technique. Here we convert function points into test points. In Test Point analysis, we usually carry out the following:

• Dynamic Test Points

• Static Test Points

• Environmental Factor

• Productivity Factor

• Primary Test Hours

• Control Factor

• Total Test Hours

You can download the detailed self explained presentation here –

http://www.eurostarconferences.com/community/member/eurostar-presentations-archive/test-effort-estimation-with-test-point-analysis.aspx

In Testing, This estimation is based on requirement specification document, or a previously created prototype of the application. To calculate FP for a project, some major components are required.

The major components are:

Unadjusted Data Function Points: i. Internal Files, ii. External Interfaces

Unadjusted Transaction Function Points: User Inputs, ii. User Outputs & iii. User Inquiries

Capers Jones basic formula:

Number of Test cases = [Number of Function Points] x 1.2

Total Actual Effort, TAE = (Number of Test cases) * (Percentage of development effort /100)

This method is done in a case when a detailed low level design document or requirement document is available (i.e measure of function point is available) & Previous data for development and testing is available. But now days, when we are using agile and iterative methodologies to deliver projects, so most of the times all this documentation is not available.

8. Percentage of development effort method

Here the assumption is that a more complex business application may require more testing effort. The test effort required is a direct proportionate or percentage of the development effort.

Note: The development effort can be estimated using line of code (LOC) or function point (FP) which is not in the our scope.

Example:

If a previous project with 500 FPs required 50 man hours for testing, the percentage of testing effort is calculated as:

P = (50 / 500) * 100 =10%

For the current project with a development effort, say 1500 FPs, the testing effort is:

Total Actual Effort, TAE = 1500 * (P/100) = 1500 * (10/100) = 150 man hours.

9. Percentage distribution – Here all the phases of SDLC are divided in parts and assigned effort in %. Like –

Project management 7%

Requirements 9%

Design 16%

Coding 26%

Test (all test phases) 27%

Documentation 9%

Installation and training 6%

2 WHAT IS TEST PLAN 

A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the 'why' and 'how' of product validation. It should be thorough enough to be useful but not so thorough that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project:
• Title
• Identification of software including version/release numbers
• Revision history of document including authors, dates, approvals
• Table of Contents
• Purpose of document, intended audience
• Objective of testing effort
• Software product overview
• Relevant related document list, such as requirements, design documents, other test plans, etc.
• Relevant standards or legal requirements
• Traceability requirements
• Relevant naming conventions and identifier conventions
• Overall software project organization and personnel/contact-info/responsibilities
• Test organization and personnel/contact-info/responsibilities
• Assumptions and dependencies
• Project risk analysis
• Testing priorities and focus
• Scope and limitations of testing
• Test outline - a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable
• Outline of data input equivalence classes, boundary value analysis, error classes
• Test environment - hardware, operating systems, other required software, data configurations, interfaces to other systems
• Test environment validity analysis - differences between the test and production systems and their impact on test validity.
• Test environment setup and configuration issues
• Software migration processes
• Software CM processes
• Test data setup requirements
• Database setup requirements
• Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs
• Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs
• Test automation - justification and overview
• Test tools to be used, including versions, patches, etc.
• Test script/test code maintenance processes and version control
• Problem tracking and resolution - tools and processes
• Project test metrics to be used
• Reporting requirements and testing deliverable
• Software entrance and exit criteria
• Initial sanity testing period and criteria
• Test suspension and restart criteria
• Personnel allocation
• Personnel pre-training needs
• Test site/location
• Outside test organizations to be utilized and their purpose, responsibilities, deliverables, contact persons, and coordination issues
• Relevant proprietary, classified, security, and licensing issues.
• Open issues
• Appendix - glossary, acronyms, etc.