Test automation estimation techniques in software: Full guide

Today’s software development standards require IT professionals to release high-quality final products and meet deadlines. The process complexity is reinforced because most projects initially have a limited budget that cannot be expanded as needed. Therefore, to remain competitive in the market, development teams need to have a clear, documented list of requirements for the final app and correctly estimate the full effort required to complete the program’s creation.

Because of these characteristics of the modern IT sector, the evaluation of software testing efforts becomes part of evaluating the software development process and software life cycle in general and the key to the financial survival of the project.

Table of Contents

What is Software Test Estimation?

Software test estimation is the phase of a project where the tester or QA engineer determines how much time, money, and effort it takes to run a particular test in a specific environment. In software test estimation techniques in other words, by evaluating all of the above resources, the tester answers two basic questions:

How long will it take to conduct testing?
What part of the budget will have to be spent on it?

The most of accurate estimation and forecasting helps to avoid exceeding the deadline and budget set for the project. The data obtained as a result of such an assessment will help correctly allocate available resources and optimize the activities of testers to get a high-quality product in the end. It is necessary to understand which resources need to be estimated and why it is so important to evaluate them.

What Should Be Estimated and Why

Estimating software testing efforts involves testing estimation evaluating four parameters:

the time required;
labor and technical resources;
the cost of the planned tests;
knowledge, skills, and abilities of specialists.

Time

One of the indicators of successful testing is completing the process on time. This component of the testing effort can be measured in person-hours. The data obtained will show just how much effort and long the testing will take and help the project manager create the optimal work schedule. Its implementation guarantees timely completed testing and, as a result, increased customer loyalty.

Labor and technical resources

The fulfillment of the set task requires a particular list of resources:

technical capabilities;
infrastructure;
testing team of the correct size.
Incorrectly estimated test requirements make its implementation impossible.

Cost

Testing at any stage of software development life cycle requires certain financial costs: equipment, wages for testers, and necessary software. All costs must be agreed upon in advance with the client because most modern projects do not accept ongoing changes in the budget.

Skills of the specialists involved

At this stage, the level of theoretical and practical training is determined. Without the right skills, knowledge, and knacks, the testing process can take longer than expected and increase the project’s cost.

All estimated parameters are closely interconnected. The number of specialists and their skills affect the time required to conduct tests. The duration of estimate testing efforts, in turn, makes adjustments to the project budget. To successfully evaluate software testing efforts, each test team chooses the best method for the assigned tasks.

Testing Effort Estimation Techniques

So how do you evaluate software testing efforts? Today’s QA teams have a wide range of evaluation methods and tools at their disposal. Let’s analyze the most effective and popular ones.

The program evaluation and review technique (PERT)

This technique is based on the use of three options for forecasting the time and money spent: the most optimistic, the most pessimistic, and the most probable period of work on the project.

The most optimistic forecast (variable O): testing is carried out as soon as possible, subject to the proper performance of duties by each team member, the absence of unforeseen circumstances, and the strict adherence to the schedule.
The most pessimistic forecast (variable P): the cost and duration of testing in the worst conditions are calculated. All kinds of failures, violations of deadlines, etc., are allowed.
The most probable scenario (variable M): the timing and budget for conducting tests are estimated considering the normal workflow, possible improvements, and difficulties.

After determining the values of all variables, an estimate (E) is calculated. Calculations are carried out using the formula: E=(O+4M+P)/6.

This technique is good because the result is realistic due to considering all possible scenarios.

The work breakdown structure (WBS)

This technique is based on dividing a large project into smaller components:

project → submodules → functionality → subfunctionality.

The main idea is that it is much easier to evaluate small modules than the entire testing process as a whole. Thus, each tester can be entrusted with a specific testing task. The assessment will certainly cover the whole process of conducting tests. It also eliminates the possibility of duplication of tasks by different quality control specialists.

Test Case Point (TCP)

In the TCP technique, ready-made test cases are used to evaluate efforts. The evaluation of total effort is performed in stages:

Classification of cases according to the degree of complexity (low, medium, high, very high).
Effort estimation for test cases of each difficulty level.

This method is suitable for situations where test cases have already been written, or their complexity is a function point of known value.

The Wideband Delphi technique

Delphi technique involves active communication of all test participants. It is based on a survey of all persons involved in this process to determine the time and labor costs of conducting tests.
At the first stage of discussion, the variety of answers is usually wide. To come to a common denominator, survey participants are asked to correct their views based on their colleagues’ responses.

The estimation process is considered complete when the number of versions is reduced so that it becomes possible to derive an average value.

The advantage of the method is that experienced specialists participate in the survey, and their answers are completely anonymous.

Function points Analysis (FPA)

This technique is based on estimating the project’s size, which will subsequently determine its final estimate of cost. To apply the method, testers must be aware of the following variables:

Non-adjusted FPC.
Total degree of influence.
Value adjustment factor.

Knowing the values of all these variables, the formula used above example to calculate adjusted FPC is applied:

Adjusted FPC=Non-adjusted FPC*VAF (Value adjustment factor).

This method allows you to evaluate projects of any size, and its results are understandable to the quality control and customers.

Agile Estimation

The name of this estimation technique speaks for itself. Whereas in previous methods, defining project requirements preceded budget and deadline planning, in the case of the current technique, the evaluation process has become more agile and supports continuous delivery. It is very relevant in today’s rapidly changing IT environment.

Agile Estimation considers both previous experience with software testing estimation and current information. Planning Poker and T-Shirt Sizing are the best known agile methods for evaluating software testing efforts.

The experience-based approach

To apply this technique to evaluating software testing efforts, the test team must have extensive relevant experience. The evaluation is done based on those projects that have already been tested by the team before.

There are two methods for applying the experience-based approach:

analog: testers take the assessment of a similar project as a basis;
expert: if the project differs from the previous ones, the assessment is based on the expert experience of the team members.

Unfortunately, this technique cannot be applied to unique projects.