Can we overestimate the importance of automation testing, especially if we strive to develop a genuinely high-quality product? Therefore, an equally important issue is optimizing this process and its structure.
Therefore Mike Cohn proposed the concept of the test automation pyramid. He described this method in his Succeeding With Agile. This concept includes three categories of software tests that brought significant benefits in development.
So the developers began to produce higher quality IT products with less energy and time consumption thanks to the software testing pyramid. But is this statement enough to describe the whole concept? Let’s understand the concept of pyramid testing in more detail!
Table of Contents
Introduction to Test Automation Pyramid
So, in short, the pyramid of testing is a framework that can help you create high-quality software while reducing the time for testing code or bug detection. It can also help you create a quality test suite.
It’s is a metaphor that refers to grouping software tests into different levels. It also gives an idea of how many tests should be in each group.
Mike Cohn’s original test pyramid consists of three levels (from bottom to top):
- Unit tests.
- Integration tests
- User interface (UI/exploratory) tests.
There are also opinions that this concept is insufficient. Someone even claims that these aspects are unfinished, making the test automation pyramid seem oversimplified.
For instance, the test automation pyramid has not even three but five main levels. So, end-to-end testing and Exploratory / Manual Tests are also added to the top branch. But it all covers the same top level.
Three Levels of Test Automation Pyramid Concept
If we consider in detail, we are talking about three basic levels of the pyramid. By the way, it has some pattern: the higher the level, the less testing. Therefore, the lower level of the pyramid involves many tests, but they are automated. At the same time, the top-level is for manual testing and scripting.
The Base Level of the Pyramid Testing: Unit Tests
These tests are small and should ensure that isolated blocks of code work properly. They test only one variable without relying on any additional ones. What benefits and features hides this software testing pyramid level:
- Automation tasks are not closed until these scripts are run on the implemented functionality.
- They are developing simultaneously as unit tests force developers to think about the problem they are solving and any edge cases they
- may encounter.
- The tests are detailed and can help pinpoint the defect.
- Run times are incredibly fast because they don’t have to rely on any user interface or external systems like a database or API.
- They are inexpensive, easy to write, easy to maintain.
Unit tests are best used for devices on simulators and emulators (virtual devices). It provides the feedback you need, detailed reports, and simple fast and time-saving testing.
The Middle Level: Integration Tests
The first difference between these tests and modular ones is that CI runs them. They also involve the integration of more than one system. So, you have more features and different types of testing to cover manually:
- Developers use this layer to test business logic without using the user interface (UI);
- By testing outside of the user interface, you can test the inputs and outputs of APIs or services without all the complexity that the user interface introduces;
- These tests are slower and more complex than unit tests because they may need to access the database or other components.
This type of testing involves both real and virtual devices, depending on the type of testing. For example, performance testing or security testing is only possible for real devices.
At the same time, acceptance testing is possible for virtual instruments. Therefore, you must strike a balance between real and virtual devices — this will help you create high-quality builds and tests.
The Top Level: User Interface Tests
This category in the QA testing pyramid includes user exploratory testing and user interface testing. It also means that they are performed manually and take more time, but they involve a much smaller number of tests according to the pattern. So, to summarize this level, we can say the following:
- User interface tests should be placed at the top of the pyramid.
- Most of your code and business logic should already be tested to this level;
- Interface tests are written to make sure that the interface itself works correctly;
- UI tests are slower and harder to write and maintain, so you need to keep them to a minimum.
The top of the pyramid of testing also assumes only real devices, not virtual ones. They are necessary for end-to-end and exploratory testing to better simulate real users’ devices, detect defects under various circumstances: application conflict, network problems, other problems.
Inverted Testing Pyramid or Ice Cream Cone
The inverted pyramid is considered not recommended, although this approach is found in practice. There’s no surprise that this concept isn’t popular:
- UI tests should be more automated.
- Long test runs. Runtime takes much longer than other types of tests because it relies on interacting with visual UI elements and doesn’t necessarily have hooks in the source code;
- Difficult to maintain, as user interface tests are difficult to write and very dependent on even the slightest changes;
- More suitable for positive path scenarios. Testing negative paths in end-to-end tests is very costly and time-consuming compared to lower-level tests;
- Waiting to write unit tests until features are complete can result in everyone doing a lot of work multiple times to solve a problem.
It’s more like an anti-pattern that some use to visualize test ratios in end-to-end testing.
What Structure Have These Test Categories
- Experienced developers offer a convenient algorithm for working with unit tests (and not only).
- Setting up test data.
- Calling the method under test.
- Verify that expected results are returned.
There is a good mnemonic for remembering this structure, as the principle of three A’s stands for Arrange, Act and Assert. Other mnemonics rooted in Behavior-Based Development (BDD) may also be used.
This pattern can be applied to other higher-level tests as well.
5 Tips About Testing Pyramid
It is worth noting that the testing pyramid in DevOps is a reliable and practical system. It will speed up your work, contribute to developing a quality product, and avoid many mistakes. Unless, of course, remember a couple of main principles:
- Write tests with different details.
- The higher the level, the less testing.
- Stick to the shape of a pyramid to create a robust, fast, and user-friendly test suite.
- Write small and fast unit tests.
- Write some more general tests.
- And there are very few high-level end-to-end tests that test the application from start to finish.
Also, try to avoid test duplication and ensure that the higher-level test focuses on the part that the lower-level tests cannot cover.
In each case, they ensure that the tests remain light and readable. Also, tests written with this structure tend to be shorter and more expressive.
Final Words
Test automation pyramid streamlines all testing processes by dividing them into three (or five) categories. This framework provides effective time management and gives testers a time-tested blueprint for shaping their projects. It also organizes all testing by levels and purposes.
So, unit testing is a basic level; it is fully automated and is applied on virtual devices. At the same time, the top-level involves manual testing with fewer tests, but on real devices.
It can be a great tool to improve your productivity, offload your team. You only need to write the tests correctly or entrust this question to the pros.
Zappletech is a team of masters of all aspects of automation testing. We have 40+ completed test automation and infrastructure projects for 6 years of experience and keep improving our service. We give you confidence in software quality and clear results without delays.