Adam Sandman was a programmer from the age of 10 and now is the Founder and CEO at Inflectra Corporation, where he is interested in technology, business, and innovation. At the Test Automation Meetup in March, he talked about the challenges of test automation, testing the best websites in the MOZ 500, and how AI and big data tools can help.
Table of Contents
Why Is Test Automation So Difficult and Unreliable?
Automation is difficult not only to implement but also to maintain. And it is unreliable in the sense that automated tests written today stop working the next day correctly. This gives the feeling that the investment in them will not pay off. These concerns are pretty valid. Therefore, it is so important to understand why this is so and what we can do about it.Web app testing can seem like a uniquely challenging area of testing compared to mobile and desktop app testing. But it’s not all bad. Therefore, it was decided to focus on this type of testing. Moreover, to analyze not just web apps in a static state but to consider them over time. The Moz Top 500 ranking of the most popular apps globally was chosen as the object of analysis.The Moz 500 Top Websites
If you are engaged in digital marketing, you probably know such concepts as page rank, Moz, domain, or page authority. These terms are used in search engine optimization to see how authoritative a website is. Moz helps you identify which sites have the most traffic and rank them based on that metric.As a rule, these are sites of successful companies with the best developers. Among them are Google, LinkedIn, Youtube, Microsoft, WordPress, and others. It was these leading companies that developed the frameworks that are actively used to create apps today.Results of Testing The Moz 500 Top Websites
All 500 sites in this ranking were scanned using some automation tools. The idea was that if you can create reliable, stable tests for these sites, perhaps these methods will work for others.Issues found:- ID’s not guaranteed to be unique. Even in cases where identifiers are unique, they are often dynamic or semi. And then they are almost entirely useless.
- Wide Variety of Attributes in Use. The most popular attributes used are class, href, style, and target.
- Dynamic Classes & Closures. The popularity of classes can become a problem because they are used to style the page, and sometimes they are generated by frameworks and become a giant tree of divs and other elements with lots of random classes. Therefore, trying to use this attribute here is problematic.
- Hidden Frames & Duplicate Elements. The problem is that when you try to click on any item in the menu, it can be duplicated: the old menu still exists, but the new one is right in front of it. It is not evident to the user which menu is relevant.
What Can We Do?
- Testers should ask developers to collaborate and listen to what testers consider helpful changes for test automation.
- Automation Tools should have analysis tools (Spy, query, and analyze multiple DOM trees simultaneously).
- Ideally, the developers and testers should be able to have a common toolset that assists in joint working.
What Can Vendors Do?
If talking about AI, we have not yet reached the point where the machine itself can say: “I am an accountant; I can do the invoice myself.” But what AI/ML can do is provide you with page browsing tools and better automation strategies for:- Detect dynamic IDs.
- Analyze identification power of attributes.
- Analyze classes.
- Identify Anchors.
- Use Machine Learning (branded as AI but not really):
