AQA Virtual Summit/MeetupsTest Automation MethodologiesTypes of Automation Testing

Svetlana Grinevich. Artificial Intelligence QA – Testing Automation with AI

Published 3 March 2022. Updated with a short answer, key takeaways and a FAQ; the talk text is unchanged. Published by Zapple.Tech, founded by the founders of CodeceptJS and Codeception, open-source testing frameworks.

Watch on YouTube
Short answer

In this talk, Svetlana Grinevich shows how machine learning can speed up testing and replace cumbersome test scripts. She builds a platform in two phases: detecting elements on a page with a convolutional neural network, then acting on them with a natural language processing model that turns BDD steps into Selenium actions.

Key takeaways
  • Why DOM-based scripts break when the interface changes
  • Two phases: element detection and action
  • Training a discovery model with a convolutional neural network
  • A natural language model that turns BDD steps into Selenium actions

Svetlana Grinevich has extensive management experience focusing on quality control and software testing. She received her master’s degree in Computer Programming from the University of Belarus, and today she has devoted herself to machine learning frameworks and IoT platforms. She talked about using machine learning to speed up testing, moving away from cumbersome test scripts, and building a natural language architecture test.

Large and small companies are constantly looking for ways to reduce costs and improve test reliability. As a result, testers often turn to AI and machine learning.

Let’s look at some of the major benefits of adding AI to the software testing process.

Software Testing Challenges

  • Automated testing is based on the DOM structure, causing traditional test scripts to break anytime the user interface changes.
  • It takes a lot of effort to cover the huge number of layers that need to be tested.

Functional testing is a set of scenarios. And each scenario is a set of pages that are a set of elements. We will create two phases on our platform. First phase: detection of elements on the page. Second phase: action with these elements. For the first one, we will make a discovery model training with a convolutional neural network, and for the second, we will use a deep learning model with natural language processing.

Training a Discovery Model with a Convolutional Neural Network (CNN)

To start building this model, you must:

  • Work with Python;
  • Install Tensorflow (a free and open-source software library for machine learning and AI);
  • Install ImageAI (additional library, to simplify the code);
  • Use Pillow and PyTesseract to make element discovery easier.

Step-by-step model building:

  • Creation of a dataset with subsequent training.
  • Creating a model to use to analyze the data.

Building a Deep Learning Model with Natural Language Processing (NLP)

It is now a trendy NLP machine learning model. It is used in various areas such as Botchat, etc.

To build an NLP model, you will need to install the Spacy library, the en_core_web_sm trained pipeline for the English language and the Pandas software library.

To build a deep learning model, you need to:

  • Collect BDD steps;
  • Classify the steps;
  • Translate the bdd steps into selenium actions.

BDD steps are a collection of really different BDD steps found in any storage.

Applied methods of natural language processing:

  • lemmatization – the meaning of words based on the dictionary;
  • stemming – extracting “rot” from words;
  • named entity recognition;
  • stop words – repeated words that do not help the context;
  • dependency parsing;
  • word and sentence similarity and tokenization.

Questions about this talk

Why do traditional test scripts break?

Svetlana Grinevich says automated testing based on the DOM structure makes traditional test scripts break whenever the user interface changes. It also takes a lot of effort to cover the huge number of layers that must be tested. Companies look for ways to reduce costs and improve test reliability, and as a result testers often turn to AI and machine learning.

How does the discovery model find elements on a page?

Svetlana Grinevich trains a discovery model with a convolutional neural network. The work uses Python, TensorFlow, ImageAI, Pillow and PyTesseract, and proceeds by creating a dataset and training a model to analyze the data. This is the first of two phases on her platform.

How are BDD steps turned into Selenium actions?

For the second phase, Svetlana Grinevich builds a deep learning model with natural language processing. She collects and classifies BDD steps and translates them into Selenium actions, using spaCy and Pandas and methods such as lemmatization, stemming, named entity recognition, dependency parsing and similarity.

Request a QA audit quote

Tell us what you ship and what worries you.

What happens after you send this

  1. You send the form: your situation and what you need tested, then your name and work email.
  2. Within one business day, we reply with a suggested first step, its scope, timeline and a quote.
  • Fixed scope
  • You decide after each step
  • The tests live in your repository
  • An NDA signed before access

“They helped us to close the positions and consult our team how to proceed further. High-quality quick feedbacks with the deep checking of the required tasks, professional approach, great understanding of the company's technologies and product, detailed reports.”

Kristina Lavrentieva, Manager at keepgo.com, Israel
  • Rated 4.9 out of 5 on Clutch, from 7 client reviews
  • 42 completed test automation and infrastructure projects
  • Founded by the founders of CodeceptJS and Codeception, open-source testing frameworks

Request a QA audit quote

    What is closest to your situation?
    Optional. The product, the platforms, how often you release, what broke last.

    You decide after our reply whether to go ahead.

    Prefer to talk first? Book a call

    One-week QA audit · written report

    Request a QA audit quote