Automated Mobile Visual Testing with Applitools


In this article, I explained how to do automated mobile visual testing with Applitools Eyes tool. In the web world, there are many solutions about visual regression testing such as PhantomCSS, Webdrivercss, Huxley, Wraith, and much more. With Applitools Eyes you can do visual comparison both web, mobile web and mobile native apps. You can use Eyes class’s functions in your selenium codes.

For a long time, I could not find enough time to write a comprehensive article. This made me very upset because as swtestacademy.com our core value is knowledge sharing. We are so happy when we share our knowledge and experience with the community.

I explained how to do Appium setup and how to write mobile automation code in this article. That’s why I do not want to go into those details in this article. I want to focus how to integrate Applitools Eyes with Appium and do a simple visual mobile test automation.

First, open Genymotion Emulator and create a new device and set its name as “GoogleGalaxyNexus”. Installation of Genymotion is described in this article and in this link.

applitools

applitools

Start GenyMotion emulator.

genymotion

Then, open Appium and check settings.

We will use sample bitbar.apk. First, you need to put this apk under C:\ApplitoolsApk\BitbarSampleApp.apk folder and then set this path in Appium settings.

appiumsettings

Set Server Adress as “127.0.0.1” and Port as “4723”

appiumsettings

And Start Appium Server.

appium

appium

Then, drag and drop BitbarSampleApp.apk to your emulator.

bitbarapk

Now, we can write our test code.

Open IntelliJ and click File -> New Project and select Maven and click Next.

Then, write:

GroupId: AppliToolsEyesDemo

ArtifactId: AppliToolsEyesDemo

And click Next.

Write “AppliToolsDemo” as project name and click “Finish”.

Then, open pom.xml and copy-paste below codes.

Project GitHub Page: https://github.com/swtestacademy/MobileVisualTestingApplitoolsEyes

Click test -> java and right click and create a new Java Class and give “AppliToolsDemo” as a class name then click OK button.

Right click “AppliToolsEyesDemo” folder and create a new File and give “TestNG.xml” as name.

After that, open “TestNG.xml” file and write below code into it.

Now, we can write our test code but before that I want to summarize our test scenario:

  • Open BitbarSampleApp.apk
  • Visually check that start screen is visually as expected?
  • Click second option “Use Testdroid Cloud”
  • Visually check that Second Option is clicked?
  • Write “SW Test Academy” to text bar
  • Click Answer button
  • Wait untile the second page appear
  • Visually check that the answer is correct
  • Close the driver.

As you can see above scenario, we will make 3 visual comparisons. When you run your test, at first, AppliTools Eyes takes screenshots of the check points and put them under our test as baseline images. When you run your tests second, third, fourth times and so on, AppliTools Eyes’s “eyes.checkWindow()” method will compare the actual screen with the baseline screen with given comparison match level as STRICT, EXACT, LAYOUT, CONTENT. In this article, we will make a STRICT comparison.

For match levels please read below pages:

http://support.applitools.com/customer/portal/articles/1664977-what-are-the-different-match-comparison-levels-in-applitools-eyes-

http://support.applitools.com/customer/portal/articles/2088359-match-levels

Now, Let’s login to Applitools cloud test management panel from http://www.applitools.com/ If you do not have an account, first you need to create a new account.

When you logged-in https://eyes.applitools.com at right corner, you will reach your profile details and you will get your API key from here.

applitools-eyes

At left pane, you can see your test runs highlighted with green when they passed and red when they failed. Also, you can all check points when they passed they comprises of green + sing and when they failed, you see red + sign. (Note: If you are a color-blind it is not easy to use this panel! Dear Applitools, please consider this comment.)

applitools

applitools

When you open an image, you can reach the image control panel and in this panel, you can many operations as shown below image.

applitools

I think this is enough for Applitools Eyes control panel. Now, let’s start to write our test code. I wrote detailed inline comments in the code.

Before running our code please make sure that your Appium server and Genymotion emulator running. If these preconditions satisfied, then you can run your test code. After that, Bitbar application will open and a new bitbar test created in Applitools control panel as shown below.

applitools-12

At first run, Applitools will save all images at checkpoints as baseline images.

applitools

We can also add some ignored comparison regions with rectangular selection. It is shown below.

applitools

For the second run, if you test will fail, Applitools Eyes will highlight problematic mismatches and your test will turn red color.

applitools-15

When your test passed, your test is highlighted as red as all the checkpoints are displayed without mismatch error.

applitools

Thats all 🙂

javafx

Database Operations in JavaFX

By Onur Baskirt / Apr 1, 2016 / 56 Comments
Before started this section, please check the first article and learn How to Start JAVAFX! http://www.swtestacademy.com/getting-started-with-javafx/ At first, part of JavaFX tutorial series, we created a sample JavaFX project, designed the draft version of the UI and set up an...
rest assured

REST API Testing with Rest Assured

By Onur Baskirt / Mar 8, 2016 / 32 Comments
Outline In this post, I will explain what is API and API testing, what is the difference between SOAP and REST services, and how to test REST APIs with Rest Assured Library. What is API? API stands for Application Programming...
extentreports

How to Write Effective CSS Locators

By Onur Baskirt / Oct 1, 2017 / 0 Comments
Hi all, in this tutorial, I will describe you how to write effective CSS locators to interrogate web elements for your automation projects. As a rule of thumb, your interrogation strategy should be in below order: First try to use...
extentreports

Selenium-11: Execute JavaScript with JavascriptExecutor

By Onur Baskirt / Jan 27, 2016 / 15 Comments
Outline Sometimes we cannot handle some conditions or problems with Webdriver, web controls don't react well against selenium commands. In this kind of situations, we use Javascript. It is useful for custom synchronizations, hide or show the web elements, change...
javafx

Getting Started with JavaFX

By Onur Baskirt / Mar 25, 2016 / 0 Comments
When I started to work in my current position, one of my task is to do manual operations for campaign products  every week. After the second week, I thought that I have to automate this task using a GUI based...
extentreports

How to Write Smart XPath Locators

By Onur Baskirt / Sep 24, 2017 / 4 Comments
Hi all, in this tutorial, I will describe you how to write smart and non-brittle XPath locators. When we write our test scripts, we generally prefer to use id, name, class, etc. these kinds of locators. However, sometimes we could not...
extentreports

Selenium-1: Quick Start to Automation with Selenium WebDriver & JAVA & JUnit & Maven & IntelliJ

By Onur Baskirt / Sep 8, 2015 / 26 Comments
Outline Selenium Webdriver is the most popular open source web test automation framework across wide range of browsers and platforms. In this tutorial you will learn how to do web test automation with Selenium Webdriver and the related tools. Audience...
page object model

Page Object Model with C#

By Ege Aksoz / Jun 18, 2017 / 8 Comments
In the previous tutorial, we’ve taken the initial steps and entered the world of automated testing. We also wrote our first automated test. From this point on, since we are not just going to write one test, we need to...
extentreports

How to Select a Date From DatePicker Using Selenium

By Onur Baskirt / Aug 13, 2016 / 6 Comments
When you need to automate a airway, hotel, or similar websites you need to deal with Datepickers and some times it is a little bit cumbersome to select a specific date on the Datepicker or calendar.  In this post, I...
extentreports

Selenium Webdriver Performance testing with Jmeter and Selenium Grid

By Ozgur Kaya / Aug 9, 2016 / 2 Comments
In this post, we will complete Selenium Webdriver Performance testing scenario using Jmeter and Selenium Grid. 1- Install Java 7 or later If necessary https://java.com/tr/download/ 2- Download latest Jmeter version 3.0 or higher. http://jmeter.apache.org/download_jmeter.cgi 3- Download Jmeter PluginsManager JAR file and...
By | 2017-01-13T21:33:42+00:00 August 13th, 2016|Appium, Mobile Automation, Selenium, Test Automation, Visual Testing|2 Comments

About the Author:

Onur Baskirt
Onur Baskirt is a senior IT professional with 10+ years of experience. He worked at Bahçesehir University, ST Microelectronics, Huawei and Ericsson as research assistant, design verification engineer and software test leader. Also, he worked as software test leader and software operations manager at Turkey's biggest technology retailer, Teknosa. Now, he is working as Head of Software Testing and Manager of two Development Teams at Kariyer.net. His current research areas are technical software testing, programming, and computer science. Formerly, he has several research works and papers on digital chip design & verification. His hobbies are sport, dancing, traveling, and nutrition. You can find detailed information about him on his linked-in page.

2 Comments

  1. Douglas September 22, 2016 at 5:45 am - Reply

    Awesome tutorial. Thank you so much!

Leave A Comment