In this article, I explained how to do automated mobile visual testing with Applitools Eyes tool. In the web world, there are many solutions about visual regression testing such as PhantomCSS, Webdrivercss, Huxley, Wraith, and much more. With Applitools Eyes you can do visual comparison both web, mobile web and mobile native apps. You can use Eyes class’s functions in your selenium codes.

For a long time, I could not find enough time to write a comprehensive article. This made me very upset because as swtestacademy.com our core value is knowledge sharing. We are so happy when we share our knowledge and experience with the community.

I explained how to do Appium setup and how to write mobile automation code in this article. That’s why I do not want to go into those details in this article. I want to focus how to integrate Applitools Eyes with Appium and do a simple visual mobile test automation.

First, open Genymotion Emulator and create a new device and set its name as “GoogleGalaxyNexus”. Installation of Genymotion is described in this article and in this link.

applitools

applitools

Start GenyMotion emulator.

genymotion

Then, open Appium and check settings.

We will use sample bitbar.apk. First, you need to put this apk under C:\ApplitoolsApk\BitbarSampleApp.apk folder and then set this path in Appium settings.

appiumsettings

Set Server Adress as “127.0.0.1” and Port as “4723”

appiumsettings

And Start Appium Server.

appium

appium

Then, drag and drop BitbarSampleApp.apk to your emulator.

bitbarapk

Now, we can write our test code.

Open IntelliJ and click File -> New Project and select Maven and click Next.

Then, write:

GroupId: AppliToolsEyesDemo

ArtifactId: AppliToolsEyesDemo

And click Next.

Write “AppliToolsDemo” as project name and click “Finish”.

Then, open pom.xml and copy-paste below codes.

Project GitHub Page: https://github.com/swtestacademy/MobileVisualTestingApplitoolsEyes

Click test -> java and right click and create a new Java Class and give “AppliToolsDemo” as a class name then click OK button.

Right click “AppliToolsEyesDemo” folder and create a new File and give “TestNG.xml” as name.

After that, open “TestNG.xml” file and write below code into it.

Now, we can write our test code but before that I want to summarize our test scenario:

  • Open BitbarSampleApp.apk
  • Visually check that start screen is visually as expected?
  • Click second option “Use Testdroid Cloud”
  • Visually check that Second Option is clicked?
  • Write “SW Test Academy” to text bar
  • Click Answer button
  • Wait untile the second page appear
  • Visually check that the answer is correct
  • Close the driver.

As you can see above scenario, we will make 3 visual comparisons. When you run your test, at first, AppliTools Eyes takes screenshots of the check points and put them under our test as baseline images. When you run your tests second, third, fourth times and so on, AppliTools Eyes’s “eyes.checkWindow()” method will compare the actual screen with the baseline screen with given comparison match level as STRICT, EXACT, LAYOUT, CONTENT. In this article, we will make a STRICT comparison.

For match levels please read below pages:

http://support.applitools.com/customer/portal/articles/1664977-what-are-the-different-match-comparison-levels-in-applitools-eyes-

http://support.applitools.com/customer/portal/articles/2088359-match-levels

Now, Let’s login to Applitools cloud test management panel from http://www.applitools.com/ If you do not have an account, first you need to create a new account.

When you logged-in https://eyes.applitools.com at right corner, you will reach your profile details and you will get your API key from here.

applitools-eyes

At left pane, you can see your test runs highlighted with green when they passed and red when they failed. Also, you can all check points when they passed they comprises of green + sing and when they failed, you see red + sign. (Note: If you are a color-blind it is not easy to use this panel! Dear Applitools, please consider this comment.)

applitools

applitools

When you open an image, you can reach the image control panel and in this panel, you can many operations as shown below image.

applitools

I think this is enough for Applitools Eyes control panel. Now, let’s start to write our test code. I wrote detailed inline comments in the code.

Before running our code please make sure that your Appium server and Genymotion emulator running. If these preconditions satisfied, then you can run your test code. After that, Bitbar application will open and a new bitbar test created in Applitools control panel as shown below.

applitools-12

At first run, Applitools will save all images at checkpoints as baseline images.

applitools

We can also add some ignored comparison regions with rectangular selection. It is shown below.

applitools

For the second run, if you test will fail, Applitools Eyes will highlight problematic mismatches and your test will turn red color.

applitools-15

When your test passed, your test is highlighted as red as all the checkpoints are displayed without mismatch error.

applitools

Thats all 🙂