Automated Visual Testing with ImageMagick and Selenium Webdriver


In this article, I will present you how to do visual test automation with ImageMagick, Selenium Webdriver, and AShot. This tool and framework combination is fully open-source and when you successfully complete the settings, you can start to do visual testing with them without any cost. 🙂

First, I want to describe the tools:

Selenium Webdriver: We will use webdriver to navigate, interrogate and manipulate the websites.

ImageMagick: ImageMagick is our image processing and comparison tool.

Link: http://www.imagemagick.org/script/index.php

Pure JAVA Interface (im4java): http://im4java.sourceforge.net/

AShot: AShot is our Webdriver Screenshot utility tool. We will capture the screenshots with AShot. It is developed by Yandex and its open source.

Link: https://github.com/yandex-qatools/ashot

Step by Step Installations

 Step-1: Selenium Webdriver

 If you do not know how to start using Selenium Webdriver, please first read this article.

 Step-2: Visual C++ Redistributable Packages for Visual Studio 2013

 Go to this link: https://www.microsoft.com/en-us/download/details.aspx?id=40784 and install it.

c++_redistributable_packages-1

c++_redistributable_packages-2

c++_redistributable_packages-3

c++_redistributable_packages-4

Step-3: ImageMagick

 Go to http://www.imagemagick.org/script/binary-releases.php and install Windows Binary Release, during installation select all options and set its installation directory location to the system path.

It is important to select all options.

image_magick-1

imagemagick

imagemagick

image_magick

imagemagick

How to Setup Project for ImageMagick and Selenium

Test Scenario:

– Open www.kariyer.net

– Add top banner cookie not to see top banner.

– Close Popup if exists

– Close Banner

– Unhide text area (“21.923 Ä°Ĺž Ä°LANI” part) of Uzman Photo

– Hover on the “UZMAN” section on main page.

uzman-1

– Wait for 2 seconds for completion of image (CSS) animation.

uzman-2

– Take a screenshot of the animated photo of this element.

– At first run, save it to the ScreenShots>$Test_Name folder as uzmanBaseline.png and uzmanActual.png.

– At second run, update uzmanActual.png, compare it with uzmanBaseline.png, and put the difference as uzmanDiff.png

– Put all differences on “Differences” folder.

– Verify that if they are similar test passed. If not test failed.

Project Setup:

Open IntelliJ and click File -> New -> Project -> Maven

Then, fill GroupId and ArtifactId as KariyerVisualTest

visual_testing-project-1

Click Next and then give a name to the project as KariyerVisualTest and click Next.

visual_testing-project-2

After that we should modify our pom.xml

Always check libraries latest versions from https://mvnrepository.com

For Selenium:

For AShot:

For IM4Java:

For TestNG:

Test Code

In below code I tried to write detailed comments. If you have trouble something please feel free to add comment. I will try to reply your questions as soon as possible.

BaseTest Class

JSWaiter (For JS and JQuery Synchronization)

KariyerVisualTest (Test Class)

POM.xml:

TestNG.xml

GitHub Link
https://github.com/swtestacademy/VisualAutomationImageMagickSelenium

Test Results:

 First Run:

Second Run (Positive Case):

Third Run (Negative Case):

 I also want to show you fail case. I open uzmanBaseline with photo editor and add some shapes and save it. After that, I rerun the test and I got below results. Test failed and also it highlighted the unmatched parts.

Before starting test I did below modifications on “uzmanBasline.png”

Then, I run the test and got below results:

For masking images with ImageMagick, I suggest you to check below article.
http://techblog.hotwire.com/2016/05/19/image-comparison-in-automated-testing/

javafx

Database Operations in JavaFX

By Onur Baskirt / Apr 1, 2016 / 56 Comments
Before started this section, please check the first article and learn How to Start JAVAFX! http://www.swtestacademy.com/getting-started-with-javafx/ At first, part of JavaFX tutorial series, we created a sample JavaFX project, designed the draft version of the UI and set up an...
rest assured

REST API Testing with Rest Assured

By Onur Baskirt / Mar 8, 2016 / 32 Comments
Outline In this post, I will explain what is API and API testing, what is the difference between SOAP and REST services, and how to test REST APIs with Rest Assured Library. What is API? API stands for Application Programming...
extentreports

How to Write Effective CSS Locators

By Onur Baskirt / Oct 1, 2017 / 0 Comments
Hi all, in this tutorial, I will describe you how to write effective CSS locators to interrogate web elements for your automation projects. As a rule of thumb, your interrogation strategy should be in below order: First try to use...
extentreports

Selenium-11: Execute JavaScript with JavascriptExecutor

By Onur Baskirt / Jan 27, 2016 / 15 Comments
Outline Sometimes we cannot handle some conditions or problems with Webdriver, web controls don't react well against selenium commands. In this kind of situations, we use Javascript. It is useful for custom synchronizations, hide or show the web elements, change...
javafx

Getting Started with JavaFX

By Onur Baskirt / Mar 25, 2016 / 0 Comments
When I started to work in my current position, one of my task is to do manual operations for campaign products  every week. After the second week, I thought that I have to automate this task using a GUI based...
extentreports

How to Write Smart XPath Locators

By Onur Baskirt / Sep 24, 2017 / 4 Comments
Hi all, in this tutorial, I will describe you how to write smart and non-brittle XPath locators. When we write our test scripts, we generally prefer to use id, name, class, etc. these kinds of locators. However, sometimes we could not...
extentreports

Selenium-1: Quick Start to Automation with Selenium WebDriver & JAVA & JUnit & Maven & IntelliJ

By Onur Baskirt / Sep 8, 2015 / 26 Comments
Outline Selenium Webdriver is the most popular open source web test automation framework across wide range of browsers and platforms. In this tutorial you will learn how to do web test automation with Selenium Webdriver and the related tools. Audience...
page object model

Page Object Model with C#

By Ege Aksoz / Jun 18, 2017 / 8 Comments
In the previous tutorial, we’ve taken the initial steps and entered the world of automated testing. We also wrote our first automated test. From this point on, since we are not just going to write one test, we need to...
extentreports

How to Select a Date From DatePicker Using Selenium

By Onur Baskirt / Aug 13, 2016 / 6 Comments
When you need to automate a airway, hotel, or similar websites you need to deal with Datepickers and some times it is a little bit cumbersome to select a specific date on the Datepicker or calendar.  In this post, I...
extentreports

Selenium Webdriver Performance testing with Jmeter and Selenium Grid

By Ozgur Kaya / Aug 9, 2016 / 2 Comments
In this post, we will complete Selenium Webdriver Performance testing scenario using Jmeter and Selenium Grid. 1- Install Java 7 or later If necessary https://java.com/tr/download/ 2- Download latest Jmeter version 3.0 or higher. http://jmeter.apache.org/download_jmeter.cgi 3- Download Jmeter PluginsManager JAR file and...
By | 2017-02-08T10:01:48+00:00 August 29th, 2016|Selenium, Test Automation, Visual Testing, Web Automation|15 Comments

About the Author:

Onur Baskirt
Onur Baskirt is a senior IT professional with 10+ years of experience. He worked at Bahçesehir University, ST Microelectronics, Huawei and Ericsson as research assistant, design verification engineer and software test leader. Also, he worked as software test leader and software operations manager at Turkey's biggest technology retailer, Teknosa. Now, he is working as Head of Software Testing and Manager of two Development Teams at Kariyer.net. His current research areas are technical software testing, programming, and computer science. Formerly, he has several research works and papers on digital chip design & verification. His hobbies are sport, dancing, traveling, and nutrition. You can find detailed information about him on his linked-in page.

15 Comments

  1. sridhar August 30, 2016 at 3:59 pm - Reply

    HI Onur, Thanks for the great article. Can we do the same setup for mac?

  2. Tibor Veres October 17, 2016 at 7:49 pm - Reply

    Hey, I always get this error. PATH is all setup
    org.im4java.core.CommandException: java.io.FileNotFoundException: compareComparison Failed!

  3. Roy March 25, 2017 at 3:49 pm - Reply

    Hi ,
    Can we use this to compare images which are of different height and width ??

    • Roy March 26, 2017 at 7:26 pm - Reply

      Can I get some help??

      • Onur Baskirt
        Onur Baskirt March 28, 2017 at 11:54 pm - Reply

        Hi Roy, you can compare any of the images but if they do not same they will fail. Do you want to compare different sizes of the same image? Would you elaborate the question a bit, please?

          • Onur Baskirt
            Onur Baskirt March 29, 2017 at 11:36 pm

            You can compare any image with any size with ImageMagick.

          • Roy March 30, 2017 at 2:03 pm

            Hi Onur ,
            But I get the below message when I start comparing images of different dimensions : –

            Image1-1350X2834
            Image2-1350X2117

            Comparison Started!
            compare.exe: image widths or heights differ `C:\Visual\ScreenShots\Baseline.jpg’ @ error/compare.c/CompareImagesCommand/995.
            org.im4java.core.CommandException: java.lang.NullPointerExceptionComparison Failed!

      • roy March 29, 2017 at 11:20 pm - Reply

        Hi ,
        I have four images as below : –
        Image 1 – 1368 X 1024
        Image 2 – 1368 X 754
        Image 3 – 967 X 798
        Image 4 – 780 X 467

        My question is can I compare as below ??
        Image 1 vs Image 2 ??
        Image 3 vs Image 4 ??

        Sometimes , it is throwing some exception , because the image dimesions is different(width & Height) .

        Please need your help .

        Thanks ,
        Roy .

  4. […] Aug 29, 2016 … Pure JAVA Interface (im4java): http://im4java.sourceforge.net/ … At second run, update uzmanActual.png, compare it with … If not test failed. …… This class implements the processing of os-commands using a ProcessBuilder. Automated Visual Testing with ImageMagick and Selenium Webdriver […]

  5. Helen October 4, 2017 at 8:49 pm - Reply

    Can I use this if I am currently using Selenium & C#?

Leave A Comment