Visual Test Automation Tools


In agile software development world, we are dealing with many test tasks such as user story testing, exploratory testing, check-list based testing, regression testing, performance testing, security testing in each sprint. Besides these testing activities, one of the test type which is considerably getting crucial, is visual regression testing.

Please, Click Here and Join Visual Test Automation Survey!

Visual regression testing focuses on to check visual contents, page layout, visual animations, and responsive design of a website/mobile app. Because of the limits of human vision, human-based visual regression testing is generally error-prone and cumbersome. Hence, automation is inevitable. It enables us to run the tests much more precisely in a short time period. Also, it saves us a significant amount of time to deal with other manual test activities in each sprint. There are several open-source and commercial solutions for visual regression testing. The well-known tools are:

Commercial Tools:

Applitools Eyes: Checks both page layout and visual content. It is very simple to use Eyes along with Selenium Webdriver and it has significantly smart comparison algorithm.

Screener.io: End-to-end visual + functional testing as a service. No coding required.

Percy.io: It supports Python Selenium, Ember.JS, Ruby Capybara/CLI and it can be used for responsive testing and visual regression testing for specific pages and elements. Also, it can be easily integrated a wide range of CI tools.

Open-Source Tools:

PhantomCSS: It makes visual comparisons between the baseline images (expected) with the images which are taken during test executions(actual). It is a combination of PhantomJS (Headless Webkit Browser), CasperJS (JS based navigation and scripting tool), and Resemble.js (Comparison Engine)

WebdriverCSS: It is an automatic visual regression-testing tool for WebdriverIO (Selenium 2.0 bindings for NodeJS).

Huxley: It does visual regressions in Web applications. Before it is supported by Facebook and Instagram but now this support no longer exists.

Wraith: Wraith uses a headless browser to create screenshots of web pages on different environments (or at different moments in time) and then creates a diff of the two images; the affected areas are highlighted in blue. It requires ImageMagick for visual comparisons, PhantomJS(WebKit) or SlimerJS(Gecko) for a headless browser, and CasperJS for interrogation/navigation/manipulation web elements. It is developed by BBC.

Shoov: Simulating any number of device/OS/Browser combinations accessing websites 24/7 in multiple, programmable scenarios. It visually detects issues as they rise and sends alerts.

Gemini: It is created by Yandex and it is especially useful to UI libraries developers. It has abilities to test separate sections, position and sizes of elements, and visual differences.

Galen Framework: It is a automated layout testing tool. It will be a good option to test responsive websites. It runs well in Selenium Grid. You can set up your tests to run in a cloud like Sauce Labs or BrowserStack. It has basic syntax, JavaScript and JAVA API support to use with Selenium Webdriver.

SikuliX: SikuliX is screenshot based GUI Automation tool. It is not a visual regression tool. It automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI’s internals or the source code of the application or web page you want to act on. I also posted a quick start guide to describe how to use SikuliX.

As you see that there are many tools available in the market for visual test automation. However, the question is that can we do visual test automation with Selenium Webdriver? In this article you can find a sample visual test automation solution by using Selenium Webdriver, ImageMagick and AShot.

In that article, We will take screenshots by AShot, interrogate/navigate/manipulate web elements by using Selenium Webdriver and visually compare the baseline and actual images with a given precision ratio by ImageMagick. This tool combination is fully open-source and after that article, you will start to do visual test automation with Selenium Webdriver without any cost.

Also there are many visual test automation tools exists in the market. You can make google and go their github pages for further information.

FBSnapshotTestCase
Needle
Rspec Page Regression
Pix-Diff
Selenium Visual Diff
Vizregress
VisualCeption
Specter
Fighting Layout Bugs
dpdxt
Huxley
SiteEffect
Shoov
BaskstopJS
CSSCritic
Grunt Photobox
VIFF
GreenOnion
Kobold
CSS Visual TestAsd
Snap And Compare
Grunt-Vigo
Hardy
Browsershots
Screenbeacon
VisualReview

My conference talk on Visual Test Automation at SeleniumCamp

javafx

Database Operations in JavaFX

By Onur Baskirt / Apr 1, 2016 / 56 Comments
Before started this section, please check the first article and learn How to Start JAVAFX! http://www.swtestacademy.com/getting-started-with-javafx/ At first, part of JavaFX tutorial series, we created a sample JavaFX project, designed the draft version of the UI and set up an...
rest assured

REST API Testing with Rest Assured

By Onur Baskirt / Mar 8, 2016 / 32 Comments
Outline In this post, I will explain what is API and API testing, what is the difference between SOAP and REST services, and how to test REST APIs with Rest Assured Library. What is API? API stands for Application Programming...
extentreports

How to Write Effective CSS Locators

By Onur Baskirt / Oct 1, 2017 / 0 Comments
Hi all, in this tutorial, I will describe you how to write effective CSS locators to interrogate web elements for your automation projects. As a rule of thumb, your interrogation strategy should be in below order: First try to use...
extentreports

Selenium-11: Execute JavaScript with JavascriptExecutor

By Onur Baskirt / Jan 27, 2016 / 15 Comments
Outline Sometimes we cannot handle some conditions or problems with Webdriver, web controls don't react well against selenium commands. In this kind of situations, we use Javascript. It is useful for custom synchronizations, hide or show the web elements, change...
javafx

Getting Started with JavaFX

By Onur Baskirt / Mar 25, 2016 / 0 Comments
When I started to work in my current position, one of my task is to do manual operations for campaign products  every week. After the second week, I thought that I have to automate this task using a GUI based...
extentreports

How to Write Smart XPath Locators

By Onur Baskirt / Sep 24, 2017 / 4 Comments
Hi all, in this tutorial, I will describe you how to write smart and non-brittle XPath locators. When we write our test scripts, we generally prefer to use id, name, class, etc. these kinds of locators. However, sometimes we could not...
extentreports

Selenium-1: Quick Start to Automation with Selenium WebDriver & JAVA & JUnit & Maven & IntelliJ

By Onur Baskirt / Sep 8, 2015 / 26 Comments
Outline Selenium Webdriver is the most popular open source web test automation framework across wide range of browsers and platforms. In this tutorial you will learn how to do web test automation with Selenium Webdriver and the related tools. Audience...
page object model

Page Object Model with C#

By Ege Aksoz / Jun 18, 2017 / 8 Comments
In the previous tutorial, we’ve taken the initial steps and entered the world of automated testing. We also wrote our first automated test. From this point on, since we are not just going to write one test, we need to...
extentreports

How to Select a Date From DatePicker Using Selenium

By Onur Baskirt / Aug 13, 2016 / 6 Comments
When you need to automate a airway, hotel, or similar websites you need to deal with Datepickers and some times it is a little bit cumbersome to select a specific date on the Datepicker or calendar.  In this post, I...
extentreports

Selenium Webdriver Performance testing with Jmeter and Selenium Grid

By Ozgur Kaya / Aug 9, 2016 / 2 Comments
In this post, we will complete Selenium Webdriver Performance testing scenario using Jmeter and Selenium Grid. 1- Install Java 7 or later If necessary https://java.com/tr/download/ 2- Download latest Jmeter version 3.0 or higher. http://jmeter.apache.org/download_jmeter.cgi 3- Download Jmeter PluginsManager JAR file and...
By | 2017-03-05T00:59:53+00:00 January 18th, 2017|Test Automation, Visual Testing, Web Automation|0 Comments

About the Author:

Onur Baskirt
Onur Baskirt is a senior IT professional with 10+ years of experience. He worked at Bahçesehir University, ST Microelectronics, Huawei and Ericsson as research assistant, design verification engineer and software test leader. Also, he worked as software test leader and software operations manager at Turkey's biggest technology retailer, Teknosa. Now, he is working as Head of Software Testing and Manager of two Development Teams at Kariyer.net. His current research areas are technical software testing, programming, and computer science. Formerly, he has several research works and papers on digital chip design & verification. His hobbies are sport, dancing, traveling, and nutrition. You can find detailed information about him on his linked-in page.

Leave A Comment