Visual Test Automation Tools

In agile software development world, we are dealing with many test tasks such as user story testing, exploratory testing, check-list based testing, regression testing, performance testing, security testing in each sprint. Besides these testing activities, one of the test type which is considerably getting crucial, is visual regression testing.

Please, Click Here and Join Visual Test Automation Survey!

Visual regression testing focuses on to check visual contents, page layout, visual animations, and responsive design of a website/mobile app. Because of the limits of human vision, human-based visual regression testing is generally error-prone and cumbersome. Hence, automation is inevitable. It enables us to run the tests much more precisely in a short time period. Also, it saves us a significant amount of time to deal with other manual test activities in each sprint. There are several open-source and commercial solutions for visual regression testing. The well-known tools are:

Commercial Tools:

Applitools Eyes: Checks both page layout and visual content. It is very simple to use Eyes along with Selenium Webdriver and it has significantly smart comparison algorithm. End-to-end visual + functional testing as a service. No coding required. It supports Python Selenium, Ember.JS, Ruby Capybara/CLI and it can be used for responsive testing and visual regression testing for specific pages and elements. Also, it can be easily integrated a wide range of CI tools.

Open-Source Tools:

PhantomCSS: It makes visual comparisons between the baseline images (expected) with the images which are taken during test executions(actual). It is a combination of PhantomJS (Headless Webkit Browser), CasperJS (JS based navigation and scripting tool), and Resemble.js (Comparison Engine)

WebdriverCSS: It is an automatic visual regression-testing tool for WebdriverIO (Selenium 2.0 bindings for NodeJS).

Huxley: It does visual regressions in Web applications. Before it is supported by Facebook and Instagram but now this support no longer exists.

Wraith: Wraith uses a headless browser to create screenshots of web pages on different environments (or at different moments in time) and then creates a diff of the two images; the affected areas are highlighted in blue. It requires ImageMagick for visual comparisons, PhantomJS(WebKit) or SlimerJS(Gecko) for a headless browser, and CasperJS for interrogation/navigation/manipulation web elements. It is developed by BBC.

Shoov: Simulating any number of device/OS/Browser combinations accessing websites 24/7 in multiple, programmable scenarios. It visually detects issues as they rise and sends alerts.

Gemini: It is created by Yandex and it is especially useful to UI libraries developers. It has abilities to test separate sections, position and sizes of elements, and visual differences.

Galen Framework: It is a automated layout testing tool. It will be a good option to test responsive websites. It runs well in Selenium Grid. You can set up your tests to run in a cloud like Sauce Labs or BrowserStack. It has basic syntax, JavaScript and JAVA API support to use with Selenium Webdriver.

SikuliX: SikuliX is screenshot based GUI Automation tool. It is not a visual regression tool. It automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI’s internals or the source code of the application or web page you want to act on. I also posted a quick start guide to describe how to use SikuliX.

As you see that there are many tools available in the market for visual test automation. However, the question is that can we do visual test automation with Selenium Webdriver? In this article you can find a sample visual test automation solution by using Selenium Webdriver, ImageMagick and AShot.

In that article, We will take screenshots by AShot, interrogate/navigate/manipulate web elements by using Selenium Webdriver and visually compare the baseline and actual images with a given precision ratio by ImageMagick. This tool combination is fully open-source and after that article, you will start to do visual test automation with Selenium Webdriver without any cost.

Also there are many visual test automation tools exists in the market. You can make google and go their github pages for further information.

Rspec Page Regression
Selenium Visual Diff
Fighting Layout Bugs
Grunt Photobox
CSS Visual TestAsd
Snap And Compare

My conference talk on Visual Test Automation at SeleniumCamp

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.