In agile software development world, we are dealing with many test tasks such as user story testing, exploratory testing, check-list based testing, regression testing, performance testing, security testing in each sprint. Besides these testing activities, one of the test type which is considerably getting crucial, is visual regression testing.
Visual regression testing focuses on to check visual contents, page layout, visual animations, and responsive design of a website/mobile app. Because of the limits of human vision, human-based visual regression testing is generally error-prone and cumbersome. Hence, automation is inevitable. It enables us to run the tests much more precisely in a short time period. Also, it saves us a significant amount of time to deal with other manual test activities in each sprint. There are several open-source and commercial solutions for visual regression testing. The well-known tools are:
Applitools Eyes: Checks both page layout and visual content. It is very simple to use Eyes along with Selenium Webdriver and it has significantly smart comparison algorithm.
Screener.io: End-to-end visual + functional testing as a service. No coding required.
Percy.io: It supports Python Selenium, Ember.JS, Ruby Capybara/CLI and it can be used for responsive testing and visual regression testing for specific pages and elements. Also, it can be easily integrated a wide range of CI tools.
PhantomCSS: It makes visual comparisons between the baseline images (expected) with the images which are taken during test executions(actual). It is a combination of PhantomJS (Headless Webkit Browser), CasperJS (JS based navigation and scripting tool), and Resemble.js (Comparison Engine)
WebdriverCSS: It is an automatic visual regression-testing tool for WebdriverIO (Selenium 2.0 bindings for NodeJS).
Huxley: It does visual regressions in Web applications. Before it is supported by Facebook and Instagram but now this support no longer exists.
Wraith: Wraith uses a headless browser to create screenshots of web pages on different environments (or at different moments in time) and then creates a diff of the two images; the affected areas are highlighted in blue. It requires ImageMagick for visual comparisons, PhantomJS(WebKit) or SlimerJS(Gecko) for a headless browser, and CasperJS for interrogation/navigation/manipulation web elements. It is developed by BBC.
Shoov: Simulating any number of device/OS/Browser combinations accessing websites 24/7 in multiple, programmable scenarios. It visually detects issues as they rise and sends alerts.
Gemini: It is created by Yandex and it is especially useful to UI libraries developers. It has abilities to test separate sections, position and sizes of elements, and visual differences.
SikuliX: SikuliX is screenshot based GUI Automation tool. It is not a visual regression tool. It automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI’s internals or the source code of the application or web page you want to act on. I also posted a quick start guide to describe how to use SikuliX.
As you see that there are many tools available in the market for visual test automation. However, the question is that can we do visual test automation with Selenium Webdriver? In this article you can find a sample visual test automation solution by using Selenium Webdriver, ImageMagick and AShot.
In that article, We will take screenshots by AShot, interrogate/navigate/manipulate web elements by using Selenium Webdriver and visually compare the baseline and actual images with a given precision ratio by ImageMagick. This tool combination is fully open-source and after that article, you will start to do visual test automation with Selenium Webdriver without any cost.
Also there are many visual test automation tools exists in the market. You can make google and go their github pages for further information.
Rspec Page Regression
Selenium Visual Diff
Fighting Layout Bugs
CSS Visual TestAsd
Snap And Compare
My conference talk on Visual Test Automation at SeleniumCamp
Onur Baskirt is a senior IT professional with 15+ years of experience. Now, he is working as a Senior Technical Consultant at Emirates Airlines in Dubai.