IRetryAnalyzer – Retry Failed Tests with TestNG


In this article, we will describe and give examples on usage of IRetryAnalyzer. It reruns the Selenium TestNG tests when they are failed. If you work on test automation project, you’d know that the most difficult part of automation is the analysis of test executions. At the end of the execution, you need to analyze failed test cases and try to figure out if there’s any false positive/flaky situation caused by network glitch, time-out or some other problem.

In order to overcome this issue, we suggest you use IRetryAnalyzer interface in your TestNG based projects.

IRetryAnalyzer Class

First of all, you need to create a separate class which implements this IRetryAnalyzer like below example:

In case, you want to decrease or increase the re-run number of test cases, you need to change the maxTry value. In this example, failed test cases will run 3 times till it passes. In case it fails the third time, test execution will stop and TestNG will mark this case as failed.

Using retryAnalyzer attribute in the @Test annotation

Next step is to associate your test cases with IRetryAnalyzer. In order to do this, you need to use the method below.

@Test(retryAnalyzer = Retry.class)
public void testCase() {
}

In TestNG Listeners article I described how to use listener. I will go on with that example and Retry Class in that project. Also I changed SampleTest.java test class as follows.

In above scenario, test1 runs 3 times and it fails during each execution.  In Retry class’s retry method, when count is 3, it is smaller than maxTry and the test goes the else branch and test fails. However, test2 passed at first execution as shown below.

IRetryAnalyzer

Using Retry Class with ITestAnnotationTransformer Interface

Due to static nature of Annotations, recompilation is needed when you want to change values. You can override this behavior at runtime with IAnnotationTransformer listener. IAnnotationTransformer is a TestNG listener which allows you to modify TestNG annotations and configure them further during runtime. 

Transform method is called for every test during test run. We can use this listener for our retry analyzer as shown below:

We need to add also this listener in our TestNG.xml file.

From now on, we do not need to specify “@Test(retryAnalyzer = Retry.class)” this annotation. We can just only use @Test annotation without retryAnalyzer attribute.

At this time, both test methods will be retried 3 times and at the 4th execution they all failed!

In this article, we showed you how to use Retry logic in your TestNG tests in two different ways. You can use both of the methods depends on your needs.

Test Source Code: https://github.com/swtestacademy/TestNGListener/tree/Retry

Best Regards.
Canber Akduygu & Onur Baskirt

javafx

Database Operations in JavaFX

By Onur Baskirt / Apr 1, 2016 / 56 Comments
Before started this section, please check the first article and learn How to Start JAVAFX! http://www.swtestacademy.com/getting-started-with-javafx/ At first, part of JavaFX tutorial series, we created a sample JavaFX project, designed the draft version of the UI and set up an...
extentreports

How to Write Smart XPath Locators

By Onur Baskirt / Sep 24, 2017 / 5 Comments
Hi all, in this tutorial, I will describe you how to write smart and non-brittle XPath locators. When we write our test scripts, we generally prefer to use id, name, class, etc. these kinds of locators. However, sometimes we could not...
extentreports

How to Write Effective CSS Locators

By Onur Baskirt / Oct 1, 2017 / 0 Comments
Hi all, in this tutorial, I will describe you how to write effective CSS locators to interrogate web elements for your automation projects. As a rule of thumb, your interrogation strategy should be in below order: First try to use...
rest assured

REST API Testing with Rest Assured

By Onur Baskirt / Mar 8, 2016 / 32 Comments
Outline In this post, I will explain what is API and API testing, what is the difference between SOAP and REST services, and how to test REST APIs with Rest Assured Library. What is API? API stands for Application Programming...
javafx

Getting Started with JavaFX

By Onur Baskirt / Mar 25, 2016 / 0 Comments
When I started to work in my current position, one of my task is to do manual operations for campaign products  every week. After the second week, I thought that I have to automate this task using a GUI based...
extentreports

Selenium-1: Quick Start to Automation with Selenium WebDriver & JAVA & JUnit & Maven & IntelliJ

By Onur Baskirt / Sep 8, 2015 / 26 Comments
Outline Selenium Webdriver is the most popular open source web test automation framework across the wide range of browsers and platforms. In this tutorial, you will learn how to do web test automation with Selenium Webdriver and the related tools....
extentreports

Selenium-11: Execute JavaScript with JavascriptExecutor

By Onur Baskirt / Jan 27, 2016 / 15 Comments
Outline Sometimes we cannot handle some conditions or problems with Webdriver, web controls don't react well against selenium commands. In this kind of situations, we use Javascript. It is useful for custom synchronizations, hide or show the web elements, change...
page object model

Page Object Model with C#

By Ege Aksoz / Jun 18, 2017 / 8 Comments
In the previous tutorial, we’ve taken the initial steps and entered the world of automated testing. We also wrote our first automated test. From this point on, since we are not just going to write one test, we need to...
extentreports

How to Select a Date From DatePicker Using Selenium

By Onur Baskirt / Aug 13, 2016 / 6 Comments
When you need to automate a airway, hotel, or similar websites you need to deal with Datepickers and some times it is a little bit cumbersome to select a specific date on the Datepicker or calendar.  In this post, I...

Getting Started with RobotFramework on Windows

By Onur Baskirt / Apr 25, 2016 / 12 Comments
What is RobotFramework? RobotFramework is a GENERIC test automation framework for acceptance testing and acceptance test-driven development (ATTD). What it means that you can do web, mobile, desktop and other test automation activities with related test libraries. These libraries can...
By | 2017-10-21T00:02:18+00:00 December 17th, 2016|Selenium, Test Automation, TestNG, Web Automation|4 Comments

About the Author:

Canberk Akduygu
Canberk Akduygu is working as a Technical QA Architect on test automation and performance testing. Formerly, he worked at TEB as QA Project Lead, Ayrotek IBM Business Partner and Yapi Kredi Bank as Java Developer. You can also follow him on Linkedin.

4 Comments

  1. Uday June 23, 2017 at 10:22 am - Reply

    Nice post.

    Really informative

  2. Andrejs August 23, 2017 at 11:59 pm - Reply

    Finally!
    Finally a tutorial with USEFUL examples, not like “System.out.println(“this is @BeforeMethod”);”
    Thank you! Your site is a gem among mounts of garbage that somehow populate the top places in search.

Leave A Comment