In this article, we will describe and give examples on usage of IRetryAnalyzer. It reruns the Selenium TestNG tests when they are failed. If you work on test automation project, you’d know that the most difficult part of automation is the analysis of test executions. At the end of the execution, you need to analyze failed test cases and try to figure out if there’s any false positive/flaky situation caused by network glitch, time-out or some other problem.

In order to overcome this issue, we suggest you use IRetryAnalyzer interface in your TestNG based projects.

IRetryAnalyzer Class

First of all, you need to create a separate class which implements this IRetryAnalyzer like below example:

In case, you want to decrease or increase the re-run number of test cases, you need to change the maxTry value. In this example, failed test cases will run 3 times till it passes. In case it fails the third time, test execution will stop and TestNG will mark this case as failed.

Using retryAnalyzer attribute in the @Test annotation

Next step is to associate your test cases with IRetryAnalyzer. In order to do this, you need to use the method below.

@Test(retryAnalyzer = Retry.class)
public void testCase() {
}

In TestNG Listeners article I described how to use listener. I will go on with that example and Retry Class in that project. Also I changed SampleTest.java test class as follows.

In above scenario, test1 runs 3 times and it fails during each execution.  In Retry class’s retry method, when count is 3, it is smaller than maxTry and the test goes the else branch and test fails. However, test2 passed at first execution as shown below.

IRetryAnalyzer

Using Retry Class with ITestAnnotationTransformer Interface

Due to static nature of Annotations, recompilation is needed when you want to change values. You can override this behavior at runtime with IAnnotationTransformer listener. IAnnotationTransformer is a TestNG listener which allows you to modify TestNG annotations and configure them further during runtime. 

Transform method is called for every test during test run. We can use this listener for our retry analyzer as shown below:

We need to add also this listener in our TestNG.xml file.

From now on, we do not need to specify “@Test(retryAnalyzer = Retry.class)” this annotation. We can just only use @Test annotation without retryAnalyzer attribute.

At this time, both test methods will be retried 3 times and at the 4th execution they all failed!

In this article, we showed you how to use Retry logic in your TestNG tests in two different ways. You can use both of the methods depends on your needs.

Test Source Code: https://github.com/swtestacademy/TestNGListener/tree/Retry

Best Regards.
Canber Akduygu & Onur Baskirt