JUnit 5 How to Repeat Failed Test

Rerun failed tests is important in test automation projects. In this tutorial, I will explain to you how to repeat JUnit 5 tests when they are failed. Before going into this feature, let’s check JUnit 5’s default repeat test feature which is annotated by @RepeatedTest. It is very similar to TestNG’s “@Test(invocationCount = 5)” annotation. It just repeats the tests with a given repeat count.

First is first, you need to add the dependencies as shown below.

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.1.0</version>
    <scope>test</scope>
</dependency>

Now, we are ready to go to write a simple test to verify JUnit 5 @RepeatedTest annotation.

Here is the sample code snippet.

@RepeatedTest(3)
public void failTest () {
   assertTrue(false);
}

Here is the result:

You can also give a name to your test with @RepeatTest annotation. The first one is LONG_DISPLAY_NAME enum is showed as follows:

@RepeatedTest(value=3, name=RepeatedTest.LONG_DISPLAY_NAME)
public void failTest () {
   assertTrue(false);
}

The result will be like this:

The other one is SHORT_DISPLAY_NAME but as you see it is a default behavior and no need to use this enum.

Also, you can use your custom names. As shown below:

@RepeatedTest(value=3, name=" My failing test {currentRepetition}/{totalRepetitions}")
public void failTest () {
   assertTrue(false);
}

You can also gather repetition info as follows:

@RepeatedTest(2)
void repeatedTestWithRepetitionInfo(RepetitionInfo repetitionInfo) {
   System.out.println("Repetition number:" + repetitionInfo.getCurrentRepetition());
   assertEquals(2, repetitionInfo.getTotalRepetitions());
}

The output will be like this:

Repetition #1
Repetition #2

Honestly, this is a dummy retry annotation. It just retries your tests without checking their results, in order to add some logic and retry the tests when they are failing, you should add rerunner-jupiter dependency in your pom.xml. (Note: This feature is not working when you are using JUnit 5.3.1 parallel test execution feature.)

<dependency>
   <groupId>io.github.artsok</groupId>
   <artifactId>rerunner-jupiter</artifactId>
   <version>LATEST</version>
</dependency>

Then you need to use @RepeatedIfExceptionsTest annotation to run your test when they are failed. This annotation has some extra features.

@RepeatedIfExceptionsTest(repeats = 3)

This annotation retries your test when it fails three times.

@RepeatedIfExceptionsTest(repeats = 5, exceptions = IOException.class)

This annotation reruns your test five times if the test fails. Set IOException.class that will be handled in test @throws IOException – error occurred.

@RepeatedIfExceptionsTest(repeats = 6, exceptions = IOException.class,
          name = “Retry failed test. Attempt {currentRepetition} of {totalRepetitions}”)

This annotation repeats your test 6 times when it fails. You can also set a custom name. Set IOException.class that will be handled in the test. Set formatter for the test. Like behavior as at {@link org.junit.jupiter.api.RepeatedTest}  @throws IOException – error occurred

@DisplayName(“Test Case Name”)
@RepeatedIfExceptionsTest(repeats = 100, minSuccess = 4)

This annotation repeats the test 100 times with minimum four consecutive successful run then stops the test execution, disables all remaining repeats.

Here are the basic usage and result of this Retry feature.

@RepeatedIfExceptionsTest(repeats = 3)
public void failTest () {
   assertTrue(false);
}

There is also another library available it is called as junit-pioneer but junit-reruner provides more features. You can try both of them and add your automation suites.

Junit-pioneer dependency is shown below:

<dependency>
   <groupId>org.junit-pioneer</groupId>
   <artifactId>junit-pioneer</artifactId>
   <version>0.2.2</version>
   <scope>test</scope>
</dependency>

And its usage is as follows:

@RepeatFailedTest(3)
public void failTest () {
   assertTrue(false);
}

Thanks.
Onur Baskirt