Integrate Test Runs with TestRail – JUnit5

In our previous article, we talked about the integration of TestNG-based automation test to TestRail. In this tutorial, I am going to tell you how to integrate JUnit into your test suite.

Tech Stack

• Java
• JUnit

Assumption

We expect you to store the automation test case in TestRail.

Here are the test cases that we keep in the TestRail app.

• Valid Login has 1 as ID
• Invalid Login has 2 as ID

Create Annotation to Associate Test Code with Test Rails ID

We already covered that part in many articles. You can refer to

• https://www.swtestacademy.com/integrate-test-automation-results-with-testrail-testng/ 
• https://www.swtestacademy.com/custom-java-annotations/

Create a Test Suite in TestRail

Firstly, we need to create a different suite for every execution that we execute so we’ll be able to store the result in different execution records. So we should use BeforeClass annotation and create a Test Suite by using TestRail API’s.  You can download the TestRails Java bindings from this site so you’ll have a base structure to reach TestRails’ API’s.

@BeforeClass
public static void createSuite() throws IOException, APIException {
	//Login to API
	client = new APIClient("https://swtestacademy.testrail.io");
	client.setUser("[email protected]");
	client.setPassword("Qwerty_123");
	//Create Test Run
	Map data = new HashMap();
	data.put("include_all",true);
	data.put("name","Test Run "+System.currentTimeMillis());
	JSONObject c = (JSONObject)client.sendPost("add_run/"+PROJECT_ID,data);
	//Extract Test Run Id
	runId = (Long)c.get("id");
}

• First, we create a Client with our username and password.
• Then we use add_run API to create a test run. You need to provide the test project Id to the endpoint and parameters. You can check the docs via this link
  • include_all means include all case
  • name will be the name of your test run for a specific Test Rails project.
• After the suite is created properly, we extract the run id from the response payload.

Now execute this method and see that Test Run is created.

Retrieve Test Case ID from Annotation

Secondly, we need to update the test case’s status after execution, I need to have the id stated in the annotation before we start the execution. By using the below code, you can get the TestRail annotations related to the test and store this value in the test context. Now test context has test run id and test case id. Before annotation is useful for this job.

@Before
public void beforeTest() throws NoSuchMethodException {
	Method m = JUnitProject.class.getMethod(testName.getMethodName());
	if (m.isAnnotationPresent(TestRails.class)) {
		TestRails ta = m.getAnnotation(TestRails.class);
		caseId = ta.id();
	}
}

Update Test Rails Status

Right now, all you need to do is to update the status of the test case related to the testing run that you created earlier. For this you have to use add_result_for_case/suiteId/caseId endpoint. Documentation can be found in the link.

JUnit has an annotation called Rule. The rule is a component that intercepts test method calls and allows us to do something before a test method is run and after a test, method has been run. We fill our HashMap object with the necessary status

@Rule
public final TestRule watchman = new TestWatcher() {
	Map data = new HashMap();

	@Override
	public Statement apply(Statement base, Description description) {
		return super.apply(base, description);
	}
	@Override
	protected void succeeded(Description description) {
		data.put("status_id", SUCCESS_STATE);
	}

	// This method gets invoked if the test fails for any reason:
	@Override
	protected void failed(Throwable e, Description description) {
		data.put("status_id", FAIL_STATE);
	}

	// This method gets called when the test finishes, regardless of status
	// If the test fails, this will be called after the method above
	@Override
	protected void finished(Description description) {
		try {
			client.sendPost("add_result_for_case/" + runId + "/" + caseId, data);
		} catch (IOException e) {
			e.printStackTrace();
		} catch (APIException e) {
			e.printStackTrace();
		}
	}

	;
};

Let’s run the test cases and see what will happen.  One test case passed and one other failed.

Finally, let’s see what happens in TestRail. Test execution created a test run in TestRail. Then it added the test cases and their statuses are updated.

The test run is created, test statuses are updated.

Then, if you open the failed case, you’ll see the error message also.

Hope that article will encourage you to create more integration into your test automation suite.

Source code can be found on GitHub (https://github.com/swtestacademy/TestRailsIntegration)

Happy testing!