How to Use TestNG Listeners

In this article I will explain and show an example of TestNG Listeners. Actually, TestNG provides many types of listeners but we generally use some of them. Thats why I did not give all type of TetsNG Listeners but you can find all of them in this article.

A TestNG listener always extends org.testng.ITestNGListener and TestNG provides us below listener types:

  • IExecutionListener
  • IAnnotationTransformer
  • ISuiteListener
  • ITestListener
  • IConfigurationListener
  • IMethodInterceptor
  • IInvokedMethodListener
  • IHookable
  • IReporter

In this article I will show the most common Listener type  ITestListener.

In our tests we implement ITestListener and use its methods. It has following methods and events:

  • onStart is invoked after the test class is instantiated and before any configuration method is called.
  • onTestSuccess is invoked on success of a test.
  • onTestFailure is invoked on failure of a test.
  • onTestSkipped is invoked whenever a test is skipped.
  • onTestFailedButWithinSuccessPercentage is invoked each time a method fails but is within the success percentage requested. Don’t worry I will explain it with example ;)
  • onFinish is invoked after all the tests have run and all their Configuration methods have been called.

Now, Lets do an example and examine how our Listener works on specific events.

We have three JAVA classes; BaseTest, Listener and SampleTest.

BaseTest:

In BaseTest, I used @BeforeMethod and @AfterMethod annotations. @BeforeMethod is triggered before the test and @AfterMethod is triggered after the test finishes.

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;

/**
 * Created by ONUR on 11.12.2016.
 */
public class BaseTest {

    @BeforeMethod
    public void BeforeMethod(){
        System.out.println("I am in Before Method! Test is starting!");
    }

    @AfterMethod
    public void AfterMethod(){
        System.out.println("I am in After Method! Test is ending!");

    }
}

SampleTest:

SampleTest is our test class,

test1 passes without any problem.

test2 always fails because it expects RuntimeExceptions but in the test method there is not any runtime exception.

test3 always skipped because it throws SkipException.

test4 is a little bit different than others.

I set %60 success percentage and 5 invocation count.  Thus, it runs 5 times and if 3 times it passes, test method is considered as passed! In the code, test method fails 2 times and passes 3 times so success ratio is (3/5)*100=60 thus we can consider this test method as passed! As you can see below picture, all the 5 tests of test4() method is shown as passed! Because we are in success ratio %60.

But, when we increase the success ratio to %80, this time one of the tests will be shown as failed! Because, %80 success ratio means “4 times tests pass” but our tests pass 3 times. Thus one of them is shown as failed.

Now, let’s set the success ratio to %100 (it is the default ratio). If all test pass, we can reach %100 success ratio but in our test method, 3 of our test pass and two of them fail. Thus, we can not reach %100 success ratio and we will see two failed tests as a result.

import org.testng.Assert;
import org.testng.SkipException;
import org.testng.annotations.Test;


/**
 * Created by ONUR on 11.12.2016.
 */
public class SampleTest extends BaseTest{


    @Test
    public void test1() {
        System.out.println("I am in test1 test method and it will pass.");
    }

    @Test(expectedExceptions=RuntimeException.class)
    public void test2() {
        System.out.println("I am in test2 test method and it will fail.");
    }

    @Test
    public void test3() {
        throw new SkipException("Skipping the test3 test method!");
    }

    private int i=0;
    @Test(successPercentage=60, invocationCount=5)
    public void test4() {
        i++;
        System.out.println("test4 test method, invocation count: " + i);
        if (i == 1 || i == 2) {
            System.out.println("test4 failed!");
            Assert.assertEquals(i, 8);
        }
    }

}

Listener:

In listener class, I implemented ITestListener class and I explained its method.

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;

/**
 * Created by ONUR on 11.12.2016.
 */
public class Listener extends BaseTest implements ITestListener {


    @Override
    public void onTestStart(ITestResult iTestResult) {
        System.out.println("I am in onTestStart method " +  getTestMethodName(iTestResult) + " start");
    }

    @Override
    public void onTestSuccess(ITestResult iTestResult) {
        System.out.println("I am in onTestSuccess method " +  getTestMethodName(iTestResult) + " succeed");
    }

    @Override
    public void onTestFailure(ITestResult iTestResult) {
        System.out.println("I am in onTestFailure method " +  getTestMethodName(iTestResult) + " failed");
    }

    @Override
    public void onTestSkipped(ITestResult iTestResult) {
        System.out.println("I am in onTestSkipped method "+  getTestMethodName(iTestResult) + " skipped");
    }

    @Override
    public void onTestFailedButWithinSuccessPercentage(ITestResult iTestResult) {
        System.out.println("Test failed but it is in defined success ratio " + getTestMethodName(iTestResult));
    }

    @Override
    public void onStart(ITestContext iTestContext) {
        System.out.println("I am in onStart method " + iTestContext.getName());
    }

    @Override
    public void onFinish(ITestContext iTestContext) {
        System.out.println("I am in onFinish method " + iTestContext.getName());
    }

    private static String getTestMethodName(ITestResult iTestResult) {
        return iTestResult.getMethod().getConstructorOrMethod().getName();
    }

}

TestNG.xml:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestNG Listener Example">
    <listeners>
        <listener class-name="Listener" />
    </listeners>

    <test name="TestNG Sample Test" preserve-order="true">
        <classes>
            <class name="SampleTest">
<!--                <methods>
                    <include name="test1"/>
                </methods>-->
            </class>
        </classes>
    </test>

</suite>

Pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>TestNGListener</groupId>
    <artifactId>TestNGListener</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.141.59</version>
    </dependency>

    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.14.3</version>
        <scope>test</scope>
    </dependency>
    </dependencies>

</project>

Test Output:

[fusion_builder_container hundred_percent="yes" overflow="visible"][fusion_builder_row][fusion_builder_column type="1_1" background_position="left top" background_color="" border_size="" border_color="" border_style="solid" spacing="yes" background_image="" background_repeat="no-repeat" padding="" margin_top="0px" margin_bottom="0px" class="" id="" animation_type="" animation_speed="0.3" animation_direction="left" hide_on_mobile="no" center_content="no" min_height="none"][TestNG] Running:
I am in onStart method TestNG Sample Test
I am in Before Method! Test is starting!
I am in onTestStart method test1 start
I am in test1 test method and it will pass.
I am in onTestSuccess method test1 succeed
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test2 start
I am in test2 test method and it will fail.
I am in onTestFailure method test2 failed
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test3 start
Test ignored.
I am in onTestSkipped method test3 skipped
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 1
test4 failed!
Test failed but it is in defined success ratio test4
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 2
test4 failed!
Test failed but it is in defined success ratio test4
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 3
I am in onTestSuccess method test4 succeed
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 4
I am in onTestSuccess method test4 succeed
I am in After Method! Test is ending!
I am in Before Method! Test is starting!
I am in onTestStart method test4 start
test4 test method, invocation count: 5
I am in onTestSuccess method test4 succeed
I am in After Method! Test is ending!
I am in onFinish method TestNG Sample Test
===============================================
TestNG Listener Example
Total tests run: 8, Failures: 3, Skips: 1
===============================================

Test Code:

https://github.com/swtestacademy/TestNGListener

References:

https://examples.javacodegeeks.com/enterprise-java/testng/testng-listeners-example/

http://www.guru99.com/listeners-selenium-webdriver.html

http://testng.org/doc/documentation-main.html#testng-listeners

http://learn-automation.com/what-is-testng-listener-and-how-to-implement-in-selenium/

http://toolsqa.com/selenium-webdriver/testng – listeners/[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]

6 thoughts on “How to Use TestNG Listeners”

    • In real projects, you can define webdriver in TestBase and if you extend TestBase in Listener class, you can reach the driver object and use it in Listener Class’s methods.

      Reply
  1. Onur, I have just started learning selenium and testNG framework. Nevertheless, let me say that is a poor coding practice. First, when implementing an inheritance it should be is-relationship. Second, the only way to reach the driver object of undergoing test is to declare driver static in BaseTest class. If you do that you are limiting you project. For example you will not be able to run parallel tests.
    Please correct me if I am wrong?

    Reply
  2. Hi Onur, this is really great. Thanks for sharing such a very detailed and useful article.
    So, here is the question: Is there any way to print a custom message on ExtentReport for failed tests. I don’t have a UI part on my project, so I am not using Selenium and not taking any screenshot. I have Rest API with Kafka, all backend staff… If it’s there any way to take a screenshot for API testing, it would be really great to use it with your example on onTestFailure method, I mean to take a screenshot for scripts in intellij let’s say, unfortunately that is just my dream…
    If test failed it is not printing test description (ExtentManager.getText().setDescription(“some info about the test “); in ExtentReport. And I would like to add some custom messages for each steps or method if test fails, if not failed –> not printing them in ExtentReport …
    Please help, thanks in advance!

    Reply
    • You may try ExtentTestManager.getTest().log(LogStatus.FAIL, “Test FAILED.”);

      If it’s there any way to take a screenshot for API testing? Honestly, I don’t know. API’s do not have any UIs. Rather than screenshots, it may be better to stick with error messages. Also please try, iTestResult.getThrowable() rather than “TEST FAILED” message for custom error message.

      @Override
      public void onTestFailure(ITestResult iTestResult) {
      System.out.println(“I am in onTestFailure method ” + getTestMethodName(iTestResult) + ” failed”);

      //Extentreports log for failed tests.
      ExtentTestManager.getTest().log(LogStatus.FAIL, iTestResult.getThrowable());
      }

      You can also check this article. http://www.swtestacademy.com/extentreports-testng/

      Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.