Reporting with ExtentReports and TestNG Listeners

Hi all, in this article I will describe how to integrate ExtentReports reporting tool with TestNG Listeners. I described how to use TestNG Listeners in this article and Canberk Akduygu and I also described how to rerun failed tests with IRetryAnalyzer in this article. Here, I will use TestNG listeners and IRetryAnalyzer interface and add ExtentReports classes to generate very beautiful reports after each test run.

If you do not know Page Object Model, before starting this article, I suggest you check my POM with JAVA article. In our Page Object Model example, we create two login tests for n11.com website. Now, I will add ExtentReports and Listener packages under test package as shown below.

Let’s start step by step! 🙂

Step-1: Add ExtentReports Maven Dependency

You should add ExtentReports dependency to your pom.xml. (I used version 2.41.2! It is stable.)

Step-2: Add ExtentReports Classes 

I created ExtentReports package under utils package and add below two classes in that package.

ExtentManager Class:

In this class, we created an ExtentReports object and it can be reachable via getReporter() method. Also, you need to set your ExtentReports report HTML file location.

ExtentTestManager Class:

  • An extentTestMap map is created. It holds the information of thread ids and ExtentTest instances.
  • ExtentReports instance is created by calling getReporter() method from ExtentManager.
  • At startTest() method, an instance of ExtentTest created and put into extentTestMap with current thread id.
  • At endTest() method, test ends and ExtentTest instance got from extentTestMap via current thread id.
  • At getTest() method, return ExtentTest instance in extentTestMap by using current thread id.

Step-3: Add Listener Classes

For retry failed tests, we should add AnnotationTransformer, Retry classes.  In order to listen to test events such as passed, failed, skipped, etc. we should add TestListener class in our project.

TestListener Class:

I added ExtentReports codes in each method and add some informative comments.

AnnotationTransformer Class:

Retry Class:

Step-4: Add Description For Tests (Optional)

You can add test descriptions in your tests and you will see them in the report as shown below.

Step-5: Add Listeners to TestNG.xml

We should add listeners in TestNG.xml as shown below.

Step-6: BaseTest Class

Step-7: Test Class

And run the test!

After test finished, you will see ExtentReportsResults.html file under ExtentReports folder as shown below.

When you open the report, you will see test results as shown below.

Detailed Test Results:

General Test Results:

GitHub Project: https://github.com/swtestacademy/ExtentReportsExample

Also, you can check How to do Test Reporting with Allure and TestNG.

Link: Allure Test Reporting Framework Integration with TestNG 

Also, you can check How to do Test Reporting with Extent Reports Version 3.

Link: Extent Reports Version 3 Reporting with TestNG Listener

Also, you can check How to do Historical Test Reporting with Klov  – ExtentReports.

Link: Klov – ExtentReports | Test Automation Reporting

Selenium Webdriver Tutorial Series

See you in next article!
-Onur

By |2018-12-04T22:42:45+00:00October 9th, 2017|Extent Reports|100 Comments

About the Author:

Onur Baskirt is a senior IT professional with 10+ years of experience. He worked at Bahçesehir University, ST Microelectronics, Huawei and Ericsson as a research assistant, design verification engineer, and software test leader. Also, he worked as software test leader and software operations manager at Turkey's biggest technology retailer, Teknosa. After Teknosa, he worked as Head of Software Testing and Manager of two Development Teams at Kariyer.net. Now, he is working as a Senior Technical Consultant at Emirates Airlines in Dubai. His current research areas are technical software testing, programming, and computer science. Formerly, he has several research works and papers on digital chip design & verification. His hobbies are sport, dancing, traveling, and nutrition. You can find detailed information about him on his linked-in page.

100 Comments

  1. Sudheer Kumar Balivada October 10, 2017 at 10:49 am - Reply

    This is really a good article. I follow all your selenium and automation testing articles and of course helped me in creating a framework at work.

    Could you please create a similar article using Allure reporting. It really helps me.

    Thanks in advance.

  2. Rizwan Jafri October 10, 2017 at 12:55 pm - Reply

    Wonderful

  3. Alberto October 11, 2017 at 3:26 pm - Reply

    hi,
    First, i’m sorry for my english.
    Second, i tell you that it is the best report for testng that i know, and i congratulate you for it. congratulations.
    but, i have a problem. i can´t watch the feature name and the step name. how can i watch it?
    Thank you

  4. Prasad October 12, 2017 at 10:19 am - Reply

    How to invoke ExtentTestManager.getTest() method in test classes and page classes?

    I am getting “Null Pointer Exception” while calling getTest() method. Can you please help me out with this?

    Thanks In Advance!

    • Onur Baskirt October 13, 2017 at 12:44 am - Reply

      ExtentTestManager class has public synchronized static methods. Thus, when you import “import utils.ExtentReports.ExtentTestManager;” line, then you can reach ExtentTestManager.endTest(); or ExtentTestManager.getTest();
      Please, check TestListener class. I called those methods several times. If you get null pointer, first you should start test with ExtentTestManager.startTest(String testName, String desc), then you can get that test by ExtentTestManager.getTest()

  5. Ray October 25, 2017 at 6:45 pm - Reply

    I Lost it at

    extent.endTest((ExtentTest)extentTestMap.get((int) (long) (Thread.currentThread().getId())));
    extent.startTest(testName, desc);

    it throws an unknown error

    • Onur Baskirt October 25, 2017 at 10:02 pm - Reply

      Ray, you should use that class (ExtentTestManager.java) in your project without any change. You will examine sample project from here. https://github.com/swtestacademy/ExtentReportsExample

      • Ken November 29, 2017 at 10:38 pm - Reply

        Same.. You lost me here. Im getting the same error.

        • Ken November 29, 2017 at 10:53 pm - Reply

          I found my own answer… I was using version 3.. This docs say version 2.

          • Onur Baskirt November 29, 2017 at 11:03 pm

            Yes Can. I implemented for Version 2. You are right.

  6. Alberto November 12, 2017 at 12:46 pm - Reply

    Hi Onur,
    Great article! Is it possible to add test steps in ExtentReport? (Like Allure?) and console logs?
    Cheers,

  7. Anton Jerald November 29, 2017 at 9:21 pm - Reply

    Hello,
    I am not seeing the definition for the BaseTest.java. What is the definition you are having there.

  8. Naeem Siddiq November 30, 2017 at 10:41 am - Reply

    Hi Onur,

    Its a great article
    I need a little help
    I want to add my class name as a heading in the report.
    Like if Class A has five Test Cases and Class B has 4 test cases then Class name i-e ” Class A ” should appear as a heading on the left side of the report and then the test cases.

    For example
    like this

    Class A
    Test Case 1
    Test Case 2

    ….

    any solution plz
    Thanks in advance

    • Onur Baskirt December 4, 2017 at 12:34 pm - Reply

      You can get the Class name as follows:
      @BeforeClass
      public void beforeClass() {
      String className = this.getClass().getName();
      System.out.println(“Class Name: ” + className + “\n”);
      }

      You can get the method name as follows:
      @BeforeMethod
      public void setupTestMethod (Method method) {
      testName = method.getName();
      System.out.println(“Test Name: ” + testName + “\n”);
      }

  9. Pedro December 6, 2017 at 6:01 pm - Reply

    Hello!
    Do you have any idea how to generate one report for multiple test suites run.
    For example we run 2 suite files with maven surefire plugin. So is there any way to create extent report that would include all suites in one report?
    I mean report should look like suite1 name->all tests in suite, suite2 name-> all tests in suite.

    Thank you.

  10. Rj February 11, 2018 at 6:26 am - Reply

    The best article for extent reports. I will definitely save this page for future reference. I read several and eventually figure everything out, but this article is so detail and easy to follow. This would of saved several painful headaches.
    Thanks

    • Onur Baskirt February 11, 2018 at 12:36 pm - Reply

      You are welcome! I am happy that the article helped you. I really appreciate your positive feedback.

  11. Jhonny February 12, 2018 at 5:38 pm - Reply

    Thanks for the info, it really help me.

  12. Nikhil February 14, 2018 at 9:33 am - Reply

    Hello Onur,

    I want show falied and pass test cases log on report.
    Can you please tell me where we to change this.
    If possible tell me class name and code.

  13. Ramzan February 28, 2018 at 11:23 am - Reply

    Wonderful article!

    I have one query please, i used same as you mentioned but logs are printed on console only not inside the test ng report. could yo uplease guide

    • Onur Baskirt March 4, 2018 at 5:34 pm - Reply

      You mean Allure report? TestNG Report is a different thing.

  14. Ramzan March 7, 2018 at 10:31 am - Reply

    Thanks for great article and response. I am using latest extent build 3.1. Could yo please give the same berief for latest build.

  15. Kushagra March 21, 2018 at 5:57 am - Reply

    HI OB,

    First of all, very nice article. Finally someone who writes practical solution that can be implemented in day2day! 🙂

    I am having an issue while implementing the ExtentTestManager class:

    public static synchronized void endTest() {
    extent.endTest((ExtentTest)extentTestMap.get((int) (long) (Thread.currentThread().getId())));
    }

    In extent.endTest(….) I am getting the exception ‘Cannot resolve method endTest(com.aventstack.extentreports.Exports)’

    Any help will be highly appreciated.

    Cheers!

  16. Sam April 12, 2018 at 11:54 am - Reply

    Thanks Onur, for the detailed tutorial implementing Extent Reports.
    I have a issue where the failed test is showing unknown tag in report.

    • Onur Baskirt April 12, 2018 at 1:06 pm - Reply

      Welcome Sam. I think you need to debug it failure condition in your environment. I can not say an exact solution without debugging your project. I suggest you put some breakpoints on especially aftermethod() and failure condition in the listener class and see what happens.

      • Sam April 25, 2018 at 2:23 am - Reply

        Hi Onur, Do I have to change something for screenshot method as I using the extent reports from Android Appium automation framework.

        I am getting Null pointer exception:

        //Take base64Screenshot screenshot.
        String base64Screenshot = “data:image/png;base64,”+((TakesScreenshot)webDriver).
        getScreenshotAs(OutputType.BASE64);

        • Onur Baskirt April 25, 2018 at 10:01 am - Reply

          Yes, you can change and check the results. If your change works well. Keep the modified version. I wrote these part for web automation. For mobile, please try your implementation and keep us updated. Thank you.

    • Sam April 12, 2018 at 8:53 pm - Reply

      I am getting Null pointer exception:

      //Take base64Screenshot screenshot.
      String base64Screenshot = “data:image/png;base64,”+((TakesScreenshot)webDriver).
      getScreenshotAs(OutputType.BASE64);

      • Onur Baskirt April 13, 2018 at 11:54 am - Reply

        I am using the same code for 1 year and did not get any null pointer exception. It is better to debug your project Sam. I am sharing the Failure code of the listener as follows.

        @Override
        public void onTestFailure(ITestResult iTestResult) {
        Object testClass = iTestResult.getInstance();
        WebDriver webDriver = ((BaseTest) testClass).getDriver();
        String base64Screenshot = “data:image/png;base64,”+((TakesScreenshot)webDriver).getScreenshotAs(OutputType.BASE64);
        ExtentTestManager.getTest().log(LogStatus.FAIL,iTestResult.getThrowable().getMessage(), ExtentTestManager.getTest().addBase64ScreenShot(base64Screenshot));
        ExtentManager.getReporter().endTest(ExtentTestManager.getTest());
        }

  17. Zuhaib Anwar Butt April 20, 2018 at 1:27 pm - Reply

    Hi
    i am facing the issue after following your model as you mentioned above,
    when i execute the code the Null Pointer Exception occurred in ExtentManager class the ,getId() method didnot any id there if you can assist me its really appreciated Thanks

    • Onur Baskirt April 20, 2018 at 2:56 pm - Reply

      Hi,

      I downloaded chromedriver 2.38
      I changed TestNG version from 6.11 to 6.14.3
      I changed Selenium version from 3.6.0 to 3.11.0

      Then, run the project and I didn’t see any problems.

      I hope this will help you.

      BR.
      Onur

  18. Raghavi April 25, 2018 at 7:27 am - Reply

    Hello Onur,

    Very good article, Thanks!!!!!!!!!! I have an issue of Screenshot when I run through Jenkins server. Screenshots are crashing only when I run through Jenkins. working fine in my local. Can you help me out?

    • Onur Baskirt April 25, 2018 at 10:00 am - Reply

      Hi, it may be about Jenkins server’s operating system. Are your local machine and Jenkins both Windows? And maybe folder permission problem. It is really hard to detect it with these inputs.

  19. Roy June 6, 2018 at 7:33 am - Reply

    Hi Onur,

    I wanted to show the validation error in the failed report. Can you please let me know how we can code it?

    • Onur Baskirt June 6, 2018 at 7:58 pm - Reply

      Hi, please check this article. http://www.swtestacademy.com/extent-reports-version-3-reporting-testng/

      Please try below codes.

      @Override
      public synchronized void onTestFailure(ITestResult result) {
      System.out.println((result.getMethod().getMethodName() + ” failed!”));
      test.get().fail(result.getThrowable());
      }

      @Override
      public synchronized void onTestSkipped(ITestResult result) {
      System.out.println((result.getMethod().getMethodName() + ” skipped!”));
      test.get().skip(result.getThrowable());
      }

  20. Zuhaib Anwar Butt June 7, 2018 at 12:46 pm - Reply

    Hi Onur,
    can you please help out in the case of Failed scenario, basically i dont need to retry the step/scenario in case of fail.
    and the second thing is if any scenario is failed the next method is mark failed instead of current failed method

    for example

    1st testcase(){
    }

    2nd testcase(){
    }

    3rd testcase(){

    }

    if the 2nd testcase(){
    }

    will failed but report will mark 3rd testcase as fail instead of 2nd which were actually failed

    can you please explain it and resolve it Thanks

  21. Vasista June 19, 2018 at 8:42 am - Reply

    Please provide BaseTest class

    tests.BaseTest;

  22. Nagaraj June 20, 2018 at 10:02 pm - Reply

    Onur I am working on appium cucumber testing extent reports. Ur posts are one stop show for me. I have learnt so many topics from your posts. Keep up the good work. Cheers 👌💐

    • Onur Baskirt June 21, 2018 at 1:19 am - Reply

      You are very welcome. I am so happy to hear your valuable and lovely feedback. Thank you! 🙂

  23. Nagaraj June 21, 2018 at 3:43 pm - Reply

    Hi Onur,

    I am able to generate Extent Reports by passing parameters from testng.xml for single devices.

    Tests are getting passed for multiple devices but not able to generate report for multiple devices. Please suggest. 🙂

  24. Nagaraj June 25, 2018 at 2:32 pm - Reply

    Hi Onur,

    Thank you very much, now I am able to generate Reports for Multiple Devices. 🙂

    • Onur Baskirt June 25, 2018 at 3:10 pm - Reply

      You are welcome Nagaraj. I am so happy to hear this. 🙂

  25. Pratik June 27, 2018 at 10:58 pm - Reply

    I am able to generate Extent Report. But, I would like to make some changes in Emailable-Report.html and generate Emailable-report.html in the form of Extent Report. Is there a way to do it? By the way, I am using Jenkins as well as the CICD tool.

    • Onur Baskirt June 27, 2018 at 11:08 pm - Reply

      This article may help you. Please try and if it works please let us know. https://learn2automate.blog/2017/09/18/emailable-extent-reports-with-jenkins/

      • Pratik June 27, 2018 at 11:29 pm - Reply

        Okay. I will give a try to it. Also, for Data Driven Testing, we mostly pass various data to one test only. TestNG invokes same test with different data but provides the results as per the test name only. For example, If I run the same test for 5 times with different set of data then I get results for test as below:
        1st Test Case -> Test_Name – Pass
        2nd Test Case -> Test_Name – Fail
        3rd Test Case -> Test_Name – Pass
        4th Test Case -> Test_Name – Fail
        5th Test Case -> Test_Name – Pass

        I would like to change the Test_Name with some respective test case number or description in the reports. Is there a way to do so?

  26. Vasu July 30, 2018 at 6:46 pm - Reply

    hi Ontur, your article are great, can you help me how can I append testng testsuitename to the file name in ExtentManager.java class

  27. Vasu July 30, 2018 at 6:53 pm - Reply

    (2) Q I had was when I am executing / debugging from from test class, what I mean is in eclipse after all set up say I want to fix something in test class and I have to do debug as testng or run as testng (note I am not doing a right click on testng.xml to be executed or run as testng suite) I am getting null pointer exception for ExtentTestManager.getTest().log(LogStatus.PASS, “Step02 -“); which makes sense in a debug mode. How do I over come this challenge of executing this set up in debug or run as testng

    • Onur Baskirt July 30, 2018 at 11:50 pm - Reply

      I really did not figure out the problem. If you are getting null pointer, you are using an object without any assignment. In debug mode, you can try to comment out extentreport parts? I really can not provide a super solution.

  28. Zohaib July 31, 2018 at 12:06 am - Reply

    Vasu
    Basically you you will get the null pointer excption when u not run you test suit via right clik on testng l.xml ans run as testng suit’ so if you need to get rid of this run you test always via right clik and testng.xml as testng suit

    If u nees to debug your test from test class put breakpoint and run it as debug but conment the extent report part from the class or method bexuz debug mode is not available for suit as per my knowledge thanks

  29. Vasu August 1, 2018 at 12:25 am - Reply

    Thank you Zohaib and Onur

  30. Rajeev August 20, 2018 at 5:40 pm - Reply

    hi Onur,

    Wonderfull Article , need you help in customizing the ‘Report’ ?
    When the Report get generated , I want to change few heading – for example on the top of the report there is a heading “Automation Report” , HOw can I change this to something else ??
    I tried finding the code for it but din’t find it..

    Thanks,
    Rajeev

  31. Manish August 29, 2018 at 8:53 am - Reply

    Hi Onur,

    Could you please help me getting null pointer exception on this line “ExtentTestManager.getTest().setDescription(“Invalid Login Scenario with wrong username and password.”) its points to here

    public static synchronized ExtentTest startTest(String testName, String desc) {
    ExtentTest test = extent.startTest(testName, desc);
    extentTestMap.put((int) (long) (Thread.currentThread().getId()), test);
    return test;

    • Onur Baskirt August 29, 2018 at 10:09 pm - Reply

      Hello Manihs,

      I don’t know your whole project structure but I can assure that it is caused by the empty object. You are using unassigned/empty/null object and getting null pointer exception. I highly suggest you put some breakpoints and debug your project. Then, I believe that you will figure out the null object.

      When I looked at your code, I do think that the problem maybe is about you can use getTest() before startTest Method call.

  32. Manish August 29, 2018 at 10:32 am - Reply

    Hi Onur,

    If a test case fails in first execution, retryAnalyzer would again execute it. if first time execution fails and second time execution passes, I want to show the results of only second execution in Extent Reports. But I am getting both the test case results in the report how to overcome this problem could you please guide me and more thing we cannot get report if we execute main class instead of testng.xml .

    Thanks

    • Onur Baskirt August 29, 2018 at 10:16 pm - Reply

      I am not %100 sure but when the test fails, the listener class reports the failure scenario and Retry class reruns the test. Maybe you can add a fail counter into your TestListener class and when the fail counter will be increment by one then you can print the result on the report. This is one of the solutions that you can apply and it is very easy to implement. Please try this and let me know. Just add an int failcounter = 0; and in onTestFailure method first check failcounter == 1, if not increment it by 1. In this way, you can only show the second result of the failed test in your report. Also you can play this number based on your retry count. Thanks.

  33. Manish August 30, 2018 at 9:55 am - Reply

    Hi Onur ,Thanks for response i will implement and let you know d status it’s very grateful these type of articles are very useful who are facing challenges in automation.

    • Onur Baskirt August 30, 2018 at 11:13 am - Reply

      Welcome Manish. I hope the proposed solution will work. 🙂

  34. Kml October 31, 2018 at 12:33 am - Reply

    Thank you for the detailed explaination! It is super helpful.

    However your exact same code with your tests above is throwing a null pointer exception. I am using v2 of extent reports. The only difference is I am not tunning t from an xml file, which shouldn’t matter. What’s the solution to this?

    Thanks in advance!

  35. Kml October 31, 2018 at 12:43 am - Reply

    Hi Onur, I am getting a null pointer exception too on getTest(). I’ve used your project without any change and have added a basic test. Please suggest a solution.

    • Onur Baskirt October 31, 2018 at 8:15 am - Reply

      I will check and get back to you.

    • Onur Baskirt October 31, 2018 at 8:30 am - Reply

      Hi, please comment out this line in base test class.

      //Maximize Window
      //driver.manage().window().maximize();

  36. Ankur Gupta October 31, 2018 at 10:07 am - Reply

    Amazing, got the things done in a single shot!!!

  37. Muni November 5, 2018 at 7:44 am - Reply

    It’s really really helpful. Do you your video channel also because it would be very easy to understand .

    • Onur Baskirt November 5, 2018 at 8:12 am - Reply

      Thank you Muni. Honestly, right now I am focusing the blog but if I will have more time, I may record some videos. Thank you again.

  38. Rohit November 29, 2018 at 12:41 am - Reply

    Hello Onur,

    Wonderful article!, I only have one question, how did you get SO good at coding? 🙂 I have been following your articles for some time now, and there’s a plethora of knowledge here and I like your coding style, is there anything that you can suggest so that people like me who are learning to be better at coding can well, be better at coding?

    Cheers, Keep up the good work 🙂

    • Onur Baskirt November 29, 2018 at 8:07 am - Reply

      Thank you Rohit. I am so happy that you like our articles. I suggest you follow some good tutorials on pluralsight or udemy for advance JAVA knowledge. I am still learning and improving myself. Also, I suggest JAVA for Testers – Alan Richardson book first. After this, you can move on with more advanced tutorials.

      • Rohit November 29, 2018 at 10:35 am - Reply

        I would definitely go through all these. A BIG Thanks to you and once again, Keep up the good work!!

  39. Aniket Samudra December 5, 2018 at 1:03 pm - Reply

    This is just beautiful. Thank you so much for the efforts. Very detailed and easy to implement steps given.
    Works well with me when ran using testng.xml. But when i am running as a pom.xml (For e.g from Jenkins), i am getting null pointer exception ExtentTestManager.getTest()

    Can you pleaseeeee help, this is the last step remaining. Any pointer to resolve the problem will be really a big help.
    Maven build fails with null pointer, but runs from testng.xml

  40. Raviteja December 17, 2018 at 3:15 pm - Reply

    Hi Onur. The solution is very helpful. Can you please explain wht the test end time and total time taken fields are not displaying for every test case but displaying only for he last one? This will really help

  41. Raviteja December 17, 2018 at 4:10 pm - Reply

    Hi onur, Thanks for such wonderful explanation. Can you provide some solution to display the test ended time and time taken to finish fields? Currently they as displayed null for starting testcaces where as it’s displayed fine for last test case.

  42. Siddharth Pradhan January 1, 2019 at 11:02 am - Reply

    Hi Onur,

    How can we add Classname.testmethod name in extent report ? in test pane.

    By the way wishing you happy new year and thanks for great article.

Leave A Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.