Test Automation Reporting with Allure and TestNG


Hi all, in my last article I explained how to integrate ExtentReports reporting framework with TestNG for shiny and beautiful test automation reports [link]. In this article, I will explain you another popular test reporting framework which is developed by Yandex QA Team – Allure.  I will add additional codes and configurations on ExtentReports project. Thus, in this code, you can both generate test reports both Allure and ExtentReports! 🙂 Let’s start to integrate Allure into our test automation projects!

Step-1: Modify Maven pom.xml 

For Allure integration, you need to add below maven dependencies into your pom.xml file.

Properties Section

Allure TestNG Dependency

Build Section

and we are done! The pom.xml settings have been finished.

Step-2: Configure Allure Features and Modify Test Codes

I will explain all features based on below picture. I did some modifications on my test file to make the report HTML file more understandable and informative.

Feature-1: Display Name

In order to make our test report more understandable, I used description property of @test annotation.

@Test (priority = 0, description=”Invalid Login Scenario with wrong username and password.”)

Also, you can add an additional description with @Description annotation:

@Description(“Test Description: Login test with wrong username and wrong password.”)

The result will be shown as below:

Feature-2: Steps

Steps are test actions in our test scenarios. They can be used any testing scenario. Thus, we need to define steps in generic place in our test project. In order to define a step, we need to use @Step annotation. In our project, steps are defined at our page classes. LogintoN11, verifyLoginUserName, verifyLoginPassword, all of this methods are our test steps. Therefore, we need to add @Step annotation above these methods as shown below. Here, we can also pass parameters with {} syntax. For example, at first method, {0} is first parameter – username, {1} is second parameter – password.

LoginPage Steps:

HomePage Steps:

Here are the results. They will look on the report as like below image.

Feature-3: Attachments

We can add attachments to our reports by using @Attachment annotation. It can return String, byte [], etc. For example, if we want to attach a screenshot we should return byte[]. Also, I need to add @Listeners({ TestListener.class }) decleration at the top of the test class.

In TestListener class, I wrote two attachment method for string attachment and screenshot attachment.

And, I called those methods when a test failed as shown below.

The result will be like below in the report.

Feature-4: Links

You can integrate your defect tracking system and test management tool with allure by using @Link annotation as shown below. [1]

In order to specify the link pattern you can use the system property in the following format: allure.link.my-link-type.pattern=https://example.org/custom/{}/path. Allure will replace {} placeholders with the value specified in the annotation. For example:

I will not use this feature in my test so I cannot show you any result for this.

Feature-5: Severity

We can order test by severity by using @Severity annotation. I used this feature in tests as shown below.

and the result will be like that.

Feature-6: Behaviour Driven Reporting (Features and Stories)

We can group tests with @Epic, @Feature, and @Stories annotations.

Here is the report result in Behaviors Section.

and that’s all. 🙂 Now, we should run the test and generate the report.

Project Code is here: https://github.com/swtestacademy/TestNGAllureReport 

Step-3: Run the Test and Generate Allure Report

You can run the test with maven command. In order to do this in IntelliJ first, you should click configurations.

Select maven, and write the maven “clean test” command as shown below and then click OK.

Now, we can run the code by clicking the green run icon.

Now, it is time to generate the report!

In order to generate a report, we should install Allure command-line interpreter.

  1. Download the latest version as zip archive from bintray.
  2. Unpack the archive to allure-commandline directory.
  3. Navigate to bin directory.
  4. Add allure to system PATH.

and finally, open a command prompt screen, go to the project directory, and write below command!

allure serve allure-results

and, you will see the beautiful Allure Test Report as shown below.

Dashboard

Categories

Suites

Graphs

Timeline

Behaviors

Packages

and we integrate Allure test framework with our automation project! 🙂

See you in next article!
-Onur

References

[1] https://docs.qameta.io/allure/2.0/#_testng

javafx

Database Operations in JavaFX

By Onur Baskirt / Apr 1, 2016 / 59 Comments
Before started this section, please check the first article and learn How to Start JAVAFX! http://www.swtestacademy.com/getting-started-with-javafx/ At first, part of JavaFX tutorial series, we created a sample JavaFX project, designed the draft version of the UI and set up an...
rest assured

REST API Testing with Rest Assured

By Onur Baskirt / Mar 8, 2016 / 32 Comments
Outline In this post, I will explain what is API and API testing, what is the difference between SOAP and REST services, and how to test REST APIs with Rest Assured Library. What is API? API stands for Application Programming...
selenium

How to Write Smart XPath Locators

By Onur Baskirt / Sep 24, 2017 / 5 Comments
Hi all, in this tutorial, I will describe you how to write smart and non-brittle XPath locators. When we write our test scripts, we generally prefer to use id, name, class, etc. these kinds of locators. However, sometimes we could not...
selenium

How to Write Effective CSS Locators

By Onur Baskirt / Oct 1, 2017 / 0 Comments
Hi all, in this tutorial, I will describe you how to write effective CSS locators to interrogate web elements for your automation projects. As a rule of thumb, your interrogation strategy should be in below order: First try to use...
selenium

Selenium-1: Quick Start to Automation with Selenium WebDriver & JAVA & JUnit & Maven & IntelliJ

By Onur Baskirt / Sep 8, 2015 / 26 Comments
Outline Selenium Webdriver is the most popular open source web test automation framework across the wide range of browsers and platforms. In this tutorial, you will learn how to do web test automation with Selenium Webdriver and the related tools....
allure

Test Automation Reporting with Allure and TestNG

By Onur Baskirt / Oct 21, 2017 / 13 Comments
Hi all, in my last article I explained how to integrate ExtentReports reporting framework with TestNG for shiny and beautiful test automation reports [link]. In this article, I will explain you another popular test reporting framework which is developed by...
selenium

Selenium-11: Execute JavaScript with JavascriptExecutor

By Onur Baskirt / Jan 27, 2016 / 16 Comments
Outline Sometimes we cannot handle some conditions or problems with Webdriver, web controls don't react well against selenium commands. In this kind of situations, we use Javascript. It is useful for custom synchronizations, hide or show the web elements, change...

Getting Started with RobotFramework on Windows

By Onur Baskirt / Apr 25, 2016 / 12 Comments
What is RobotFramework? RobotFramework is a GENERIC test automation framework for acceptance testing and acceptance test-driven development (ATTD). What it means that you can do web, mobile, desktop and other test automation activities with related test libraries. These libraries can...
page object model

Page Object Model with C#

By Ege Aksoz / Jun 18, 2017 / 8 Comments
In the previous tutorial, we’ve taken the initial steps and entered the world of automated testing. We also wrote our first automated test. From this point on, since we are not just going to write one test, we need to...
javafx

Getting Started with JavaFX

By Onur Baskirt / Mar 25, 2016 / 0 Comments
When I started to work in my current position, one of my task is to do manual operations for campaign products  every week. After the second week, I thought that I have to automate this task using a GUI based...
By | 2017-10-22T13:22:17+00:00 October 21st, 2017|Selenium, Test Automation, Test Frameworks, TestNG|13 Comments

About the Author:

Onur Baskirt

Onur Baskirt is a senior IT professional with 10+ years of experience. He worked at Bahçesehir University, ST Microelectronics, Huawei and Ericsson as research assistant, design verification engineer and software test leader. Also, he worked as software test leader and software operations manager at Turkey’s biggest technology retailer, Teknosa. Now, he is working as Head of Software Testing and Manager of two Development Teams at Kariyer.net. His current research areas are technical software testing, programming, and computer science. Formerly, he has several research works and papers on digital chip design & verification. His hobbies are sport, dancing, traveling, and nutrition. You can find detailed information about him on his linked-in page.

13 Comments

  1. Sudheer October 23, 2017 at 3:24 pm - Reply

    Thank you so much Onur.
    This is really helpful.

  2. Dj October 23, 2017 at 4:31 pm - Reply

    Hi why not serenity-bdd

  3. Ashish K October 23, 2017 at 6:46 pm - Reply

    Detailed information provided in the article, really good to read the article. thank you!

  4. Djelloul October 24, 2017 at 8:44 pm - Reply

    Thanks again

    Link to the serenity-bdd

    http://thucydides.info/docs/serenity-staging/

  5. Asu October 26, 2017 at 5:41 pm - Reply

    Thanks. Wonderful sharing.

  6. jj November 24, 2017 at 8:11 am - Reply

    Hi Onur,
    Is this framework built on both Allure and ExtentReport?
    Can you just use Allure but still use the annotations and tags you used in ExtentReport?

    • Onur Baskirt
      Onur Baskirt November 24, 2017 at 10:30 pm - Reply

      Yes, it is for both of them. But you can modify it as you wish. 🙂

Leave A Comment