Selenium Parallel Tests Using Selenium Grid and TestNG

Hi all, in the previous article I explained and showed two common techniques to run your selenium tests in parallel with Selenium Grid and JUnit. In this post, I will show you to run your tests in parallel with another popular testing framework, TestNG.

Prerequisites

Nice to have: Before starting to read this post it is better to read previous selenium tutorials.

For Mac users, please check this article for Selenium Grid usage: https://www.swtestacademy.com/selenium-grid-on-mac/

For Windows users, please check this article for Selenium Grid usage: https://www.swtestacademy.com/selenium-grid/

Test Scenario:

We have two test classes. The first one has three test methods that open www.google.com and check that title is “Google”. The second one has two test methods and they open Google and Yandex and then check their titles. Tests are very simple but our aim is to run them in parallel with different browsers. We will run the first test with Chrome and the second one with Firefox. We will configure this within TestNG.xml file.

Making Your TestNG Tests Thread-Safe:

It is very important to construct our tests thread-safe in order to run them in parallel without a problem. We have to make sure that shared resources are isolated within each thread. Thus, we need to initialize all related resources within the test method. Also, we need to keep test-specific resources thread-local and keep your static class members as static that is really needed to be static. Apply these to all the classes that are loading during the test execution.

Test Architecture:

I used three JAVA files for our test. These are FirstTest.java, SecondTest.java, and BaseTest.java. Also, I did the configurations in TestNG.xml file.

  • In BaseTest class, I created ThreadLocal <>() webdriver (ThreadLocalMap) for thread-safe test execution and I got the TestNG parameter (browser) with @Parameter annotation.
  • I created and configured Browser Capabilities by using OptionsManager class and set our local grid address in TestBase class.
  • CapabilityFactory returns browser Capabilities based on browser name.
  • In BaseTest class, getDriver() method returns the created driver.
  • FirstTest and SecondTest classes extend TestBase class and comprise of their test code.

Test Code:

Before run the test, you need to trigger Selenium Grid!
It is described at first section of this article. For Mac users please check here.

OptionsManager.java

I will start with Browser Options. You can set your options based on your requirements or you can use empty options. It is up to you.

import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.firefox.FirefoxProfile;

public class OptionsManager {

    //Get Chrome Options
    public static ChromeOptions getChromeOptions() {
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        options.addArguments("--ignore-certificate-errors");
        options.addArguments("--disable-popup-blocking");
        //options.addArguments("--incognito");
        return options;
    }

    //Get Firefox Options
    public static FirefoxOptions getFirefoxOptions () {
        FirefoxOptions options = new FirefoxOptions();
        FirefoxProfile profile = new FirefoxProfile();
        //Accept Untrusted Certificates
        profile.setAcceptUntrustedCertificates(true);
        profile.setAssumeUntrustedCertificateIssuer(false);
        //Use No Proxy Settings
        profile.setPreference("network.proxy.type", 0);
        //Set Firefox profile to capabilities
        options.setCapability(FirefoxDriver.PROFILE, profile);
        return options;
    }
}

CapabilityFactory.java

We can get browser capabilities from an instance of this class. We can get the browser name from TestNG.xml file.

import org.openqa.selenium.Capabilities;

public class CapabilityFactory {
    public Capabilities capabilities;

    public Capabilities getCapabilities (String browser) {
        if (browser.equals("firefox"))
            capabilities = OptionsManager.getFirefoxOptions();
        else
            capabilities = OptionsManager.getChromeOptions();
        return capabilities;
    }
}

BaseTest.java

We can use CapabilityFactory to get browser options and we will use ThreadLocal for ThreadSafe execution.

import java.net.MalformedURLException;
import java.net.URL;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Parameters;

public class BaseTest {
    //Declare ThreadLocal Driver (ThreadLocalMap) for ThreadSafe Tests
    protected static ThreadLocal<RemoteWebDriver> driver = new ThreadLocal<>();
    public CapabilityFactory capabilityFactory = new CapabilityFactory();

    @BeforeMethod
    @Parameters(value={"browser"})
    public void setup (String browser) throws MalformedURLException {
        //Set Browser to ThreadLocalMap
        driver.set(new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capabilityFactory.getCapabilities(browser)));
    }

    public WebDriver getDriver() {
        //Get driver from ThreadLocalMap
        return driver.get();
    }

    @AfterMethod
    public void tearDown() {
        getDriver().quit();
    }

    @AfterClass void terminate () {
        //Remove the ThreadLocalMap element
        driver.remove();
    }
}

FirstTest.java

import org.testng.Assert;
import org.testng.annotations.Test;

public class FirstTest extends TestBase {
    @Test
    public void GOOGLE1() {
        System.out.println("Google1 Test Started! " + "Thread Id: " +  Thread.currentThread().getId());
        getDriver().navigate().to("http://www.google.com");
        System.out.println("Google1 Test's Page title is: " + getDriver().getTitle() +" " + "Thread Id: " +  Thread.currentThread().getId());
        Assert.assertEquals(getDriver().getTitle(), "Google");
        System.out.println("Google1 Test Ended! " + "Thread Id: " +  Thread.currentThread().getId());
    }

    @Test
    public void GOOGLE2() {
        System.out.println("Google2 Test Started! " + "Thread Id: " +  Thread.currentThread().getId());
        getDriver().navigate().to("http://www.google.com");
        System.out.println("Google2 Test's Page title is: " + getDriver().getTitle() +" " + "Thread Id: " +  Thread.currentThread().getId());
        Assert.assertEquals(getDriver().getTitle(), "Google");
        System.out.println("Google2 Test Ended! " + "Thread Id: " +  Thread.currentThread().getId());
    }

    @Test
    public void GOOGLE3() {
        System.out.println("Google3 Test Started! " + "Thread Id: " +  Thread.currentThread().getId());
        getDriver().navigate().to("http://www.google.com");
        System.out.println("Google3 Test's Page title is: " + getDriver().getTitle() +" " + "Thread Id: " +  Thread.currentThread().getId());
        Assert.assertEquals(getDriver().getTitle(), "Google");
        System.out.println("Google3 Test Ended! " + "Thread Id: " +  Thread.currentThread().getId());
    }
}

SecondTest.java

import org.testng.Assert;
import org.testng.annotations.Test;

public class SecondTest extends TestBase{
    @Test
    public void GOOGLE4() {
        System.out.println("Google4 Test Started! " + "Thread Id: " +  Thread.currentThread().getId());
        getDriver().navigate().to("http://www.google.com");
        System.out.println("Google4 Test's Page title is: " + getDriver().getTitle() +" " + "Thread Id: "+ Thread.currentThread().getId());
        Assert.assertEquals(getDriver().getTitle(), "Google");
        System.out.println("Google4 Test Ended! " + "Thread Id: " +  Thread.currentThread().getId());
    }

    @Test
    public void YANDEX() {
        System.out.println("Yandex Test Started! " + "Thread Id: " +  Thread.currentThread().getId());
        getDriver().navigate().to("http://www.yandex.com");
        System.out.println("Yandex Test's Page title is: " + getDriver().getTitle() +" " + "Thread Id: " + Thread.currentThread().getId());
        Assert.assertEquals(getDriver().getTitle(), "Yandex");
        System.out.println("Yandex Test Ended! " + "Thread Id: " +  Thread.currentThread().getId());
    }
}

TestNG.xml

You should create TestNG.xml under the top level of your project directory. If you are using IntelliJ IDEA, just right click your project name and then create a file and name is TestNG.xml and then copy and paste the below code into that file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite thread-count="2" name="Suite" parallel="tests" >
    <test name="com.FirstTest" parallel="methods" thread-count="5">
        <parameter name="browser" value="chrome"/>
        <classes>
            <class name="FirstTest">
                <methods>
                    <include name="GOOGLE1" />
                    <include name="GOOGLE2" />
                    <include name="GOOGLE3" />
                </methods>
            </class>
        </classes>
    </test> <!-- First Test -->
    <test name="com.SecondTest"  parallel="methods" thread-count="5">
        <parameter name="browser" value="firefox"/>
        <classes>
            <class name="SecondTest">
                <methods>
                    <include name="GOOGLE4" />
                    <include name="YANDEX" />
                </methods>
            </class>
        </classes>
    </test> <!-- Second Test -->
</suite> <!-- Suite -->

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>TestNGParallel</groupId>
    <artifactId>TestNGParallel</artifactId>
    <version>1.0-SNAPSHOT</version>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.141.59</version>
        </dependency>

        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.12</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

Test Results:

Test GitHub Project

Github: https://github.com/swtestacademy/TestNGParallel

References

http://stackoverflow.com/questions/30353996/selenium-and-parallelized-junit-webdriver-instances

https://wiki.saucelabs.com/display/DOCS/Parallel+Testing+in+Java+with+Maven+and+TestNG

https://www.browserstack.com/automate/java#speed-up-testing

http://blog.wedoqa.com/2013/07/how-to-run-parallel-tests-with-selenium-webdriver-and-testng-2/

https://rationaleemotions.wordpress.com/2013/07/31/parallel-webdriver-executions-using-testng/

Selenium Webdriver Tutorial Series

[fusion_widget_area name=”avada-custom-sidebar-seleniumwidget” title_size=”” title_color=”” background_color=”” padding_top=”” padding_right=”” padding_bottom=”” padding_left=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=””][/fusion_widget_area]

30 thoughts on “Selenium Parallel Tests Using Selenium Grid and TestNG”

  1. How do you implement parallel testing with page factory. I tried this with page factory and its throwing thread safety issues.
    Can you write a post for parallel testing when page factory is used

    Reply
    • Hi Pooja, You don’t need to do it. Open a directory on C:\ driver as “drivers”. Download and put chrome and firefox driver executables in that folder. After that, open environmental variables on windows. Add “C:\drivers” location into path variable. Then restart the PC. After that, you do not need to declare system.property line for driver executables in your automation codes.

      For parallel tests, you need to do it in grid configuration also. You can find details in this article. http://www.swtestacademy.com/parallel-tests-selenium-grid-junit/

      Reply
  2. Hi. I have couple of questions about parallel running.
    1- If I run test locally, Is it a good practice to use Selenium Grid?
    2- Is it a common practice to call getDriver on each method when you run tests parallel? (Currently I call getDriver on class level and and methods in classes are dependant with each other, dependsOnMethods = {“ABC”},alwaysRun = true)

    Thanks a lot! I learn a lot from your website.

    Reply
    • 1- Yes, you can use Selenium Grid for parallel test execution but also you can use Docker-Selenium, Zalenium, and Selenoid (Yandex) too.
      2- If you run your test in parallel, it better not to use dependent methods. Each test (method)should be atomic and has a single responsibility. In this way, you can assure parallelism very well.

      I appreciate your feedback. Thank you! :)

      Reply
  3. Thanks for this post Onur.

    Can you please explain –
    1. your one test i.e. @Test is equivalent to 1 test case ?

    Example : I am working on Salesforce application and there are 3 test cases
    1. TC1 – Login into Salesforce .
    2. TC2 – Click on Agencies and create task
    3. TC3 – Click on Lead and create task.

    So will structure be like –
    @Test
    LoginintoApp()

    @Test
    ClickOnAgency

    @Test
    CreateTaskForAgency

    In Such case My Login into app method will run in each @test which will make unnecessary login step execution thrice.

    Could you please explain your approach how you would segregate above test cases in different class file and @test

    Reply
    • For parallel execution, if a test case needs login, we need to login first. But if you run them in sequential order, then you can use TestNG priority or dependson features. A test case needs login and then do sth. you need to write the login case in it first, then do the required actions. I understand it is an unnecessary login step execution time but if you run for example 50 tests or more in parallel, it will not be a big problem. For this, I suggest you use Docker-Selenium or Selenoid solutions. Also, you need to write your tests with Page Object Model. So, you can write login actions only once in page class and use it in test classes.

      Reply
  4. Thanks for the post Onur,

    Please let me know if there is way to run the TestNg tests in parallel, based on the number of nodes associated in a selenium grid hub. Instead of explicitly specifying the thread count, it should dynamically adjust the parallel run

    Ex: if the nodes are 6 and 6 test cases, each test case should in one node. And if the nodes are 3 and test cases are 6, it should run 3 test cases first and then start the other 3 instead of starting all 6 at the same time

    Reply
    • Hi Saravanan, Maybe you will do it in this way. You need to write a shell script to trigger selenium grid with for example 5 nodes. After that, you need to write a second script, to read the testng.xml file, modify thread-count as 5, then save it. After these operations, you can start your test automation project with maven command with modified testng.xml file. I hope you will get my strategy. You can modify the testng.xml file and trigger the grid by using .bat files, Python, Perl, Java, PowerShell etc. it is up to you. ;)

      Reply
  5. Hi Onur. Thanks for posting this. I can tell you definitely understand the issues with selenium grid, parallel execution, and having it be thread safe. Currently, my test were not written with thread safety in mind and they are able to execute fine when I run just one instance from Jenkins. If I try to run it in parallel using selenium grid, they fail to finish executing. It looks like it will be too much overhead for me to convert my entire test suite to be thread safe given my timeline. Is there any other recommendation to have my test run “in parallel” without making it thread safe? The only other ideas I have is to have multiple instances of Jenkins. I know it’s not the best solution, but I am looking for a workaround to have my sets of test run in parallel. Worst comes to worst, I will setup multiple instances of Jenkins on different machines. Thank you!

    Reply
    • Hi Steve, one of the other solutions is Selenoid. We also tried this and will write an Article with Kaan Sariveli (our new automation engineer). If you have a time, you can also try Selenoid. It is not too hard to use. But thread-safety will be again needed. On the other hand, multiple Jenkins instances on different machines is a workaround solution but it comprises of manual effort too. At the same time, you need to take care of data while you are doing parallel execution and your tests should be as atomic as possible. Because in our case many tests have a scenario with login and we provide different login information for each test. As I understand, you need a very urgent solution. I suggest you do the way that you know such as multiple Jenkins, then you can enhance this solution in the mid or long run.

      Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.