Selenium Performance Testing with JMeter and Selenium Grid

In this post, we will complete the Selenium Performance testing scenario using JMeter and Selenium Grid.

1- Install Java 7 or later If necessary

https://java.com/tr/download/

2- Download latest Jmeter version 3.0 or higher.

http://jmeter.apache.org/download_jmeter.cgi

1

3- Download Jmeter PluginsManager JAR file and put it to Jmeter’s lib/ext directory. Then, start JMeter and go to “Options” menu to access the Plugins Manager. Open Jmeter Plugins Manager and Install “Selenium/Webdriver Support” plugin.  Restart Jmeter to take changes for Selenium.

https://jmeter-plugins.org/wiki/PluginsManager/

2

Now you can use the following libraries with numerous script languages !!

Libraries are: [fusion_builder_container hundred_percent=”yes” overflow=”visible”][fusion_builder_row][fusion_builder_column type=”1_1″ background_position=”left top” background_color=”” border_size=”” border_color=”” border_style=”solid” spacing=”yes” background_image=”” background_repeat=”no-repeat” padding=”” margin_top=”0px” margin_bottom=”0px” class=”” id=”” animation_type=”” animation_speed=”0.3″ animation_direction=”left” hide_on_mobile=”no” center_content=”no” min_height=”none”][selenium-firefox-driver, selenium-ie-driver, selenium-chrome-driver, selenium-api, jsyntaxpane, cglib-nodep, phantomjsdriver, commons-exec, selenium-remote-driver, selenium-htmlunit-driver, htmlunit-core-js, sac, selenium-support, cssparser, htmlunit, gson, nekohtml, guava]

4- Download latest stable version of  Selenium Grid (selenium-server-standalone.jar)

http://selenium-release.storage.googleapis.com/index.html

3

NOTE: If you are using Selenium Grid with your own PC, browser windows will be popout at your PC and will be harder. You should test it on your own PC but you will need an test automation machine. I recommend that, you should use a headless Linux ubuntu server for Selenium Grid execution. The following command must be working successfully.

5- Download your browser driver. We will use Chrome driver in this example.

http://www.seleniumhq.org/download/

4

6- Run Selenium Grid Hub

Open the terminal windows or command prompt. Go to the JAR file location and Execute the following command in the first window. This will start the selenium hub.

java -jar selenium-server-standalone-2.53.1.jar -role hub -port 4444

5

7- Register node to Selenium Grid Hub

Go to the JAR file location and Execute the following command in the second new terminal window. This will start and configure the nodes which you will use it for performance testing. If you need to do a performance testing with 100 concurrent users, you must set maxInstances value to 100. The following code only registers the chrome browsers. You should configure it with mixed browser settings as well.

java -jar selenium-server-standalone-2.53.1.jar -role node -hub http://localhost:4444/grid/register -maxSession 100 -browser browserName=”chrome”,version=ANY,platform=WINDOWS,maxInstances=50 -Dwebdriver.chrome.driver=path/to/the/chromedriver/chromedriver_ForMac

6

7

8- Check your Selenium Grid is up and running correctly

8

9- Run Jmeter from bin/jmeter directory. Add the following elements to your project.

  • Right click to Test Plan and Add Threads (Users->Thread Group)
  • Right click to Thread Group and Add Config Element -> Simple Controller
  • Right click to Simple Controller and Add Logic Controller -> jp@gc-Remote Driver Config
  • Right click to Simple Controller and Sampler -> jp@gc WebDriver Sample
  • Right click to Simple Controller and Sampler -> jp@gc WebDriver Sample
  • Right click to Simple Controller and Listeners -> View Results Tree
  • Right click to Simple Controller and Listeners -> Aggregate Graph

9

10- Click Remote Driver Config and write your selenium grid URL. You should change 127.0.0.1 to your real existing selenium grid server IP/url. Change capability to Chrome. If you want to use different browser, you must reRegister node to Grid Hub with different settings.

http://127.0.0.1:4444/wd/hub

10

11- Click WebDriver Sample on the left menu. You should write your test scenarios with a lot of scripting languages to here. You should rename your webdriver samples for understandable test scenarios.

There are some useful examples @this link:

https://jmeter-plugins.org/wiki/WebDriverSampler/

11

12- Write test asserts

Right click to any Webdriver Sampler and Add Assertions→ Response Assertion.

Right click to any Webdriver Sampler and Add Assertions→ Duration Assertion.

Right click to any Webdriver Sampler and Add Assertions→ Size Assertion…..

12

Note: Any response codes except 2xx or 3xx will fail automatically. For example 500 Server Error response code will be automatically fail by Jmeter.

13- View request and response details.(View Results Tree)

13

14- View Overall test results listed or graphical. (Aggregate Graph)

14

15- Run a Load Test with concurrent 10 users(10 Thread). And You should increase this value step by step. Set Loop count for longer testing.

15

That’s it ! :)

16

Note: Don’t forget that you must disable or Configure for “Log/Display only Errors” View Results Tree if you don’t use it. If you don’t do that while running load tests with multiple users, You should get not enough memory or memory leaks problems.

Thanks.
Ozgur

12 thoughts on “Selenium Performance Testing with JMeter and Selenium Grid”

  1. Hi Ozgur, thanks for a wonderful article! I tried to follow the instructions mentioned. However I am facing following error. I am using Ubuntu 14.04 as my Operating System. If you have any information, can you please share.

    ERROR – jmeter.threads.JMeterThread: Test failed! org.openqa.selenium.WebDriverException: Error forwarding the new session cannot find : Capabilities [{proxy={proxyAutoconfigUrl=null, autodetect=false, socksUsername=null, socksPassword=null, httpProxy=null, proxyType=SYSTEM, noProxy=null, ftpProxy=null, socksProxy=null, hCode=XXXXXXXXX, class=org.openqa.selenium.Proxy, sslProxy=null}, browserName=chrome, javascriptEnabled=true, chromeOptions={args=[], extensions=[]}, version=, platform=ANY}]

    Caused by: org.openqa.selenium.WebDriverException: Error forwarding the new session cannot find : Capabilities [{proxy={proxyAutoconfigUrl=null, autodetect=false, socksUsername=null, socksPassword=null, httpProxy=null, proxyType=SYSTEM, noProxy=null, ftpProxy=null, socksProxy=null, hCode=342647017, class=org.openqa.selenium.Proxy, sslProxy=null}, browserName=chrome, javascriptEnabled=true, chromeOptions={args=[], extensions=[]}, version=, platform=ANY}]
    Build info: version: ‘2.52.0’, revision: ‘4c2593cfc3689a7fcd7be52549167e5ccc93ad28’, time: ‘2016-02-11 11:22:43’
    System info: host: ‘hostname’, ip: ‘127.0.1.1’, os.name: ‘Linux’, os.arch: ‘amd64’, os.version: ‘3.13.0-107-generic’, java.version: ‘1.8.0_66’
    Driver info: driver.version: unknown
    at org.openqa.grid.web.servlet.handler.RequestHandler.process(RequestHandler.java:115)

    Reply
  2. Hi, I’ve tried to follow the instructions, however it fails with the following:
    Test failed! org.openqa.selenium.WebDriverException: Error forwarding the new session cannot find : Capabilities [{proxy={proxyAutoconfigUrl=null, autodetect=false, socksUsername=null, socksPassword=null, httpProxy=null, proxyType=SYSTEM, noProxy=null, ftpProxy=null, socksProxy=null, hCode=313338742, class=org.openqa.selenium.Proxy, sslProxy=null}, browserName=chrome, javascriptEnabled=true, chromeOptions={args=[], extensions=[]}, version=, platform=ANY}]
    Command duration or timeout: 29 milliseconds
    Build info: version: ‘2.52.0’, revision: ‘4c2593cfc3689a7fcd7be52549167e5ccc93ad28’, time: ‘2016-02-11 11:22:43’
    System info: host: ‘OPT-LAB-APP’, ip: ‘10.170.220.34’, os.name: ‘Windows Server 2012 R2’, os.arch: ‘amd64’, os.version: ‘6.3’, java.version: ‘1.8.0_111’
    Driver info: org.openqa.selenium.remote.RemoteWebDriver

    Reply
  3. Now I’m confused since I’ve installed and run Selenium Grid Hub 2.53 as suggested in the article. The 2.52.0 is the selenium drivers version of jar plugins files in the Jmeter /lib folder, so I realize that it is what is actually working although I’ve configured jp@gc-Remote Driver Config as suggested

    Reply
    • The latest version is 3.8.1 you can find it here: http://docs.seleniumhq.org/download/ Actually, I am not the owner of the article but maybe this version will solve your problems. If not, I will ask Ozgur for help. Use both JMeter’s and Selenium’s latest versions and try again. Libraries are changing regularly and old codes or configs are not working sometimes. As I know, until selenium 3.7.X Capabilities are changed with Options.

      Reply
  4. I’ve deployed the latest version is 3.8.1, but it doesn’t matter. The both selenium-remote-driver-2.52.0.jar from Jmeter /lib folder and Selenium Stanalone are involved, at least I can see the Selenium error in Windows command line where I’ve registered it, and at the same time the error log in the Jmeter referencing to 2.52.0. Moreover, if I delete selenium-remote-driver-2.52.0.jar from lib I can’t add jp@gc-Remote Driver Config element in the Jmeter…

    Reply
  5. Hi to all, Thanks for the update Nir. And Alex this article was written last year on my macbook. I am tried these all steps on my windows with latest and it worked after some changes. So I recommend you, Please use the latest versions all of the required items. Because I have used Chrome Driver v2.2 with this article, But I should say you it will not work with latest Chrome vedrsion which has been istalled on your computer and you will get strage errors. And I recommend you give access to Wİndows Defender, firewall and Antivirus programs on your computer. Because it’s blocking some features of the java JAR file. It should cause to fail. Working command for Node registration on Windows is here. There is an ordering change on the commands on Windows…… java -Dwebdriver.chrome.driver=chromedriver.exe -jar selenium.jar -role node -hub http://localhost:4444/grid/register -maxSession 100 -browser browserName=”chrome”,version=ANY,platform=WINDOWS,maxInstances=50

    Reply
  6. Well, these are the steps I’ve done on my Windows Server 2012:
    – added jp@jc Remote Driver and configured it “http://127.0.0.1:4444/wd/hub” selecting Capability “CHROME”;
    – added jp@gc WebDriver Sampler with some WDS simple request example;
    – Installed Selenium Grid Hub 3.8.1 and started it: java -jar selenium-server-standalone-3.8.1.jar -role hub -port 4444
    – Registered as recommended: java -Dwebdriver.chrome.driver=chromedriver.exe -jar selenium-server-standalone-3.8.1.jar -role node -hub http://localhost:4444/grid/register -maxSession 100 -browser browserName=”chrome”,version=ANY,platform=WINDOWS,maxInstances=50
    – Downloaded latest chromdriver.exe 2.35 and added it to Windows environment Variables Path
    – Restarted Windows.
    Running Jmeter, receiving as follows:
    in Windows CLI:
    09:39:12.087 INFO – Got a request to create a new session: Capabilities {browserName: chrome, chromeOptions: {args: [], extensions: []}, javascriptEnabled: true, proxy: {autodetect: false, class: org.openqa.selenium.Proxy, ftpProxy: null, httpProxy: null, noProxy: null, proxyAutoconfigUrl: null, proxyType: system, socksPassword: null, socksProxy: null, socksUsername: null, sslProxy: null}, version: }
    09:39:12.089 INFO – Trying to create a new session on test slot {seleniumProtocol=WebDriver, se:CONFIG_UUID=021b4ab9-9021-404a-b6ee-0a0b878c21f4, browserName=chrome, maxInstances=50, version=ANY, platform=WINDOWS}

    In Jemeter log:

    Caused by: org.openqa.selenium.SessionNotCreatedException: Unable to create new service: ChromeDriverService
    Build info: version: ‘3.8.1’, revision: ‘6e95a6684b’, time: ‘2017-12-01T19:05:32.194Z’
    System info: host: ‘OPT-LAB-APP’, ip: ‘10.170.220.34’, os.name: ‘Windows Server 2012 R2’, os.arch: ‘amd64’, os.version: ‘6.3’, java.version: ‘1.8.0_111’
    Driver info: driver.version: unknown
    Build info: version: ‘2.52.0’, revision: ‘4c2593cfc3689a7fcd7be52549167e5ccc93ad28’, time: ‘2016-02-11 11:22:43’
    System info: host: ‘OPT-LAB-APP’, ip: ‘10.170.220.34’, os.name: ‘Windows Server 2012 R2’, os.arch: ‘amd64’, os.version: ‘6.3’, java.version: ‘1.8.0_111’
    Driver info: driver.version: unknown

    There is nothing about the latest chromdriver .exe which I ‘ve installed. Which Chrome Driver have you mentioned in your reply ?
    Thanks a lot

    Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.