Agile Test Management with TestRail


In agile world, you need to manage your test process with very lean and fast tools. At Kariyer.net we searched many solutions for test case management and our first priorities are two-way JIRA integration, ultra-fast and easy usage, short learning path and period, strong reporting and monitoring capabilities, lean and simple UI, quick and helpful support and informative documentation, and so on… Finally, we finalized this process and selected TestRail as our test case management tool.

In this article, I explained how we started to manage test cycles during two weeks of agile scrum sprints with TestRail.

I notice you again, this not a complete and comprehensive TestRail documentation, the explanations and knowledge in this article is our way to use TestRail. (When I prepared this article TestRail version was 5.2.1.3472)

Note: TestRail 5.3 announced! Please check the new features on https://blog.gurock.com/testrail-5-3-agile-test-management/

For extensive TestRail documentation please visit below link.
http://www.gurock.com/testrail/support/

Special Thanks: I thank Dennis Gurock for his help and support during and after our evaluation period.

TestRail Screens

Dashboard Screen

When you logged in TestRail, you first see the projects page as dashboard. In this page, you can add a new project from left menu. Also, you can see the activity graph for all projects for the last 7-14-30-60-90 days.

When you want to go the project details, you should simply click the project name.

testrail

When you go to the project page, you will see a page which is shown below. In this screen, you can access milestones, test runs, test plans, test cases, and reports and see project activity graph.

Milestones: We can collect many sprints in one milestone and manage them.

Test Plans: You can use Test Plans as Agile Sprints.

Test Runs: Test Runs are the test executions/runs in a test plan (sprint).

Test Cases: Test Cases are our test scenarios according to user needs.

Reports: You can create customized and detailed test reports on this page.

testrail

TestRail comprises of 7 tabs. These are summarized below.

Overview: General view of the projects

Todo: In this page, you can see the TODO tasks which are assigned to the specific user.

testrail

Milestones: You can merge and collect your test plans into milestones.

Test Runs & Results: It comprises of all test plans (sprints) and the test execution cycles (test runs) and their results.

testrail

Test Cases: You can write and organize your test cases on this page.

Reports: All kind of detailed reports will be generated on this page.

Administration: You can manage and do all kind of settings in this page. Only admin users can see this tab.

testrail

Step by Step Test Rail Usage in Agile (Scrum) Sprints

1- Preparing Test Cases

First, click “Test Cases” tab and then click “Add Section” to add a section. It can be our sprint name. We should order and organize our test cases under sections and subsections. We are using below way to organize test cases.

Sprints = Sections

Stories = Subsections

In order to provide standardization, we use a naming format for sections.

Section Naming Format: Sprint-{$Current_Sprint_Number}-{$Sprint_Deadline}

Example Section Name: Sprint-142-2016.07.29

You can find your actual sprint number in JIRA. In order to get this, go to JIRA Agile board and click the icon which is shown on below image.

testrail-6.1

testrail

testrail

After the declaration of a section, we need to create stories (subsections) and under subsection, we should create test cases of the related story. We can do this by clicking “Add Subsection” link and then a subsection will be automatically created. We should also be careful to give a name to the subsection.

Subsection Format: JIRA Ticket ID: Ticket Summary

Example: ISVRNSOA-103: Uygunluk – Pozisyona göre puanlama

testrail

test rail

After this step, we can add our test cases under the subsections by clicking “Add Case” link.

testrail-11

testrail-12

After that we can click edit button to write our test cases.

testrail-13

testrail-14

You can click arrow icon which is shown on below image and open “3 Column View Mode”. In this view mode, you can see the details of each test case.

Note: I hope in future releases of TestRail, we can also edit test cases in this mode. It will increase our test case writing speed dramatically.

testrail-15

testrail-16

Also, you can filter sections and subsections with the filter which is located at the left pane.

testrail-17

You can configure test case column by clicking Columns which is shown on below image.

testrail-18

testrail-19

testrail-20

testrail-21

testrail-22

2- Create a Test Plan and Run Test Cases

Test Plan should be our Sprints. In order to add a test a test plan, click “Test Runs & Results” tab and then click “Add Test Plan” button.

testrail-23

We should be careful about naming convention again for the test plans. We are using “Sprint-$Sprint_Number-$Sprint_Deadline” as a naming format.

Example: Sprint-142-29.07.2016

testrail-24

After creating test plan, we need to create a test run and add test cases into it. After this step, we can execute test cases. To add a test run, click “Edit” button of test plan and then click “Add Test Run(s)” button on the left pane.

testrail-25

testrail-26

We can add test cases by clicking “select cases” link which are shown on below image.

testrail-27

testrail-28

Also, we can change the assignee of the test cases by clicking “change” link.

testrail-29

testrail-30

testrail-31

testrail-32

At last step, we click “Save Test Plan” button and click “OK” button to save settings.

testrail-33

Now, our test plan and test run are ready. We can click Test Run and go to Test Run screen and execute our test cases.

testrail-34

When we click Test Run, we will see below page. In here we can see test cases scenarios and their results/status. We can change test case statuses by clicking them.

testrail-35

In order to change the result of multiple test cases, we should select the related test cases by clicking the check-boxes which are located right side of them, then click “Add Results” button and change their results.

testrail-36

When the tests are executed, their statuses changes simultaneously.

testrail-37

In order to switch “3 column view” mode, we click > arrow icon. We can see the details of test cases and execute them in this mode.

testrail-38

testrail-39

References are our stories. When you hover on them, you can see their details.

testrail-40

3- Create Test Reports

In order to create a report, click “Reports” tab. In Reports page, you can see many report options and types in the right pane. If you want to report a high-level sprint summary report, you can click + sign of Plan under the Summary section.

testrail-41

After clicking + sign, select related test plan (=sprint) in Report Options section.

testrail-42

You can add test case columns by clicking Tests tab and “Add Column” link.

testrail-43

testrail-44

You can configure the privacy settings and scheduling period of your report at “Access & Scheduling” section.

testrail-45

If you want to schedule your report for a specific periodical time, you need to select check-boxes.

testrail-46

Also, you can send the report as a mail.

testrail-47

And finally, when you click “Add Report” button, TestRail will start to create your test report.

testrail-48

Also, you can use report link to share the report to your colleagues.

testrail-49

4- TestRail JIRA Integration

TestRail and JIRA have two ways integration. You can do the settings in administration page. You can see the details in this page. In TestRail:

– You can reach story content by writing story ids in “References” text boxes.

– You can see the content preview of the defects by writing defect ids into “Defects” text boxes.

– You can automatically open defects inside TestRail by clicking “push” link at defects section.

testrail-50

testrail-51

We can also see the test cases names, and test results in JIRA on related story page.

testrail-52

testrail-53

In TestRail section in JIRA you can see dashboard, projects etc.

testrail-54

You can do many more things with TestRail, I wanted to describe the basics in this article. I hope this helps you for you AGILE Test Management life cycle. Thanks. 🙂

 

javafx

Database Operations in JavaFX

By Onur Baskirt / Apr 1, 2016 / 56 Comments
Before started this section, please check the first article and learn How to Start JAVAFX! http://www.swtestacademy.com/getting-started-with-javafx/ At first, part of JavaFX tutorial series, we created a sample JavaFX project, designed the draft version of the UI and set up an...
rest assured

REST API Testing with Rest Assured

By Onur Baskirt / Mar 8, 2016 / 32 Comments
Outline In this post, I will explain what is API and API testing, what is the difference between SOAP and REST services, and how to test REST APIs with Rest Assured Library. What is API? API stands for Application Programming...
extentreports

How to Write Effective CSS Locators

By Onur Baskirt / Oct 1, 2017 / 0 Comments
Hi all, in this tutorial, I will describe you how to write effective CSS locators to interrogate web elements for your automation projects. As a rule of thumb, your interrogation strategy should be in below order: First try to use...
extentreports

Selenium-11: Execute JavaScript with JavascriptExecutor

By Onur Baskirt / Jan 27, 2016 / 15 Comments
Outline Sometimes we cannot handle some conditions or problems with Webdriver, web controls don't react well against selenium commands. In this kind of situations, we use Javascript. It is useful for custom synchronizations, hide or show the web elements, change...
javafx

Getting Started with JavaFX

By Onur Baskirt / Mar 25, 2016 / 0 Comments
When I started to work in my current position, one of my task is to do manual operations for campaign products  every week. After the second week, I thought that I have to automate this task using a GUI based...
extentreports

How to Write Smart XPath Locators

By Onur Baskirt / Sep 24, 2017 / 4 Comments
Hi all, in this tutorial, I will describe you how to write smart and non-brittle XPath locators. When we write our test scripts, we generally prefer to use id, name, class, etc. these kinds of locators. However, sometimes we could not...
extentreports

Selenium-1: Quick Start to Automation with Selenium WebDriver & JAVA & JUnit & Maven & IntelliJ

By Onur Baskirt / Sep 8, 2015 / 26 Comments
Outline Selenium Webdriver is the most popular open source web test automation framework across wide range of browsers and platforms. In this tutorial you will learn how to do web test automation with Selenium Webdriver and the related tools. Audience...
page object model

Page Object Model with C#

By Ege Aksoz / Jun 18, 2017 / 8 Comments
In the previous tutorial, we’ve taken the initial steps and entered the world of automated testing. We also wrote our first automated test. From this point on, since we are not just going to write one test, we need to...
extentreports

How to Select a Date From DatePicker Using Selenium

By Onur Baskirt / Aug 13, 2016 / 6 Comments
When you need to automate a airway, hotel, or similar websites you need to deal with Datepickers and some times it is a little bit cumbersome to select a specific date on the Datepicker or calendar.  In this post, I...
extentreports

Selenium Webdriver Performance testing with Jmeter and Selenium Grid

By Ozgur Kaya / Aug 9, 2016 / 2 Comments
In this post, we will complete Selenium Webdriver Performance testing scenario using Jmeter and Selenium Grid. 1- Install Java 7 or later If necessary https://java.com/tr/download/ 2- Download latest Jmeter version 3.0 or higher. http://jmeter.apache.org/download_jmeter.cgi 3- Download Jmeter PluginsManager JAR file and...
By | 2017-01-13T21:33:42+00:00 August 22nd, 2016|Test Management|0 Comments

About the Author:

Onur Baskirt
Onur Baskirt is a senior IT professional with 10+ years of experience. He worked at Bahçesehir University, ST Microelectronics, Huawei and Ericsson as research assistant, design verification engineer and software test leader. Also, he worked as software test leader and software operations manager at Turkey's biggest technology retailer, Teknosa. Now, he is working as Head of Software Testing and Manager of two Development Teams at Kariyer.net. His current research areas are technical software testing, programming, and computer science. Formerly, he has several research works and papers on digital chip design & verification. His hobbies are sport, dancing, traveling, and nutrition. You can find detailed information about him on his linked-in page.

Leave A Comment