In agile world, you need to manage your test process with very lean and fast tools. At Kariyer.net we searched many solutions for test case management and our first priorities are two-way JIRA integration, ultra-fast and easy usage, short learning path and period, strong reporting and monitoring capabilities, lean and simple UI, quick and helpful support and informative documentation, and so on… Finally, we finalized this process and selected TestRail as our test case management tool.

In this article, I explained how we started to manage test cycles during two weeks of agile scrum sprints with TestRail.

I notice you again, this not a complete and comprehensive TestRail documentation, the explanations and knowledge in this article is our way to use TestRail. (When I prepared this article TestRail version was 5.2.1.3472)

Note: TestRail 5.3 announced! Please check the new features on https://blog.gurock.com/testrail-5-3-agile-test-management/

For extensive TestRail documentation please visit below link.
http://www.gurock.com/testrail/support/

Special Thanks: I thank Dennis Gurock for his help and support during and after our evaluation period.

TestRail Screens

Dashboard Screen

When you logged in TestRail, you first see the projects page as dashboard. In this page, you can add a new project from left menu. Also, you can see the activity graph for all projects for the last 7-14-30-60-90 days.

When you want to go the project details, you should simply click the project name.

testrail

When you go to the project page, you will see a page which is shown below. In this screen, you can access milestones, test runs, test plans, test cases, and reports and see project activity graph.

Milestones: We can collect many sprints in one milestone and manage them.

Test Plans: You can use Test Plans as Agile Sprints.

Test Runs: Test Runs are the test executions/runs in a test plan (sprint).

Test Cases: Test Cases are our test scenarios according to user needs.

Reports: You can create customized and detailed test reports on this page.

testrail

TestRail comprises of 7 tabs. These are summarized below.

Overview: General view of the projects

Todo: In this page, you can see the TODO tasks which are assigned to the specific user.

testrail

Milestones: You can merge and collect your test plans into milestones.

Test Runs & Results: It comprises of all test plans (sprints) and the test execution cycles (test runs) and their results.

testrail

Test Cases: You can write and organize your test cases on this page.

Reports: All kind of detailed reports will be generated on this page.

Administration: You can manage and do all kind of settings in this page. Only admin users can see this tab.

testrail

Step by Step Test Rail Usage in Agile (Scrum) Sprints

1- Preparing Test Cases

First, click “Test Cases” tab and then click “Add Section” to add a section. It can be our sprint name. We should order and organize our test cases under sections and subsections. We are using below way to organize test cases.

Sprints = Sections

Stories = Subsections

In order to provide standardization, we use a naming format for sections.

Section Naming Format: Sprint-{$Current_Sprint_Number}-{$Sprint_Deadline}

Example Section Name: Sprint-142-2016.07.29

You can find your actual sprint number in JIRA. In order to get this, go to JIRA Agile board and click the icon which is shown on below image.

testrail-6.1

testrail

testrail

After the declaration of a section, we need to create stories (subsections) and under subsection, we should create test cases of the related story. We can do this by clicking “Add Subsection” link and then a subsection will be automatically created. We should also be careful to give a name to the subsection.

Subsection Format: JIRA Ticket ID: Ticket Summary

Example: ISVRNSOA-103: Uygunluk – Pozisyona göre puanlama

testrail

test rail

After this step, we can add our test cases under the subsections by clicking “Add Case” link.

testrail-11

testrail-12

After that we can click edit button to write our test cases.

testrail-13

testrail-14

You can click arrow icon which is shown on below image and open “3 Column View Mode”. In this view mode, you can see the details of each test case.

Note: I hope in future releases of TestRail, we can also edit test cases in this mode. It will increase our test case writing speed dramatically.

testrail-15

testrail-16

Also, you can filter sections and subsections with the filter which is located at the left pane.

testrail-17

You can configure test case column by clicking Columns which is shown on below image.

testrail-18

testrail-19

testrail-20

testrail-21

testrail-22

2- Create a Test Plan and Run Test Cases

Test Plan should be our Sprints. In order to add a test a test plan, click “Test Runs & Results” tab and then click “Add Test Plan” button.

testrail-23

We should be careful about naming convention again for the test plans. We are using “Sprint-$Sprint_Number-$Sprint_Deadline” as a naming format.

Example: Sprint-142-29.07.2016

testrail-24

After creating test plan, we need to create a test run and add test cases into it. After this step, we can execute test cases. To add a test run, click “Edit” button of test plan and then click “Add Test Run(s)” button on the left pane.

testrail-25

testrail-26

We can add test cases by clicking “select cases” link which are shown on below image.

testrail-27

testrail-28

Also, we can change the assignee of the test cases by clicking “change” link.

testrail-29

testrail-30

testrail-31

testrail-32

At last step, we click “Save Test Plan” button and click “OK” button to save settings.

testrail-33

Now, our test plan and test run are ready. We can click Test Run and go to Test Run screen and execute our test cases.

testrail-34

When we click Test Run, we will see below page. In here we can see test cases scenarios and their results/status. We can change test case statuses by clicking them.

testrail-35

In order to change the result of multiple test cases, we should select the related test cases by clicking the check-boxes which are located right side of them, then click “Add Results” button and change their results.

testrail-36

When the tests are executed, their statuses changes simultaneously.

testrail-37

In order to switch “3 column view” mode, we click > arrow icon. We can see the details of test cases and execute them in this mode.

testrail-38

testrail-39

References are our stories. When you hover on them, you can see their details.

testrail-40

3- Create Test Reports

In order to create a report, click “Reports” tab. In Reports page, you can see many report options and types in the right pane. If you want to report a high-level sprint summary report, you can click + sign of Plan under the Summary section.

testrail-41

After clicking + sign, select related test plan (=sprint) in Report Options section.

testrail-42

You can add test case columns by clicking Tests tab and “Add Column” link.

testrail-43

testrail-44

You can configure the privacy settings and scheduling period of your report at “Access & Scheduling” section.

testrail-45

If you want to schedule your report for a specific periodical time, you need to select check-boxes.

testrail-46

Also, you can send the report as a mail.

testrail-47

And finally, when you click “Add Report” button, TestRail will start to create your test report.

testrail-48

Also, you can use report link to share the report to your colleagues.

testrail-49

4- TestRail JIRA Integration

TestRail and JIRA have two ways integration. You can do the settings in administration page. You can see the details in this page. In TestRail:

– You can reach story content by writing story ids in “References” text boxes.

– You can see the content preview of the defects by writing defect ids into “Defects” text boxes.

– You can automatically open defects inside TestRail by clicking “push” link at defects section.

testrail-50

testrail-51

We can also see the test cases names, and test results in JIRA on related story page.

testrail-52

testrail-53

In TestRail section in JIRA you can see dashboard, projects etc.

testrail-54

You can do many more things with TestRail, I wanted to describe the basics in this article. I hope this helps you for you AGILE Test Management life cycle. Thanks. 🙂