8 Important Software Testing Techniques


In this article, I will explain important software testing techniques which help you during test and User Acceptance Testing (UAT) processes.

Disclaimer: The knowledge in this article is based on my experiences, understanding, and observation. If you don’t agree on anything in the article, please write a comment and we will discuss the thing that you disagree.

What is User Acceptance Test (UAT)?

Product Owners (PO) and business units generally work on requirements and focus them. Thus, they can use the requirement-based and activity-based test techniques during UAT process. As we testers, we can also use these techniques in test process as well. I am going to explain some of these techniques in this article. But first, let start with what UAT testing is.

In agile methodologies, it is the test activity which performed by generally product owners after the completion of the development and software testing process of the stories. In waterfall and V-model processes, these tests are generally performed by analysts or business units. The purpose of the UAT is to determine whether the user’s requirements for the requested job have been met and to get the approval for the live deployment.

A sample entry and exit criteria for the user acceptance test are summarized in the following table.

Entry Criteria

● Code Written
● Code Reviewed
● Unit Tests Executed and Defects Fixed
● Tests Executed
● Resolved all High and Medium Priority Levels of Defects
● Prepared Test Data to be used in UAT

Exit Criteria

● All UAT Scenarios Executed
● All Acceptance Criteria Tested
● Reached Target Quality Level (> = 90%), (Passed Test Cases / Passed + Failed Test Cases)
● Defects Resolved (Priority> = Major)

1) User Story Testing (AGILE)

A user story can be described as a requested feature that is in the software from the perspective of the end user in agile software development life cycle. In user story, we have to specify the demand, the reason of the demand, and the user who is requesting it.

Definition of Done (DOD) defines the completion criteria such as code is done, unit test is done, testing is done, UAT is done, etc… and Scrum guide states that Scrum team (developers, testers, PO etc.) owns  and  responsible for DOD.

Also, acceptance criteria should be expressed clearly by POs (development team may help PO) and at least one test scenario for each acceptance criteria should be prepared in the test process of a user story and these acceptance criteria has to be tested carefully.

The test entry and exit criteria must also be defined before starting the test run. Here is an example.

Test Entry Criteria

● Does the requested user, the need and the reason of necessity are clearly understood?
● The risk associated with the demand specified in the User Story?
● The impact analysis specified in User Stoy?
● DOD’s acceptance criteria stated clearly?
● Non-functional requirements identified with expected metrics? (Performance, security, etc.)
● If there are integrations of the development, are they specified?
● Development Done?
● Static Code Analysis Done?
Unit Tests were written and defects fixed?
● Code Review?
● Test Scenarios Written?
● Test Cases reviewed by the PO / Analyst?
● Critical tests reviewed and executed with Developers in Dev Environment? (Desk Check)
● Test environment prepared for testing?
● Data required for the test prepared in the test environment?
● Database changes transferred to the test environment?
● Configurational settings applied to the test environment?
● Prerequisites identified before starting the test?

Test Exit Criteria

● All Test Scenarios Executed and All Acceptance Criteria Tested
● Functional and Non-Functional needs were tested.
● Target Quality Level Meet  (> = 85%), (Passed Test Cases / Passed + Failed Test Cases)
● Defects Fixed (Priority> = Medium)

Sample User Story

User Story 1:

As a Product Owner [User], in order to advertise Kariyer.net welcome campaign [reason of demand], I would like the advertisement banner to be added to the top banner section of the Kariyer.net homepage [demand].

Risks:

  • Homepage speed may be reduced
  • A bug in the animation of banner will affect the homepage appearance
  • Continuous deletion of cookies can cause the banner to be continuously visible on the user side.
  • The function of the Close Banner icon critical. It should work continuously and successfully.

Impact Analysis:

  • The banner loading function may be affected in the Admin panel.

Definition of Done:

  • Code Writing Done
  • Code Review Done
  • Unit Testing Done
  • UAT Done

Acceptance Criteria:

  • When Kariyer.net homepage is opened, the top banner is displayed as 200×200 for 8 seconds and then it should be seen as 60×60.
  • When the user clicks on the banner, it should be directed to the Welcome page.
  • If a user visited Kariyer.net more than 4 times from the same computer, AA-kobiBannerClosed cookie value should be 4 and more and banner should not be displayed.
  • The top right corner of the banner must have a cross-shaped closing icon and the banner must be closed when it is clicked.
  • If the banner was previously turned off by the user, it should not be displayed again.

test techniques

2) Use Case Testing

A use case defines the operations that a user/actor performs in the system to achieve a specific purpose. Functional requirements of a system can be defined and managed using use cases. In this way, the scope of the desired or requested job is determined. Tests scenarios are prepared by taking into consideration the inputs and outputs of the steps determined by the user to reach a specific purpose. During the tests, the results of the tests are determined by comparing the expected outputs with the actual outputs.

When writing Use-cases, generally business language is preferred instead of technical language. Therefore they are often used in writing acceptance tests. In order to cover all requirements, at least one test scenario is prepared for each requirement. By this way, test coverage can be increased and we can measure this coverage also by using traceability matrix. In the traceability matrix, we create a matrix table with test scenarios and requirements and put a cross sign in the relevant field if it meets the requirements for each test case. The goal is to cover all the requirements.

Scenario No S.1
Scenario Name: Changing Login Password
Connected Requirements R.4
Affected Screens Home Page, Login Page, Account Page
Prerequisites An existing user on the system
Normal Flow: 1. The user opens the Kariyer.net homepage and clicks on the Member Login button.

2. Login to the site with the existing username and password on the login screen.

3. Click the “My Account Settings” in the pop-up menu.

4. Click the “Change Password” link under the “Personal Information Settings:” section on the Account Settings page.

5. The old password is entered in the password change section.

6. Then, enter the new password.

7. When entering the new password, the security level on the left side is displayed as 3 levels (weak-medium-strong).

8. Enter the new password 2 times as medium-strong and click “Save”.

9. At the end of this process, the password change operation is expected to be performed successfully.

Alternative Flows: 1.       If the current password is entered incorrectly, you will be warned: “You entered your password incorrectly!”.

2.       If a weak password is entered, you will be warned: “You can not change your current password with a weak password”.

3.       You can not change the password by leaving any or all of the password fields blank.

Usage Predictions: It is a scenario that can be used intensively by users. The functionally correct operation is critical.
Hypothesis: Very complex password requirement can prevent us from being user-friendly.
Notes:  

Sample Test Case

For test cases, you can use excel. For the sake of an example, all areas of the first successful test case are shown below.

Test Name      

Changing Password Successfully with (Moderate Complexity)

Test Steps             

  1. Open Home Page.
  2. Click Login Button
  3. Go to Profile and click on “Account Settings”.
  4. Click “Change Password”.
  5. Enter the current password and new passwords (medium).
  6. Click Save button.

Prerequisites

This can be done with an existing user in the system.

Test Data

  • User Name: test@test.com
  • Current Password: kariyer1234+ New Password: asdf1234

Test Priority

High

Expected Results

It is expected that the password will be changed successfully and the information message “Changed successfully” will be informed you that the password has been changed.

3) Checklist Based Testing

In agile processes, we create a general checklist that is created independently from user stories. If there is not any risk is specified in the User Story, all or some of these checklist items may be used according to the scope of the user story. During execution of these tests, if you find a defect, you should extend the scope of the checklist by adding the failed scenario. Thus, we can increase risk items in the checklist for subsequent sprints.

Example

  • All links in the system (Web / Mobile) should work correctly.
  • There should not be a grammar error in the system writings.
  • Font sizes, fonts, should be as expected.
  • There should not be any pictures that can not be loaded/broken in the system.
  • Pictures, text, etc. The alignment between the other components must be as expected.
  • All buttons must work properly and each of them directs the user to the corresponding operation.
  • Each page should have the main page logo and should be redirected to the main page when clicked.
  • Warning, information messages should be displayed in the correct format.
  • If the page is responsive, it should be checked at all resolutions.
  • All components on the site (dropdown, checkbox, radio button, etc.) should work correctly.
  • Special conditions (numeric, alphanumeric, etc.) in the input fields must be checked.
  • Operations can not be performed by leaving the required fields blank.
  • Any operation of the site must not last more than 3 to 15 seconds.
  • … etc.

4) Exploratory Testing

First of all, the exploratory testing is not a random or ad-hoc test. One of the biggest misconceptions about this test technique is that the exploratory testing is perceived as a random, non-testable, non-observable test technique. The exploratory testing is a test approach based on learning and exploring the product at the same time by using the experience, domain knowledge, analytical and intellectual knowledge of a test engineer in agile processes.

Before starting exploratory testing, a preparation should be done. Regardless of exploratory testing method selection, we should prepare a plan for the scope of functionality, tools to be used, test data, environment etc. This plan will guide the tester during test execution process. Another important point of exploratory testing is documentation is fully completed after the tests are finished.

Although it is not mandatory, “session-based testing” technique is generally preferred as an exploratory testing technique. This technique involves the following steps:

Main Activities:

– Test Session Time (It should be a few hours)

– Session Activities

  • Session setup
  •  Test Design and Test Execution
  • Defect Searching
  • Reporting

– The purposes of the test should be specified

– The goal of the test should be specified.

– Functionality included in the test (test report – charter) should be written.

Test Report During & After the Test Process:

– Test Report (Charter) [Specifies the test function.]

– The Person who Performed Test

– Start Date and Time

– Session Metrics (Metrics Collected During Test Implementation and Defect Searching)

– Test Data

– Test Notes

– Results

– Errors

Recommended resources

5) Experienced Based Testing

This testing technique based on the knowledge, skills, and experience of the person who will make the test. In this testing technique; test planning, test strategy, test inputs, and test scenarios are determined by the experience of the person performing the test. In order to prefer this technique, it must be an experienced candidate with sufficient technical and business knowledge to perform this test.

It is easier to understand what is going right or wrong during the test because the experiences gained in past projects are taken into consideration. This person can perform tests by using techniques like exploratory testing, which will make it easier to use past experience and intellectual/analytical knowledge. When we have very short test execution time or there is a lack of sufficient documentation on the project, etc. make sense to use this test technique. If the system being tested comprises of high risks, it is not preferable to use the Experienced Based Testing technique alone because the context of the requirement should be totally covered.

6) User Journey Test

The User Journey test is taking into account the various road maps and journeys of a typical user on the system. In these tests, the most critical journeys a user would make within the site are determined and then the scenarios of these journeys are written. Thus, the user’s interactions with the system are covered as much as possible. These tests are usually “end-to-end” tests, so they may take more time to run than other tests, but the coverage percentage of these tests are higher than the other ones.

Since the “User Journey” tests are comprehensive and extensive tests, the numbers are relatively low compared to other tests. Especially, “user journey” tests can be created taking into account the most critical “use-case” scenarios. The most plausible behavior here is that positive basic scenarios, called “happy-path”, should be run first.

For an example on kariyer.net, it is an important “user-journey” test for a user to enter the site, login, then search for the “sales representative” keyword and apply to the first job ad successfully. In this “user journey”, opening the site, logging in, making a successful call in the search bar, opening a related announcement page and then submitting the application from this announcement page is covered. As you can see here, the broad coverage of the “user journey” tests beneficial in early detection of critical faults in the software development process especially before starting extensive testing. Unlike User Story tests, user journey tests are not tied to user stories. When there are changes introduced by a new user-story, existing user journey tests are updated taking these new changes into account. New user stories rarely cause new user journey tests. For this to happen, new features must be added to the system.

7) Risk-Based Testing

One of the most fundamental objectives of risk-based test techniques is to find the most critical and most important errors as early as possible with lowest cost. Risks are the things that we do not know exactly what will happen, but we know the probabilities of what might happen. Briefly, they are possible problems. When these possibilities are unknown, they are called uncertainties. Therefore, we can think of the general definition of the magnitude of the risk is the multiplication of the likelihood of problems and their impact. Thus, in risk-based tests, we prioritise and test the most error-prone functionalities.

Magnitude of Risk = Likelihood * Impact

The early adoption of the risk-based test approach in projects is important for the early detection of critical problems.

The most basic steps of the risk-based test are summarized below:

1- First, risks are identified and a prioritized risk list is prepared.

2- Make a test plan according to the prioritized risk list and tests are executed for each risk.

3- As a result of the tests, some risks eliminate and some of them arise. New risks are tested by considering their test effort.

At this point, our most fundamental goal will be to find the most important defects.

If you are responsible for testing a product with a very high failure cost, you will need to do a very detailed risk analysis. Statistical models can be used here. One of the most known model is the Failure Mode Effect Analysis (FMEA) model.

In short, if we will be referring to the FMEA model, we calculate the risk score by taking 3 metrics with 5 scales. These:

Severity

Description Importance
Level
Point
Data loss, hardware or security issues Urgent 1
Loss of functionality, not even have a workaround High 2
Loss of functionality, which has a workaround Medium 3
Partial loss of functionality Low 4
Cosmetic or trivial N/A 5


Priority

Description Importance
Level
Point
Total loss of system value Urgent 1
Unacceptable system value loss High 2
Possible reduction in system value Medium 3
Acceptable reduction in system value Low 4
Negligible reduction in system value N/A 5


Likelihood

Description Importance
Level
Point
Negligible reduction in system value Urgent 1
Some users are affected High 2
Possible influences for some users Medium 3
Limited impact of few users Low 4
Impact not to be deducted from actual use N/A 5

 

All these three qualities (Severity, Priority, and Probability) are calculated separately within themselves, then these values are multiplied together to obtain a Risk Priority Number (RPN).

Risk Priority Number (RPN) = S * P * L

Based on this RPN value, we can determine the scope of the test. Lower RPN indicates higher risk.

8) Heuristic Risk-Based Testing by James Bach

(First published in Software Testing and Quality Engineering Magazine, 11/99 Copyright 1999, James Bach)

You can find the details in this link: http://www.satisfice.com/articles/hrbt.pdf

I just want to give a summary of Heuristic Risk-Based Testing.

At the beginning of your projects, your risk analysis may be incomplete or incorrect in some degree because it is not possible to estimate everything 100% at first. However, as your project progresses and your product improves, your estimates and risk analysis will become increasingly stronger. According to James Bach, the two most critical factors for risk are experiences and teamwork. Over a period of time, products or technologies begin to reveal characteristic problems. It is important to observe and learn about them. Besides, it is very critical to do risk analysis with people with different perspectives. Also, using risk-based lists helps us to negotiate with the management about test workforce effectiveness. We can show them that we use existing test workforce for the most critical points of the product or we may need to explain them we need the extra workforce to cover all the risk areas. These risk-based lists give you extra power to negotiate with the management.

In this article, we covered very important test techniques. I hope you enjoyed to read it. All the things in this article based on my understanding and experiences. If you disagree anything, please do not hesitate to write a comment and If you convince me I will fix the thing that you are right. 😉

Thanks.
-Onur

javafx

Database Operations in JavaFX

By Onur Baskirt / Apr 1, 2016 / 56 Comments
Before started this section, please check the first article and learn How to Start JAVAFX! http://www.swtestacademy.com/getting-started-with-javafx/ At first, part of JavaFX tutorial series, we created a sample JavaFX project, designed the draft version of the UI and set up an...
rest assured

REST API Testing with Rest Assured

By Onur Baskirt / Mar 8, 2016 / 32 Comments
Outline In this post, I will explain what is API and API testing, what is the difference between SOAP and REST services, and how to test REST APIs with Rest Assured Library. What is API? API stands for Application Programming...
extentreports

How to Write Effective CSS Locators

By Onur Baskirt / Oct 1, 2017 / 0 Comments
Hi all, in this tutorial, I will describe you how to write effective CSS locators to interrogate web elements for your automation projects. As a rule of thumb, your interrogation strategy should be in below order: First try to use...
extentreports

Selenium-11: Execute JavaScript with JavascriptExecutor

By Onur Baskirt / Jan 27, 2016 / 15 Comments
Outline Sometimes we cannot handle some conditions or problems with Webdriver, web controls don't react well against selenium commands. In this kind of situations, we use Javascript. It is useful for custom synchronizations, hide or show the web elements, change...
javafx

Getting Started with JavaFX

By Onur Baskirt / Mar 25, 2016 / 0 Comments
When I started to work in my current position, one of my task is to do manual operations for campaign products  every week. After the second week, I thought that I have to automate this task using a GUI based...
extentreports

How to Write Smart XPath Locators

By Onur Baskirt / Sep 24, 2017 / 4 Comments
Hi all, in this tutorial, I will describe you how to write smart and non-brittle XPath locators. When we write our test scripts, we generally prefer to use id, name, class, etc. these kinds of locators. However, sometimes we could not...
extentreports

Selenium-1: Quick Start to Automation with Selenium WebDriver & JAVA & JUnit & Maven & IntelliJ

By Onur Baskirt / Sep 8, 2015 / 26 Comments
Outline Selenium Webdriver is the most popular open source web test automation framework across wide range of browsers and platforms. In this tutorial you will learn how to do web test automation with Selenium Webdriver and the related tools. Audience...
page object model

Page Object Model with C#

By Ege Aksoz / Jun 18, 2017 / 8 Comments
In the previous tutorial, we’ve taken the initial steps and entered the world of automated testing. We also wrote our first automated test. From this point on, since we are not just going to write one test, we need to...
extentreports

How to Select a Date From DatePicker Using Selenium

By Onur Baskirt / Aug 13, 2016 / 6 Comments
When you need to automate a airway, hotel, or similar websites you need to deal with Datepickers and some times it is a little bit cumbersome to select a specific date on the Datepicker or calendar.  In this post, I...
extentreports

Selenium Webdriver Performance testing with Jmeter and Selenium Grid

By Ozgur Kaya / Aug 9, 2016 / 2 Comments
In this post, we will complete Selenium Webdriver Performance testing scenario using Jmeter and Selenium Grid. 1- Install Java 7 or later If necessary https://java.com/tr/download/ 2- Download latest Jmeter version 3.0 or higher. http://jmeter.apache.org/download_jmeter.cgi 3- Download Jmeter PluginsManager JAR file and...
By | 2017-05-18T19:45:47+00:00 March 30th, 2017|Test Management|2 Comments

About the Author:

Onur Baskirt
Onur Baskirt is a senior IT professional with 10+ years of experience. He worked at Bahçesehir University, ST Microelectronics, Huawei and Ericsson as research assistant, design verification engineer and software test leader. Also, he worked as software test leader and software operations manager at Turkey's biggest technology retailer, Teknosa. Now, he is working as Head of Software Testing and Manager of two Development Teams at Kariyer.net. His current research areas are technical software testing, programming, and computer science. Formerly, he has several research works and papers on digital chip design & verification. His hobbies are sport, dancing, traveling, and nutrition. You can find detailed information about him on his linked-in page.

2 Comments

  1. BHARAT VARSHNEY March 30, 2017 at 1:50 pm - Reply

    Nice Explanation!!

Leave A Comment