Hi all, this article consists of two main sections these are “Describing the initial situation for your Quality Assurance processes” and “Review & Compare Testing Tools“. Let’s start to elaborate on each of them in detail.

PART 1 – Describing the initial situation for your Quality Assurance

What is expected of you? Do not forget that, Your primary goal is defining, measuring, and improving the quality of the software. Three major steps in the software testing process are planning, execution, and reporting. You are responsible for working as expected applications.

5W1H is the abbreviation summarising the following six questions: What? Who? Where? When? Why? How? This method consists of asking a systematic set of questions to collect all the data necessary to draw up a report of the existing situation with the aim of identifying the true nature of the problem and describing the context precisely. That should be asked for every project! (5W1H).

Below, I used the 5W1H method as a quality problem-resolution support tool.

Questions & Answers

  • WHAT do you need?

 I want to manage all test processes with a single tool with less risk and less effort. Should also support Test Automation,  Manual Testing,  Android & iOS Mobile Testing, Web Testing API Middleware Testing and everything must be integrated.

  • WHO will use this tool?

All test team members and all other team members on projects.

  • WHERE do you want to use it?

Test Automation, Remote Device, Beta Testing, UAT Testing, Manual Testing

  • WHEN do you need this tool?

Getting fast visual feedback. Writing manual test cases and automating them. Finding more bugs. Having a speedy and efficient process.

  • WHY do you need a Testing Tool?

Tool A is good on Manual Testing, Tool B is good on Automation, Tool C is good on Mobile Device Farm, Tool D is good but difficult to install and use, Tool E requires XScript knowledge, Tool F is the best but you can’t use tool F on your computer’s OS, Tool G is the best for Web, Tool H is free but installation & configuration will take 1 day for you and new joiners, Why you are using Tool I, just try J tool, Tool K has no test reporting abilities,  infinite loop .. I don’t have enough time anymore for researching & solving the problems in the existing solution that I use. Believe me, the letters in the alphabet are not enough. There is a critical need, especially in mobile tests.

  • HOW can I use it?

All popular test automation frameworks and coding languages should be supported for Automation. Mobile, Web, and API applications under testing should be supported. I want to use the tool on all Macbook / Windows / Linux OS, not limited. I want to use only Real Mobile Devices for mobile testing, not simulators/emulators. Creating codeless test scenarios should be perfect for some team members, but I want to write code when I want. I want to convert manual test cases to automated test cases when I need to. I want to do end-to-end cross-testing. I want to run parallel tests with different browsers and real devices. Easy installation and usage needed. Actually I don’t want to install anything on my computer. Built-in tool integrations (JIRA, Jenkins, Slack, Mail, Git, AWS Device Farm,..), Built-in TestData (Database, CSV, Excel,..) integrations.

Let’s dive deeper, and let’s get to know the problems that you will encounter in the future with Open Source Testing Tools.

Open-source testing tools requirements:

The most popular tools for test automation are Selenium, Appium, Cucumber, Jmeter, Postman.

1. Hardware Requirements

A powerful Macbook needed for automation engineers. Because You will need lots of things for development and debugging test automation. Mac-mini, USB extender, iOS, and Android Devices. Some guys still have to code with VBscript and use Windows laptops. It’s not your fault 🙁

Conclusion: There must be a different and common way of doing this, we can’t dedicate all devices to all engineers. First of all, it should be in a shareable environment.

2. Software Installations & Configurations

Example for Installation of Appium and all dependencies on your computer for mobile test automation development

Conclusion: I am not a mobile developer, do I really have to install so many things on the MacBook? I am installing more than an Android or iOS developer. How will I keep them updated? How do I upgrade to the new SDK version?

3. Get Required Access Permissions

  • Test Database Access Permissions
  • Backend System Access Permissions
  • Git Repository Access
  • Test Management Tool Access
  • Issue management Access
  • Notification app Access

Conclusion: If I have just started this job, I may not even know which systems I should have access to. It will also take a while for anyone new to the team to open, close, or update account/access to these systems. there must be a definitive solution to doing this in a different way.

4. Inspect Existing Test Automation Project

  • *Automation Project’s Programming Language and Framework.
  • Supported Platforms (iOS, Android, Web, API, Hybrid, Desktop..)
  • Test Automation Scenarios
  • Merge and branching strategies

Conclusion:  Suppose that You feel better with Javascript automation projects. But The existing project had been written in JAVA. It will take time to learn. You can’t change the framework or language. There are thousands of lines of codes and a sprint has been started. Time is not enough for all and the team is not ready for big changes. There are common challenges in your code. False Positives & False Negatives (Flaky Tests) And your existing test automation framework doesn’t support API testing. Let’s over engine this issue. Where is the right place for your mobile automation code? Should it be in the app or outside of the app?

5. Application package to be tested

*You have to test the right packages at all times. You have to access APK(Android) and IPA(iOS Real Device) files on mobile projects.

*Package from the Latest Branch / Specified branch

Conclusion:  Will I compile and create the application package to be tested, or will it come to me automatically or manually from the continuous integration tool?

6. Emulator vs Simulator vs Real Device

  • Emulators and Simulators are free and they have no duration limitations like cloud services. But can’t replicate real user conditions like hardware and software configurations, work slower than real devices, consume a lot of memory and CPU on your computer.
  • Emulators / Simulators are good at Unit Testing, Static Code analysis, development. They are weak at UI Regression Testing and Compatibility testing

Conclusion:  Real devices provide test runs under real-world conditions. If your test is passing on an emulator/simulator that doesn’t mean it will pass on Real Device.

7. On-Premise (In-House) vs Cloud Device Farm

You need a device farm for compatibility testing and high device coverage.

Conclusion: Cloud farms have got limitations, pricing, and slowness problems. Can I believe that the mobile phone on which I ran the test is real and only dedicated to me? I prefer other users not to access my application and automation code.

8. Interactive Live Mobile App Testing on Real Device

  • We are going to automate 100 percent of our regressions.
  • Automation and manual test engineers are independent teams.

Conclusion: Manually exploratory testing is very important now and will remain so in the future. There is a problem of device sharing between manual and automation test engineers. Devices in on-premise device farms also have the same problem. You should not separate Manual and automation engineers. The goals of both teams are the same. Measuring the quality of the software. There is actually 1 team. You should not automate everything, Welcome to the real world.

9. Start to Automate

  • Inspect ID, XPATH,.. of the mobile elements on APP, and write your automation code on IDE. 
  • Element locator problems (There is no ID or Name, Dynamic ID’s,..)

Conclusion: Using Appium with your favorite development IDE (IntelliJ, Pycharm, WebStorm, VisualStudio, Eclipse, Ride.. ) Can’t we do this with a single application instead of 2 different client applications with hours of installation and configurations? Should we centrally manage the Appium server and client applications? Can I get support from the development team for element locator problems? HTML concepts and basic programming knowledge are required. Reusable elements can help you.

10. Integration Requirements

  • JIRA issue management / Slack notification / Email / Jenkins..
  • Auto-fill and submit test run Defects to Jira
  • Basic reporting with Email
  • In Real-Time Get Slack notification messages for test execution.