Software Quality Report is finally ready! As you may all noticed, we conducted a questionnaire on QA activities at the end of 2018 and after that, we missed 2019 because of our workload and the Covid-19 crisis. One month ago, we decided to conduct another survey, and thanks to all responders, we got more than 60 people from all over the world (Ukraine, India, Netherlands, South Africa, United States, UK, etc…).
Let’s see the results…
How do you describe your company?
Here’s the overview of the participants.
What is Your Title in Your Company?
We are on the right track, most of the participants are working in the QA area. But we think we need to focus on developers in the future as development activities and testing activities start to become more attached to each other since DevOps activities continuously grow.
Which automation activities do you have in the company?
As we can see, almost every company automate their web applications. Then it’s followed by API Automation by 78% compared to 70% of the last year. API is skyrocketing! We can conclude that companies believe the value coming from both activities. 63% of the participants state that mobile automation is in their life. Last year it was 50%. There is a big difference in mobile automation. It’s likely that it will be increased next year also. We still see so many people doing desktop application automation. There is a slight decrease but it is still close to 33% of last year.
Which development language are you using in your automation projects?
Java is still leading the top as development language on test automation projects. This year C# and Javascript changed their position dramatically. Javascript adoption skyrocketed. It was 19% last year and it’s 41%. It doubled its adoption.
This was our prediction of last year :) So we are on the right track.
There are many libraries built on JS, we expect it to be adopted more and surpass C# in near future.
There are many JS libraries published on GitHub with many boilerplate projects. Don’t you dare not trying one of these :)
Edit: Somehow Python is aggregated under Others options, but there are 5 participants who use Python also. Thanks to our careful readers, we realized that mistake and corrected it!
Which automation tools/libraries are you using?
We did not have a dramatic change in the tools and libraries that people are using. Except for Cypress. It increased its adoption from 2.5% to 7% this year and it will continue growing as we see many people adopting it in their projects and Cypress gives a very good signs on new features.
When digging in the responses we realized that people don’t stick one tool and library. They use two, three maybe four different libraries. It’s good to have more tools in your skillset. Don’t forget that not every tool is suitable for every problem you are facing. It better to be a Swiss army knife. But we expect this list to be a little bit stable for a few years :)
According to our responders, we see ConceptJS and Katalon usage too.
For compatibility issues, which browsers are tested in web automation projects?
This question got almost the same responses compared to last year’s survey. We see that priorities are still the same.
How many UI automation test cases does your team/company maintain?
People automate! It’s not easy to maintain a test suite of 250+ but 40% of our responders state that they can do it! That’s great. When we correlate those numbers with the tool stack we can say that all those 250+ tests are developed with only one tool. Many different development languages and tools are playing a huge role to overcome this challenge.
How many API tests did you or your team develop and maintain?
When looking at the numbers, I think that there’s more to improve on API testing. In a world of complicated applications with so many integrations, the number of automated test cases seems less than we expected.
Do you use any cloud providers for web or mobile automation?
Last survey was stating that 60% of the users did not use cloud providers. This year, this rate is 38% which shows that people need cloud providers. It’s not easy to manage all those environments and browser versions. But still, there’s a big portion for cloud providers. BrowserStack is the most adopted tool followed by Saucelabs. There is a big portion who uses their own solution, either custom build grid or open-source tools like Selenoid or Zalenium. There is also a new player which is Amazon. They started to give support to Desktop automation and they already have 10% of our readers.
For Zalenium and Selenoid, you can check below links
https://github.com/zalando/zalenium
https://github.com/aerokube/selenoid
You can read about Selenoid in those articles also
https://www.swtestacademy.com/selenoid-tutorial/
https://www.swtestacademy.com/docker-selenium-tutorial/
Which tools do you use for Performance Testing?
The leading tool is still Apache JMeter with its support for almost all protocols. We see that the number for “No, we don’t do performance tests” decreases from 25% to 19%, it’s not a big number but it’s good to see that people adopt performance testing.
We have a newcomer on the list which is Karate. With its unique support for API, Web, and Desktop testing, they added performance testing capability to Karate. I believe the adoption will increase in any area related to testing in near future.
We plan to do another survey next year. Stay tuned!
Canberk Akduygu
Canberk Akduygu is a Test Lead working in the Netherlands