Hi all, in this article I will share my experience in microservices testing. Most of the companies are implementing the microservices architecture to have the capability to develop, test, and deploy the services independently and faster. Of course, these architectures come with a lot of complexities and in order to test these systems effectively, we need to aware of the system architecture very well.

Let’s start with a simple architectural view of microservices. Generally, we have clients/channels/consumers such as Web, Mobile Web, Mobile Apps (iOS and Android), and Desktop. We may have some downstream or external services which do business-critical operations such as loyalty operations, customers’ data-related operations,  and they may hold some critical data of the business. These operations and data depend on the company’s sector. Between clients and the external systems, we have a middleware layer which does communication, translation, and some business operations as well.

microservices top level architecture

In the above architecture, at the middleware level, we may have several services. They communicate with each other, clients, and external systems. In order to test this layer, we have to know also the internal architecture of middleware. Generally, it generally consists of API gateway || service mesh, microservices, services’ data stores, and some other elements such as message queues.

Requests from the channels are generally forwarded and routed to the services via Gateway. It has to be tested very well because all communication is going through the gateway. After the gateway level, we have microservices. Some of them connected with each other ( we can call them composite services), also they have interaction with data stores, external systems, and of course the API gateway. In an ideal microservices architecture, each service has its own database.

I tried to explain the very high-level architecture of microservices because I want to focus on more testing of microservices. In a typical microservices architecture, we need to deal with the below areas and more. These challenges also create complex testing needs, types, and solutions as well.

microservices architecture

As you have seen, we have multiple layers and components in this architecture thus we need to modify our “famous test pyramid” and we will have several test types. I modified the test pyramid for microservices testing as follows. This pyramid does not consist of non-functional testing types such as load testing, stress testing, spike testing, load balancing testing, chaos testing, data replication testing, accessibility testing, security & vulnerability testing, etc. These testing types also should be under the testing umbrella of microservices testing. If there are also UI elements that exist in our scope, we need to do UI related testing such as UI automation, visual testing, accessibility testing, etc. But UI related tests mostly belong to channel testing.

microservices test pryramid

I want to proceed with explaining all of these test types one by one. With these testing types, we will test the microservices effectively but before going further, I would like to share some categorization about these tests as Pre-Deployment and After-Deployment tests.

Pre-Deployment Tests

These tests should be written inside the project of the service (i.e. under the test package of the service) and for each service CI/CD pipeline we should run them before deployment phase. A sample view of these packages may look like below.

microservices testing types

If you have a JAVA based microservices and if you use maven as a dependency management solution, you can initiate all of these tests in CI/CD pipeline in the below order by using maven commands. Also, if you use tools like ansible etc. you can put this mvn commands in your playbook script and manage your stages. For example, you can trigger component tests with below mvn command:

Also, you can modify these commands based on your needs and microservice requirements. Then in the pipeline, they can run in below order:

microservices pipeline

Now, lets elaborate on these testing types more.

Unit Tests

A unit test focuses on a single “unit of code” to isolate each part of the program and show that the individual parts are correct. Generally, developers are writing unit tests by using unit testing libraries such as JUnit, mockito, etc. They are directly calling the implementation methods in unit tests. They do not need to start the service locally and hit the service via its endpoint. They can directly call the implemented methods for each unit level functional test.

unit tests

Integration Tests

They verify the communication paths and interactions between components to detect interface defects. Such as data store connections. For integration tests, we also do not need to start the service locally. We can call any implemented method for the required test but this time we need to check the integration with the external systems such as a DB connection or other service connection. In our test scope, there should be an external system integration.

integration tests

Component Tests

In a microservice architecture, the components are the services themselves. Here we need to isolate each component (service) from its peers or collaborators and writing tests at this granularity. We can use tools like WireMock to mock the external system or other services. Also, we can use in-memory databases to mock DBs but this will create a bit more complexity. Ideally, we should mock all the external dependencies and test the service in isolation. In component tests, we should start the service locally and automatically (If u have a Reactive SpringBoot WebFlux project, you can use WebTestClient and you can use @AutoConfigureWebTestClient annotation to do this) and when service started, we should hit its endpoint to test our functional requirements.

We need to cover most of the functional test scenarios at this level as much as we can because component tests run before deployment and if there is a functional problem that exists in our service, we can detect this before deployment. It complies shift-left approach and early testing.

component

Contract Tests (Consumer-Driven Contract Tests)

Contracts are a series of agreements between the service provider and the consumer. The providers can test their service against these contracts to make sure they are not breaking any consumers thereby helping to keep the service backward compatible.

contract tests

If you use Spring Boot then you can use Spring Cloud Contract or you can use PACT for contract testing. Pact was written for Ruby but then it is now available for many different languages e.g. JavaScript, Python, and Java.

Before starting contract tests, there should be an agreement between consumer/channels and providers/middleware/external systems. Then, we should start to use the defined contracts to write contract tests.

Post-Deployment Tests

System Integration Tests (E2E Tests)

System Integration tests (SIT) ensure that we are building the right system and how the application behaves. We can test the flow of an application right from start to finish is behaving as expected. We should not use any mocks/stubs for system integration tests. All system components should be integrated with each other. In this way, we can ensure that system-level integration is working as expected. These tests take more time than others. Therefore, we need to test the critical business flows in the SIT level.

system integration testing

If you have JAVA based tech stack, you can use rest-assured to write SIT tests. For these tests, we should better follow the below approach.

  • Backlog prioritization.
  • Gathering high-level business rules. (specs/tests)
  • Elaborating those business rules and creating the steps of them. (steps)
  • Gathering required test data, endpoints, and expected results. (data & assertions)
  • Starting to write test scenarios based on the above inputs. (coding)

After exploration and discovery, scenarios formalized and specs (tests) will be formed as shown below. You can also cucumber to add a Gherkin flavor to your tests or you can create your own step classes to hide the details inside step classes’ methods. Then, tests will look more readable as shown below.

microservices testing flow

test automation scripting in microservices

Performance Tests

After deployment to test or your stage environment (we are doing PT in the stage environment and SIT for test environment), you can start to run your service level performance testing. In microservices, I suggest testing the service performance in several ways. First, better to mock all dependencies as much as possible and hit the service endpoint directly to measure standalone service performance testing. This test will give you an isolated service performance which means you focus the service’s performance.

For these tests, you can use several technologies or tools such as Gatling, JMeter, Locust, Taurus, and so on. If you have a JAVA based stack, I suggest you go with Gatling or JMeter. If you choose Gatling, better to create a maven based project and integrate your tests in your pipeline. Also, we should check the server behavior as well by using some APM tools such as NewRelic, DynaTrace, AppDynamics, etc. These tools give extra details and it is nice to use them along with performance test results.

performance testing

Secondly, we will test the service performance by hitting the service over the gateway. This time we will add the gateway factors in our performance tests. In real life all requests from the channels pass-through the gateway to the services. That’s why it is also important to do performance testing over the gateway. Here we may not use mocks/stubs but if we have any communication with external systems, we should think about performance test data very carefully before starting performance tests and if external systems are slow, it will affect our performance test results and by using APM tools we should figure out the problematic areas.

If we have hybrid server architecture, which means if some of the servers are on-premise and some of them are in the cloud (AWS, Azure, etc.), it is better to do performance testing for each server. In this way, we will have an idea about on-premise and cloud performances.

Also, in a hybrid architecture, there will be data replications between on-premise and cloud servers. Therefore, we need to do XDCR (Cross Data Center Replication) tests to find out the maximum performance of the data replication. For example, we will create a shopping cart on-premise, read it in the cloud server, update it on-premise, delete a product in the cloud, read it on-premise again. We can generate many scenarios and all of these scenarios will run flawlessly with given pause intervals between each operation. We need to fine-tune that pause period and it should be bigger than data replication time.

Also, we need to perform extra performance tests. They may not be in pipeline but we need to think about them for evaluating system performance and stability. I suggest us to perform spike testing by applying some sudden spike loads and also check auto-scaling functionality in these tests.

Also, we should add some endurance test jobs in our CI/CD platform such as Jenkins to parametrize our test execution time and run the endurance tests for long period of time to check the service behavior and stability in the long run.

If we have multiple servers and especially hybrid cloud architecture, it is better to check load-balancing. It can be done at the production level. Here we need to be very careful about test data.

Also, we should use some chaos testing tools such as chaos monkey it randomly terminates virtual machine instances and containers that run inside of your production environment and in this way we can test system resiliency.

Security and Vulnerability Tests and Scans

In the service pipeline after deployment, we should add automated vulnerability and security scans by using some tools such as Zed Attack Proxy, Netsparker, etc. In each new PR, we can scan our service with OWASP security and vulnerability rules. If you use ZAP, you can refer here and here to scan your APIs.

Exploratory Testing

Exploratory testing is an unscripted QA testing technique used to discover unknown issues during and after the software development process. We should do exploratory testing throughout the testing life cycle. We are generally starting exploratory testing after the requirement analysis phase as early as possible. In this way, we can find the risks and problems earlier and this will also help automated test efforts. We use many inputs and outputs of exploratory testing sessions for automation. Especially, for API exploratory testing, we use POSTMAN. For exploratory testing details, you can check here.

exploratory tests

Test Data Approach

One of the big headaches in testing and automation is test data. For test data, I suggest below guidelines:

  • It is better to have some test data generation services and by calling that service methods/endpoints to get freshly created test data for test usage.
  • If there is no chance to create the test data automatically, better to do test data requirement analysis and create the test data manually or get help from the relevant team for test data needs.
  • Try to create a fresh test data before starting a test and delete it after test completion.
  • In Test and Stage environments, I suggest masking the critical test data.

Pre-Release Testing Approach

Before releasing your services to your customers which means the production environment, we should do some checks. We can de-attach one of the server from load-balancer and do critical regression testing which covers most business-critical scenarios and main user journeys on this server. It is also better to get UAT approval before this phase. If the service pipeline is green, SIT tests passed, PT tests passed, UAT is approved, the final checks on production passed then we are OK to proceed further. When the new version of the service consuming in production, we should always monitor it with several tools such as APM, monitoring, and logging tools.

If you have any questions or doubts please do not hesitate to write a comment and get in touch with the swtestacademy community.

Thanks for reading.
-Onur