Outline

One automation framework we have experience with is Appium, which is an open source test framework for use with native, hybrid and mobile web apps. It drives iOS and Android apps using the WebDriver protocol. A technique used to organize and drive Appium test cases is through the Page Object Model (POM). POM is a design pattern to create Object Repositories for UI elements.

Hello, my name is James Koch, Certified Software Tester and Solutions Architect for Quilmont, an automation firm based out of Myrtle Beach, South Carolina. Quilmont provides organizations with a comprehensive testing practice by fully utilizing industry leading software testing tools for mobile, web, and desktop applications.

Under this model, for each page in the application there should be a corresponding page class similar to the structure below. Using POM reduces the amount of duplicated code and means that if the UI changes, the fix only needs to be applied in one place. POM helps make code more readable, maintainable, and reusable. Through Appium, WebDriver provides a way to map it to a real web page. The PageFactory class provides a convenient way of initializing and mapping the Page Object fields. For more control over identifying elements in the pages and mapping them to our Page Object fields, we use @FindBy annotation.

aws

WebDriver and POM scripting allows code to be executed more universally, but an issue development teams run into frequently is the lack of ability to run test on multiple devices. With thousands of different types of devices on the market, making sure your application is functional and compatible on an array of devices is vital to the success of your product. A powerful tool on the market used to test a wide variety of devices is Amazon Web Service, Device Farm.

Device Farm is an app testing service that enables you to test your Android, iOS, and Fire OS apps on real, physical phones and tablets that are hosted by Amazon Web Services. Device Farm gives testers instant feed back with a comprehensive test report containing high-level results, low-level logs, pixel-to-pixel screenshots, and performance data as tests are completed. Device Farm allows you to upload your own tests or use built-in, script-free compatibility smoke tests. Because testing is automatically performed in parallel, tests on multiple devices begin in minutes.

Getting Started

Prerequisites

  • The example we will be working with comes from the AWS Lab’s Github sample application tests, follow the link to download.
  • The IDE we will be working in will be IntelliJ IDEA 14.1.
  • In order to run the test locally you need to set up Android SDK and Appium to the capabilities in your script. For my instance, when testing locally, I will be using an Nexus 7 AVD with KitKat (API level 4.4) along with Appium to monitor our script.

IntelliJ

Once you add your sample tests into the IDE, let’s diagnose how AWS set this file up knowing what we already know about POM. The image above shows the file structure which is tightly organized to track the flow of our script. All tests are going to start off on the Test Base Page, which is an abstract base for all  of the Android tests within the package. It is responsible for setting up the Appium Driver. You will notice all tests have a setup, execution, and teardown suites. In @BeforeSuite we are setting up our appium client in order to connect to Device Farm’s appium server. Device Farm creates custom settings and capabilities at the server level, so setting up desired capabilities is not necessary unless one is running a test locally.

Running Tests Locally

This is similar to my last blog post, only this time Appium will be launching the device and we are using POM. If you are having trouble setting up your virtual device,  refer to my last article Writing a Simple Test Case for Appium Using an Android Virtual Device and Eclipse. The following are my capabilities in Appium and on my AVD:

Android:

android

Appium (I used a basic APK, but you will specify it to be your application):

ios

Corresponding code on Test Base Page:

Once everything is set up, open the test you wish to execute. For our intents and purposes we will be demonstrating the Login example. Launch the Appium server. Then in IntelliJ, right click and run the test! You will observe that Appium launched the device and the execution. Once the test finishes, it will proceed with the teardown suite, thus giving user TestNG results right in IntelliJ.

Running Tests in the AWS Device Farm Platform

Now that we have an understanding of how Appium communicates with POM, we can run all of our tests simultaneously in Device Farm on multiple devices! The example link from Github will explain how to set up your POM file if you are building from scratch, but if you are just working off their example, it is already formatted correctly. Device Farm will ignore your local capabilities when imported, but depending on how you set it up, it can effect your execution. For me it did not. Follow these steps to package your tests up correctly to be brought into the platform:

Step 1: Go into your Maven Appium Directory

Go into your Appium Maven project directory in the terminal or command prompt.

Step 2: Package the Test Content

Run the following Maven command to package the test content:

mvn clean package -DskipTests=true

Step 3: Locate the zip-with-dependencies.zip file

Once the Maven command above is finished it will produce a “zip-with-dependencies.zip” file in your target folder. You will upload this file when creating a run on AWS Device Farm.

aws

Now we are ready to execute in Device Farm. Log on through https://console.aws.amazon.com/devicefarm, create a new project, and name it:

aws

Select your project and continue to add your APK file:

aws

aws

On the next page you will be prompt to configure and add your test. This will be the zip dependency file we created using Maven:

aws

Now we execute all of our tests simultaneously on the devices that users specify the tests for. Once finished, Device Farm will push out our results for all the tests breaking down the run by setup, execution, and teardown:

device farm

These are the results of a specific device:

device farm

Note the different tabs for results. AWS allows user to print screenshots in their code, record logical reports, and documents the performance of the application.

Utilizing Smoke Tests in Device Farm

Device Farm also has the ability to run smoke tests which are application specific just by uploading your APK. Smoke tests are preliminary tests to reveal simple failures severe enough to reject a prospective software release. A subset of test cases that cover the most important functionality of a component or system is selected and run to ascertain if crucial functions of a program correctly work. This can be useful for firms with no automation as the random data can find flaws in the code unbeknownst to manual testers.

For more information on creating a test run in Device Farm, visit http://docs.aws.amazon.com/devicefarm/latest/developerguide/how-to-create-test-run.html.

Conclusion

 In Device Farm, screenshots and extensive detailing are provided in a tight, organized fashion for developer review and to efficiently diagnose correct and incorrect behaviors in the software. The ability to run all of our tests on multiple devices at the same time decrease wasted test time exponentially. No longer do teams have to split up test plans, execute on one device, then switch devices and run through the whole regression again! With thousands of different devices on the market AWS Device Farm helps teams get through releases faster. The reason I bring POM up is to demonstrate how if you need to do a code change, it will only be done in one place correcting the behavior at all levels. In my years of experience in software testing, this type of process and platform is innovative and trend setting. Using this technique is not only cost effective, but will ensure your application has minimal flaws and universally device applicable. Thank you for reading and if you have any questions you can contact me through our company website www.quilmont.com.

Visit Quilmont Website

qm logo7_email

Quilmont was established in 2007 by our owner and CEO Patrick Quilter. Patrick has over 15 years of experience in software development life cycle & test automation architecture. Quilmont owns a patented product called Test Case Manager, an automation framework that allows firms to get more out of their scripting tools. Quilmont was founded in Washington D.C., being so, we are still connected with several prime government contractors in the area that reach out to us when they need help; we are currently registered in the SAM database. Our partnerships include Polarion, HP, and Keynote Studios. We provide organizations with a comprehensive testing practice by fully utilizing industry leading software testing tools.

Quilmont has had projects of all different kinds and scope; from working with startups, government contracts, and even large fortune 500 companies like Berkshire Hathaway/Geico. Unlike the other products and consulting services on the market, Quilmont provides its clients with the most cost effective and best personalized solution to fit its automation needs. We provide customer service and support, and offer training. At Quilmont it is our mission to form strategic relationships with other enthusiastic automated testing organizations. We utilize next generation technologies to virtually distribute our solutions globally while providing our workforce with opportunities to enhance their IT skills.

Visit Quilmont Website