Getting Started With AWS Device Farm Using Appium and the Page Object Model


Outline

One automation framework we have experience with is Appium, which is an open source test framework for use with native, hybrid and mobile web apps. It drives iOS and Android apps using the WebDriver protocol. A technique used to organize and drive Appium test cases is through the Page Object Model (POM). POM is a design pattern to create Object Repositories for UI elements.

Hello, my name is James Koch, Certified Software Tester and Solutions Architect for Quilmont, an automation firm based out of Myrtle Beach, South Carolina. Quilmont provides organizations with a comprehensive testing practice by fully utilizing industry leading software testing tools for mobile, web, and desktop applications.

Under this model, for each page in the application there should be a corresponding page class similar to the structure below. Using POM reduces the amount of duplicated code and means that if the UI changes, the fix only needs to be applied in one place. POM helps make code more readable, maintainable, and reusable. Through Appium, WebDriver provides a way to map it to a real web page. The PageFactory class provides a convenient way of initializing and mapping the Page Object fields. For more control over identifying elements in the pages and mapping them to our Page Object fields, we use @FindBy annotation.

aws

WebDriver and POM scripting allows code to be executed more universally, but an issue development teams run into frequently is the lack of ability to run test on multiple devices. With thousands of different types of devices on the market, making sure your application is functional and compatible on an array of devices is vital to the success of your product. A powerful tool on the market used to test a wide variety of devices is Amazon Web Service, Device Farm.

Device Farm is an app testing service that enables you to test your Android, iOS, and Fire OS apps on real, physical phones and tablets that are hosted by Amazon Web Services. Device Farm gives testers instant feed back with a comprehensive test report containing high-level results, low-level logs, pixel-to-pixel screenshots, and performance data as tests are completed. Device Farm allows you to upload your own tests or use built-in, script-free compatibility smoke tests. Because testing is automatically performed in parallel, tests on multiple devices begin in minutes.

Getting Started

Prerequisites

  • The example we will be working with comes from the AWS Lab’s Github sample application tests, follow the link to download.
  • The IDE we will be working in will be IntelliJ IDEA 14.1.
  • In order to run the test locally you need to set up Android SDK and Appium to the capabilities in your script. For my instance, when testing locally, I will be using an Nexus 7 AVD with KitKat (API level 4.4) along with Appium to monitor our script.

IntelliJ

Once you add your sample tests into the IDE, let’s diagnose how AWS set this file up knowing what we already know about POM. The image above shows the file structure which is tightly organized to track the flow of our script. All tests are going to start off on the Test Base Page, which is an abstract base for all  of the Android tests within the package. It is responsible for setting up the Appium Driver. You will notice all tests have a setup, execution, and teardown suites. In @BeforeSuite we are setting up our appium client in order to connect to Device Farm’s appium server. Device Farm creates custom settings and capabilities at the server level, so setting up desired capabilities is not necessary unless one is running a test locally.

Running Tests Locally

This is similar to my last blog post, only this time Appium will be launching the device and we are using POM. If you are having trouble setting up your virtual device,  refer to my last article Writing a Simple Test Case for Appium Using an Android Virtual Device and Eclipse. The following are my capabilities in Appium and on my AVD:

Android:

android

Appium (I used a basic APK, but you will specify it to be your application):

ios

Corresponding code on Test Base Page:

Once everything is set up, open the test you wish to execute. For our intents and purposes we will be demonstrating the Login example. Launch the Appium server. Then in IntelliJ, right click and run the test! You will observe that Appium launched the device and the execution. Once the test finishes, it will proceed with the teardown suite, thus giving user TestNG results right in IntelliJ.

Running Tests in the AWS Device Farm Platform

Now that we have an understanding of how Appium communicates with POM, we can run all of our tests simultaneously in Device Farm on multiple devices! The example link from Github will explain how to set up your POM file if you are building from scratch, but if you are just working off their example, it is already formatted correctly. Device Farm will ignore your local capabilities when imported, but depending on how you set it up, it can effect your execution. For me it did not. Follow these steps to package your tests up correctly to be brought into the platform:

Step 1: Go into your Maven Appium Directory

Go into your Appium Maven project directory in the terminal or command prompt.

Step 2: Package the Test Content

Run the following Maven command to package the test content:

mvn clean package -DskipTests=true

Step 3: Locate the zip-with-dependencies.zip file

Once the Maven command above is finished it will produce a “zip-with-dependencies.zip” file in your target folder. You will upload this file when creating a run on AWS Device Farm.

aws

Now we are ready to execute in Device Farm. Log on through https://console.aws.amazon.com/devicefarm, create a new project, and name it:

aws

Select your project and continue to add your APK file:

aws

aws

On the next page you will be prompt to configure and add your test. This will be the zip dependency file we created using Maven:

aws

Now we execute all of our tests simultaneously on the devices that users specify the tests for. Once finished, Device Farm will push out our results for all the tests breaking down the run by setup, execution, and teardown:

device farm

These are the results of a specific device:

device farm

Note the different tabs for results. AWS allows user to print screenshots in their code, record logical reports, and documents the performance of the application.

Utilizing Smoke Tests in Device Farm

Device Farm also has the ability to run smoke tests which are application specific just by uploading your APK. Smoke tests are preliminary tests to reveal simple failures severe enough to reject a prospective software release. A subset of test cases that cover the most important functionality of a component or system is selected and run to ascertain if crucial functions of a program correctly work. This can be useful for firms with no automation as the random data can find flaws in the code unbeknownst to manual testers.

For more information on creating a test run in Device Farm, visit http://docs.aws.amazon.com/devicefarm/latest/developerguide/how-to-create-test-run.html.

Conclusion

 In Device Farm, screenshots and extensive detailing are provided in a tight, organized fashion for developer review and to efficiently diagnose correct and incorrect behaviors in the software. The ability to run all of our tests on multiple devices at the same time decrease wasted test time exponentially. No longer do teams have to split up test plans, execute on one device, then switch devices and run through the whole regression again! With thousands of different devices on the market AWS Device Farm helps teams get through releases faster. The reason I bring POM up is to demonstrate how if you need to do a code change, it will only be done in one place correcting the behavior at all levels. In my years of experience in software testing, this type of process and platform is innovative and trend setting. Using this technique is not only cost effective, but will ensure your application has minimal flaws and universally device applicable. Thank you for reading and if you have any questions you can contact me through our company website www.quilmont.com.

Visit Quilmont Website

qm logo7_email

Quilmont was established in 2007 by our owner and CEO Patrick Quilter. Patrick has over 15 years of experience in software development life cycle & test automation architecture. Quilmont owns a patented product called Test Case Manager, an automation framework that allows firms to get more out of their scripting tools. Quilmont was founded in Washington D.C., being so, we are still connected with several prime government contractors in the area that reach out to us when they need help; we are currently registered in the SAM database. Our partnerships include Polarion, HP, and Keynote Studios. We provide organizations with a comprehensive testing practice by fully utilizing industry leading software testing tools.

Quilmont has had projects of all different kinds and scope; from working with startups, government contracts, and even large fortune 500 companies like Berkshire Hathaway/Geico. Unlike the other products and consulting services on the market, Quilmont provides its clients with the most cost effective and best personalized solution to fit its automation needs. We provide customer service and support, and offer training. At Quilmont it is our mission to form strategic relationships with other enthusiastic automated testing organizations. We utilize next generation technologies to virtually distribute our solutions globally while providing our workforce with opportunities to enhance their IT skills.

Visit Quilmont Website

javafx

Database Operations in JavaFX

By Onur Baskirt / Apr 1, 2016 / 56 Comments
Before started this section, please check the first article and learn How to Start JAVAFX! http://www.swtestacademy.com/getting-started-with-javafx/ At first, part of JavaFX tutorial series, we created a sample JavaFX project, designed the draft version of the UI and set up an...
rest assured

REST API Testing with Rest Assured

By Onur Baskirt / Mar 8, 2016 / 32 Comments
Outline In this post, I will explain what is API and API testing, what is the difference between SOAP and REST services, and how to test REST APIs with Rest Assured Library. What is API? API stands for Application Programming...
extentreports

How to Write Effective CSS Locators

By Onur Baskirt / Oct 1, 2017 / 0 Comments
Hi all, in this tutorial, I will describe you how to write effective CSS locators to interrogate web elements for your automation projects. As a rule of thumb, your interrogation strategy should be in below order: First try to use...
extentreports

Selenium-11: Execute JavaScript with JavascriptExecutor

By Onur Baskirt / Jan 27, 2016 / 15 Comments
Outline Sometimes we cannot handle some conditions or problems with Webdriver, web controls don't react well against selenium commands. In this kind of situations, we use Javascript. It is useful for custom synchronizations, hide or show the web elements, change...
javafx

Getting Started with JavaFX

By Onur Baskirt / Mar 25, 2016 / 0 Comments
When I started to work in my current position, one of my task is to do manual operations for campaign products  every week. After the second week, I thought that I have to automate this task using a GUI based...
extentreports

How to Write Smart XPath Locators

By Onur Baskirt / Sep 24, 2017 / 4 Comments
Hi all, in this tutorial, I will describe you how to write smart and non-brittle XPath locators. When we write our test scripts, we generally prefer to use id, name, class, etc. these kinds of locators. However, sometimes we could not...
extentreports

Selenium-1: Quick Start to Automation with Selenium WebDriver & JAVA & JUnit & Maven & IntelliJ

By Onur Baskirt / Sep 8, 2015 / 26 Comments
Outline Selenium Webdriver is the most popular open source web test automation framework across wide range of browsers and platforms. In this tutorial you will learn how to do web test automation with Selenium Webdriver and the related tools. Audience...
page object model

Page Object Model with C#

By Ege Aksoz / Jun 18, 2017 / 8 Comments
In the previous tutorial, we’ve taken the initial steps and entered the world of automated testing. We also wrote our first automated test. From this point on, since we are not just going to write one test, we need to...
extentreports

How to Select a Date From DatePicker Using Selenium

By Onur Baskirt / Aug 13, 2016 / 6 Comments
When you need to automate a airway, hotel, or similar websites you need to deal with Datepickers and some times it is a little bit cumbersome to select a specific date on the Datepicker or calendar.  In this post, I...
extentreports

Selenium Webdriver Performance testing with Jmeter and Selenium Grid

By Ozgur Kaya / Aug 9, 2016 / 2 Comments
In this post, we will complete Selenium Webdriver Performance testing scenario using Jmeter and Selenium Grid. 1- Install Java 7 or later If necessary https://java.com/tr/download/ 2- Download latest Jmeter version 3.0 or higher. http://jmeter.apache.org/download_jmeter.cgi 3- Download Jmeter PluginsManager JAR file and...
By | 2017-01-13T21:33:43+00:00 March 21st, 2016|Appium, AWS, Mobile Automation, Test Automation|16 Comments

About the Author:

James Koch
Hello! My name is James Koch, CTFL, Solutions Architect for Quilmont, an advanced automation firm based out of Myrtle Beach, South Carolina. I currently work remotely, in charge of our Atlanta territory as our company is growing all across the south east U.S. I am a certified software tester through the International Software Test Qualification Board(ISTQB) and am an active member of the ASTQB. I started as a software test engineer with Sirius XM Satellite Radio and worked on everything from their mobile application, to regression testing on CE and OEM devices. That is where my passion for testing software and QA began. With a high focus in Automated Testing, Total Quality Management, and Software Quality Assurance, I strive to understand, diagnose, and correct behaviors by formulating solutions in software and other day-to-day business operations to reach optimal performance. I apply my operations background to software development and work well with Agile and DevOps practices. I understand philosophies for automating testing by being well versed in the most popular frameworks, the latest being Appium. I apply automation frameworks to develop, execute, organize, and maintain test plans to better assist clients getting through testing in an effective efficient manner. I've developed Web Services for clients to easily interact with their in-house systems. I possess the ability to coordinate multi-organizational structures such as OEM Applications Engineering, Contract Manufacturing and Test Validation Sites. I posses analytic skills in planning, process, and creating efficiency. I have knowledge of Hardware circuitry design in the areas of power supplies and microprocessors. My hobbies include lacrosse, fishing, hiking and camping with my dog and wonderful fiancé. My relationship with Software Test Academy was reciprocated in February 2016 and it is a pleasure to contribute. For Quilmont and myself, it is our mission to form strategic relationships with other enthusiastic automated testing organizations. We utilize next generation technologies to virtually distribute our solutions globally while providing our workforce with opportunities to enhance their IT skills.

16 Comments

  1. Aniket March 21, 2016 at 4:22 pm - Reply

    Is it possible to see the tests actually running on devices from device farm?

    • James Koch
      James Koch March 21, 2016 at 7:08 pm - Reply

      Hello Aniket, thanks for your question. AWS Device Farm supports frameworks like Robotium that have record and playback scripting tools. If you wish to use TestNG or JUnit You can insert language into your script that captures screen shots:

      public boolean takeScreenshot(final String name) {
      String screenshotDirectory = System.getProperty("appium.screenshots.dir", System.getProperty("java.io.tmpdir", ""));
      File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
      return screenshot.renameTo(new File(screenshotDirectory, String.format("%s.png", name)));
      }

      This is an important feature for reporting. You can put this method on your Abstract BasePage or Abstract TestBase.

    • James Koch
      James Koch April 18, 2016 at 5:43 pm - Reply

      AWS Device Farm now captures video for Android automated tests. With this functionality you can view a video of the device’s screen and your app as they appeared while your tests ran. No configuration is necessary and video capture is automatic.

      Once a device has finished execution, you can view videos by navigating to the “Files” tab on the device, suite, or test level. If you’re using the API, you can access the videos by using ListArtifacts. Currently, videos are available for each test in a separate MPEG-4 (.mp4) file.

  2. Mallikarjunareddy Duvoori March 21, 2016 at 8:07 pm - Reply

    Hello James,

    I have few doubts
    1. Currently I have a TestNG project, can u run this project in Device farm ?
    2. Actually where are this Physical device located ? In cloude ? Can we see the execution on those devices ?

    Thank you for the wonderful work on Appium. Keep doing this for us.

    • James Koch
      James Koch March 21, 2016 at 8:32 pm - Reply

      Yes you should be able to run your TestNG project in Device Farm. The physical devices are not located in the cloud, they are hosted on site @ AWS. Please clarify your doubts.

    • James Koch
      James Koch April 18, 2016 at 5:43 pm - Reply

      AWS Device Farm now captures video for Android automated tests. With this functionality you can view a video of the device’s screen and your app as they appeared while your tests ran. No configuration is necessary and video capture is automatic.

      Once a device has finished execution, you can view videos by navigating to the “Files” tab on the device, suite, or test level. If you’re using the API, you can access the videos by using ListArtifacts. Currently, videos are available for each test in a separate MPEG-4 (.mp4) file.

  3. Srikanth March 21, 2016 at 8:44 pm - Reply

    Hi james,

    In sauce labs devices we actually see the device video where our scripts are executing and its easy to analyse the test run easly. Is AWS also have any feature like that?

    • James Koch
      James Koch April 18, 2016 at 5:43 pm - Reply

      AWS Device Farm now captures video for Android automated tests. With this functionality you can view a video of the device’s screen and your app as they appeared while your tests ran. No configuration is necessary and video capture is automatic.

      Once a device has finished execution, you can view videos by navigating to the “Files” tab on the device, suite, or test level. If you’re using the API, you can access the videos by using ListArtifacts. Currently, videos are available for each test in a separate MPEG-4 (.mp4) file.

  4. Mallikarjunareddy Duvoori March 22, 2016 at 4:52 am - Reply

    Hi James,

    Can I use Eclipse instead of IntelliJ IDEA 14.1. For this execution ?

    • James Koch
      James Koch March 22, 2016 at 5:18 am - Reply

      Yes, you most certainly can. I find Intelij to be easier to create, maintain, and execute Maven projects however. It is just my opinion that the shortcuts, library accessibility, and directory organization is better with Intelij. I still use Eclipse to build web services and that is the beauty of open source. You get to build your automation framework however it meets your product and project’s needs. For example, you could use Appium with most all popular programming language’s client libraries.

  5. prasanth April 28, 2016 at 7:55 am - Reply

    Hi James,
    I am trying to write test case for hybrid app which is done in framework7….Is it possible to write test cases using class name for particular button or field without knowing element id….in appium java testng.If yes…can u post the example…

    Thanks in advance… 🙂

    • James Koch
      James Koch April 28, 2016 at 5:06 pm - Reply

      Hello! Yes it is possible. You can use className, xpath, id, or name as well. Using className comes in handy when you have multiple text boxes to fill out for example.

      public void LogIn() {

      List textFieldsList = driver.findElementsByClassName(“android.widget.EditText”);
      textFieldsList.get(0).sendKeys(“username”);
      textFieldsList.get(1).sendKeys(“password”);
      logInButton.click();
      driver.findElement(By.name(“OK”)).click();
      }

  6. prasanth May 4, 2016 at 7:37 am - Reply

    Hi James

    How to overcome this error…

    unknown error: Element is not clickable at point (525, 878). Other element would receive the click: (Session info: webview=37.0.0.0) (Driver info: chromedriver=2.18.343837 (52eb4041461e46a6b73308ebb19e85787ced4281),platform=Linux 3.13.0-53-generic x86_64) (WARNING: The server did not provide any stacktrace information) Command duration or timeout: 562 milliseconds Build info: version: ‘2.46.0’, revision: ‘61506a4624b13675f24581e453592342b7485d71’, time: ‘2015-06-04 10:22:50’ System info: host: ‘ip-172-31-13-65’, ip: ‘172.31.13.65’, os.name: ‘Linux’, os.arch: ‘amd64’, os.version: ‘3.13.0-53-generic’, java.version: ‘1.8.0_65’ Driver info: io.appium.java_client.android.AndroidDriver Capabilities [{networkConnectionEnabled=true, desired={}, platformVersion=5.0.2, warnings={}, webStorageEnabled=false, locationContextEnabled=false, browserName=Android, takesScreenshot=true, javascriptEnabled=true, databaseEnabled=false, deviceName=0a4d2378, platform=LINUX}] Session ID: 7da2698c-a684-4d70-ad93-99ba0d958320

  7. naga lakshmi kokkiligadda September 15, 2016 at 8:26 pm - Reply

    is it possible to test Hybrid Apps on AWS?

  8. NK September 21, 2016 at 10:04 am - Reply

    Hi,
    I am using Amazon Device farm -Appium Java TestNG for testing of my android application. I am new to ADF. I am uploading my app and dependency zip, after completing the test, I am getting attached Unique failure error:

    I have tried two type of code to identify elements :

    Using Index of elements
    Resource id and Content description of elements Following is my code:
    driver.findElement(By.xpath(“//android.widget.EditText[contains(@resource-id,’username’)]”)).sendKeys(“testuser11@mailinator.com”);
    driver.findElement(By.xpath(“//android.widget.EditText[contains(@resource-id,’password’)]”)).sendKeys(“Commonapp1!”);

    driver.findElement(By.xpath(“//android.widget.Button[contains(@content-desc,’Log In’)]”)).click();;

    Please resolve my issue. Have I missed something?

  9. Prashant July 25, 2017 at 12:05 pm - Reply

    Hi,

    I have created Appium Test Project with Cucumber Gherkin. I have written feature files for testing the Android Apk which internally use appium for testing.

    But when I am trying to upload it to the AWS device farm for testing, It is not accepting my test project ( build-with-dependencies.zip ).

    Project structure is similar to:
    +—src
    | +—main
    | | \—assembly
    | | +——— zip.xml
    | \—test
    | +—java
    | | \—com
    | | \—myapp
    | | \—testproject
    | | \—bdd
    | | +—pages
    | | | +—android
    | | | +—ios
    | | +—steps
    | \—resources
    | +—features
    +—pom.xml

    As I have written feature files, in which section should I upload my test zip ( under “Create a new Run” – “Configure a Test” section )

    Thanks,
    Prashant

Leave A Comment